Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Context: For 2.5 years, I wrote Java-based desktop software at software company A (the supplier). The software also has a server and a web component. The software was custom-built for company B (the client). After a short time not being involved, I left company A and joined company B.

The software is being developed in a scope-to-budget kind of way, where business analysis is done by the supplier. There is no clear list of requirements, and the relationship is very open and trusting.

The problem: A new module of the software is currently in its client validation phase. The client does free-testing, but is unable to provide a conclusive overview of

  • what has been tested; what specific test steps were used?
    • Currently, it is very difficult to write a good bug report. "I cliked around and suddenly: error message!"
  • what are the zones which were not tested (due to blocking bugs), and what did we plan to test?
  • did we understand the functional scope correctly?

As the software was developed scope-to-budget, we do not even have a good overview of all features that have been developed. Currently, there is a constant back-and-forth between client and supplier. The client does free-testing, discovers a 10-odd number of bugs, some of which are blocking. The supplier then delivers a new version, after which the client does some more free-testing and discovers more bugs (some of those could have also been found in the old version).

The solution: I want to solve this issue by introducing Gherkin-style tests, which may or may not be automated. The goal is two-fold:

  • Force us (the client) to describe how we think functionality should work. This ensures we are indeed reporting bugs, and not just misunderstanding the functional scope.
  • Make it clear how well we cover the full software

My question: Are there any good tools to manage Gherkin-syntax (Given-When-Then) test cases in manual client-side validation? Something which can manage Features, and all of the different scenario's? And ideally, can easily record the result of a manual testing run ?

share|improve this question
1  
Not a direct answer, so I'll use a comment. It sounds like the reality is that you need to hire an experienced QA lead/manager to come in and help address the problem. By your description, I would guess there would be benefits beyond just solving this issue that you could get. Without a primary owner and responsible party, doing a grassroots effort may be difficult. On the plus side, the fact that you as a developer are interested and participating in finding solutions is a very good thing, definitely keep that up. –  Sam Woods Mar 13 at 20:08
    
I fully agree: hiring a professional to do software validation is always a good thing. Until then, a grassroots effort will have to do (it will be better than the current free-testing grey zone) –  parasietje Mar 17 at 8:55

1 Answer 1

Great idea a manual test runner and reporter for Cucumber.

Seems someone has similar ideas and put it online:

If this doesn't work out of the box, it should not be to hard to create something similar from scratch. You can catch each Given/When/Then and let it ask a question on the console to verify "did it pass?" yourself, with a little bit of coding effort.

If you need to cut up the test runs, because you have multiple testers in a single test run. You can let the runner output the results in JSON instead of HTML and later combine all the JSON files to create a single report with cucumber-reporting. I have successfully used this technique to report on parallel runs in the past.

share|improve this answer
    
Thanks for the link to cucumbler. Unfortunately, no code has been created to fulfill the requirements (it is only a sort of wish-list). So I guess it has become time to make something myself... –  parasietje Mar 13 at 8:34
    
Your right, updated my answer. The blog post has a concept implementation which you could use. Not very advanced. Still I really like the idea :) Goodluck –  Niels van Reijmersdal Mar 13 at 9:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.