5

Context: For 2.5 years, I wrote Java-based desktop software at software company A (the supplier). The software also has a server and a web component. The software was custom-built for company B (the client). After a short time not being involved, I left company A and joined company B.

The software is being developed in a scope-to-budget kind of way, where business analysis is done by the supplier. There is no clear list of requirements, and the relationship is very open and trusting.

The problem: A new module of the software is currently in its client validation phase. The client does free-testing, but is unable to provide a conclusive overview of

  • what has been tested; what specific test steps were used?
    • Currently, it is very difficult to write a good bug report. "I cliked around and suddenly: error message!"
  • what are the zones which were not tested (due to blocking bugs), and what did we plan to test?
  • did we understand the functional scope correctly?

As the software was developed scope-to-budget, we do not even have a good overview of all features that have been developed. Currently, there is a constant back-and-forth between client and supplier. The client does free-testing, discovers a 10-odd number of bugs, some of which are blocking. The supplier then delivers a new version, after which the client does some more free-testing and discovers more bugs (some of those could have also been found in the old version).

The solution: I want to solve this issue by introducing Gherkin-style tests, which may or may not be automated. The goal is two-fold:

  • Force us (the client) to describe how we think functionality should work. This ensures we are indeed reporting bugs, and not just misunderstanding the functional scope.
  • Make it clear how well we cover the full software

My question: Are there any good tools to manage Gherkin-syntax (Given-When-Then) test cases in manual client-side validation? Something which can manage Features, and all of the different scenario's? And ideally, can easily record the result of a manual testing run ?

Improve this question
2
  • 1
    Not a direct answer, so I'll use a comment. It sounds like the reality is that you need to hire an experienced QA lead/manager to come in and help address the problem. By your description, I would guess there would be benefits beyond just solving this issue that you could get. Without a primary owner and responsible party, doing a grassroots effort may be difficult. On the plus side, the fact that you as a developer are interested and participating in finding solutions is a very good thing, definitely keep that up.
    – Sam Woods
    Commented Mar 13, 2014 at 20:08
  • I fully agree: hiring a professional to do software validation is always a good thing. Until then, a grassroots effort will have to do (it will be better than the current free-testing grey zone)
    – parasietje
    Commented Mar 17, 2014 at 8:55

3 Answers 3

Reset to default
1

Great idea a manual test runner and reporter for Cucumber.

Seems someone has similar ideas and put it online:

  • Cucumbumler, see https://github.com/TimWalker/cucumbumbler/wiki and http:// technicaldebt.com /manual-cucumber-tests/ [as of 9/9/2019 the site is infested with a trojan. If/when the site is cleaned, this edit can be reverted] - for a concept

If this doesn't work out of the box, it should not be to hard to create something similar from scratch. You can catch each Given/When/Then and let it ask a question on the console to verify "did it pass?" yourself, with a little bit of coding effort.

If you need to cut up the test runs, because you have multiple testers in a single test run. You can let the runner output the results in JSON instead of HTML and later combine all the JSON files to create a single report with cucumber-reporting. I have successfully used this technique to report on parallel runs in the past.

Improve this answer
3
  • Thanks for the link to cucumbler. Unfortunately, no code has been created to fulfill the requirements (it is only a sort of wish-list). So I guess it has become time to make something myself...
    – parasietje
    Commented Mar 13, 2014 at 8:34
  • Your right, updated my answer. The blog post has a concept implementation which you could use. Not very advanced. Still I really like the idea :) Goodluck
    – Niels van Reijmersdal
    Commented Mar 13, 2014 at 9:54
  • Be very careful with that link to technicaldebt.com. Sophos just picked it up as containing a trojan (JS/RefC-Gen).
    – steinybot
    Commented Sep 9, 2019 at 2:25
0

I came across a similar issue in our organisation. We wanted to practice BDD but many of our projects didn't lend themselves well to automation - either we were limited by time or by technical expertise.

We built a simple RubyGem called Crudecumber to allow testers to step through the tests manually and pass, fail or skip each step using the keyboard in the command line. An HTML report is automatically generated and opened at the end of the test run.

At the moment there are limitations in that it won't work in Windows or with Cucumber 2.0. It is open source however so this may be fixed in the future. I hope this helps.

Improve this answer
0

There is an easy alternative approach. There is a SaaS tool called TestRigor which will automatically detect (in particular) functionality introduced in the new version.

Improve this answer
1
  • Just a reminder that if you have a personal affiliation with the a product in an answer, such affiliation must be explicitly disclosed in every answer that suggests it, just for transparency's sake. Read a brief summary of our policy here. Admittedly, this is not a great question to begin with because it doesn't have an "answer" - someone is just asking for good tools, which is not particularly answerable (although this OP did much more than most who are looking for software recomendations)
    – corsiKa ♦
    Commented Jan 24, 2021 at 2:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.