0

We write our manual test protocols in an in-house XML format, and have a simple tool for executing them electronically (vs. paper and pencil) and saving the results.

This has worked well, but we're struggling with the test management side of things. Currently we have scripts that:

  • Show which requirements are traced to which (already written) tests
  • Generate a summary of tests that need to be executed
  • Summarize the pass/fail results for tests that have been executed for a given test round

However, these are rather ad-hoc, and we don't have a good way of managing things or answering questions such as:

  • Test authoring
    • Have we written a test for feature X yet?
    • How many planned test cases do we have left to write?
  • Test results across test rounds
    • Does this particular test have a history of failing?
    • Are we ready to re-execute the tests that failed during the last round?
    • Which tests should we run in the next round?

We've tried using JIRA, but since there's no connection between our actual protocols and their associated JIRA items, there's a lot of manual work to keep JIRA in sync, and it inevitably falls behind.

From what I've seen, a lot of test management tools require you to write and run your manual tests in their tool. We would like to keep our in-house format, or at the very least a text-based format (as in, the source files should be text-based; exporting to a text-based format doesn't count) that we can keep in source control and easily grep, edit, etc. along with our code.

Is anyone aware of any test management tools that support integrating existing manual tests without migrating them to the tool itself?

Improve this question

1 Answer 1

Reset to default
1

Your company has already invested in a custom XML format and a tool to execute these XML scenarios automatically. I would suggest then to write your own in-house management tool as well. It doesn't have to be complex, and will give you exactly what you need.

  • Reading key fields from XML scenarios is the main advantage
  • Write test results to a database for your metrics
  • The tool can be as simple as a list of XML files, and a grid showing the results and details of the selected XML.
  • Same for the features/requirements which you'll have to parse from somewhere; the feature-to-XML mapping should just be another XML field.
Improve this answer
1
  • Yeah, that may very well be where we end up. We're open to switching formats, but I don't think our team will be willing to give up the advantages of a text-based format that can be stored in git, easily batch updated with sed, grep, etc. so part of this was checking if tools exist that keep their tests in such a format.
    – c32hedge
    Jul 11, 2017 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.