2

I am looking for advice/best practices on managing tests,results and requirements, the scenario I have in mind is one where there are a set of requirements, for which code,tests are generated, for these tests the appropriate results are stored in a configuration management system such as IBM's CM Synergy, what then happens is that the requirements and code may get updated, subsequently, and a whole new set of tests would be generated, these updates may happen every 10 weeks, I am looking to get insights on the best strategies/practices in managing all these tests(hundreds),and their results against various baseline requirements.

How can I track what tests, relate to what code updates,and in turn what relates to which set of baseline requirements

Improve this question

4 Answers 4

Reset to default
3

The short answer (as others have said) is by any method that works for you.

The longer answer is that there are no best practices, only practices that are better or worse in any situation.

Some things you can consider:

  • Use version control and maintain branches or labels that match the versioning of the code you're testing. One method I've used successfully with Mercurial version control was to have a branch of test code per major version of source code. It's not the only method: you can match the test code to each application build if needed.
  • Use labels to mark the addition of new requirements in your test code. This method means that when your test code works correctly with a new requirement, you add a label to that branch that's flagged with the application build that covers the requirement and the requirement itself, such as "Application 4.5.0 New feature 3.6"
  • Include the requirement in the reporting/naming of the tests, such as "Test 5.1: New feature 3.6: Add user" (this requires some manual input in the form of code comments that can be scraped - but your developers should be doing this anyway).
  • Include a reference to the work item that covers the new/updated requirement. This method works well with integrated application life cycle management systems where each check in is linked to one or more work items, and each test is also linked to one or more work items (it can be helpful to have a few "bucket" items for non-specific development in these cases). If, for instance, you're working with the Microsoft Team Foundation Server system, each code check in is linked to a TFS item, and your requirements are maintained in TFS items. Your test code is also linked to TFS items, so the traceability is built in, and the test results automatically link to TFS items.

There are as many ways to manage this as you can imagine: what matters is that it works for you and your team. I would suggest that given the number of tests involved you look at an automated method of managing everything (possibly IBM Rational since it's likely to have built-in integration to CM Synergy and if you have Synergy you may already have the Rational on-site).

It's possible to build the integration yourself, or to take a free tool such as TestLink (which does have the capacity to connect requirements to tests) and use its API to link code to tests and requirements. This is more time-consuming than setting up tools with pre-built integration, but still feasible.

Improve this answer
8

I totally agree with milinpatel17. There is no specific answer for this. It is up to an individual or a team to decide which format will get their job done in better way for them.

Improve this answer
3

In testing, there is no such thing as best practice. There is also a good/better/appropriate practice.

To manage tests and link them to the requirements, you can do it in several different ways.

  1. Mind mapping your test ideas/cases against requirements. Use xmind, freeplane, etc.

  2. Make metrics in excel spreadsheets.

  3. Use test management tools like TestLink.

  4. Make a chart in your own format.

  5. ...

There are so many ways to do this. Only you can decide which one works out well for you. Be creative. Think out of the box and come up with something really new. Try it out. And if you succeed, write a blog so the world can learn from it.

Improve this answer
0

I'll give you advice, not really full blown best practices. Hope this helps.

Managing test cases in a project that has regularly changing requirements is a place where test management ( or ALM ) tools can help a lot. I have used HP ALM and Meliora Testlab in this kind of situations, but probably the IBM support similar features. Any tool with good linking and visibility features between requirements and test cases should work.

  • Maintain requirements up to date in your tool. Test designers need to know the situation as well as business and developers.
  • If you have a big project, maintain your requirements and test cases in a tree structure. You will find your relevant content much more easily than from a long flat list.
  • Maintain the links between the test cases and requirements. Now, once a requirements changes you know exactly what test cases you need to execute.
    • Consider using lightweight test cases. Responsible tester can get great results with open test cases. When a tester can read the latest requirement during the testing, the test case does not need to be so detailed.
    • Consider using tags to help find relevant requirements / test cases / defects
    • Link test cases to defects ( well, tools tend to do this automatically ). Anyway you need to re-test the test when it is fixed.
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.