Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I am doing study on software testing and I have a question. Is their any need of test suites to do automated software testing?

share|improve this question

6 Answers 6

As always with testing, it depends on the situation and exactly what you mean by the question.

If you're asking if there's ever a need to group automated tests into a suite, then yes, there are times when this is a good idea. If you want to know if it's always needed, then the answer is no - not all automation requires that automated tests be grouped into suites.

Some other things to consider:

  • As a general rule test suites will be grouped according to the purpose they serve. A small group of tests that covers basic functionality might be used in a smoke test suite, where a set of tests covering a specific feature development will be in a different suite.
  • Where there are a large number of automated tests (not unit tests), they will often be grouped into suites so that multiple test systems can be used to run the automation in parallel. I've been at places where the number of automated tests made it necessary to do this because running all of them in succession would take over a week. By using test suites and multiple machines (mostly virtual) the test runs could be done overnight so that regressions were typically found within 24 hours of being introduced.
  • Load tests will usually be kept as a separate test suite.
  • Manual tests may also be arranged into test suites, particularly if they are used for regression or it's likely that they'll be reused at some point. While there may be an overall goal of automating the manual suites, it's not uncommon for automation to lag behind manual testing. There are also always tests that either can't be automated or aren't worth automating. These can be grouped into test suites as well.
  • No matter what your planning or automation strategy is, the number of tests you have will grow over time. Even with the best test maintenance actively pruning obsolete tests, applications grow more complex as features are added, increasing the number of tests required to cover the application to a reasonable degree (it's impossible to completely test anything with even a modest degree of complexity).
share|improve this answer
    
Nice answer. And as always, it depends. –  Joe Strazzere Dec 26 '13 at 16:34

It's not a compulsion to have test suites but they can be very beneficial. For instance, we can categorize our test cases on the basis of functionality/requirements in these test suites and we can execute test suite corresponding to that functionality for any changes, in this way we can manage our test cases in much better way. And we can execute all test cases in a test suite at once instead of running them one by one.

share|improve this answer

Each test takes some time to run. If the combined time is too large, the tests will be run less frequently as it is an effort to do so. Test suites provide an easy way to group tests logically and run only relevant set of tests. Other needs could be for example to run only tests that work in certain environment.

Note that it is possible to gather test suites automatically. You could only run tests that took less than 0.1 seconds to execute on the last run or tests that use files that have been changed since the last run or tests that have been tagged with a certain tag.

Typical scenario is to make it possible to run unit tests of features easily and fast separately, run larger test set with each commit to version control system and possibly run even more for example each night if some tests take a long time or need certain setup.

share|improve this answer

In short - if you are implementing automation methods for your testing, then its always good to have an automation test suite. It helps you to maintain your test scripts, test data, libraries, results etc.. Also you don't need to trigger the scripts manually, you can simply schedule your test runs.. so there are lot of benefits of having an organised suite.

share|improve this answer

If you're using the Build-Deploy-Test workflow and use multiple agents to run your tests, then Test Suites are a natural way to organize this workflow (one test suite per agent).

share|improve this answer

Agreeing with all responses so far. But I will add a management item to the list. If you are reporting on testing progress to the boss or higher, it is often helpful to report in broad categories, which may or may not be drillable into finer detail as desired. These categories translate to suites in the test system because it is here you will put your statistics from.

In my case, our organization has over 2000 tests, which are ever increasing. To report progress on testing, show bottlenecks or problem areas, I report on perhaps 30 areas, which correspond to feature areas in the software. Thus, you do not need to go very deep to find out that Import is showing problem areas or Installation is having issues in a given build. And it also shows quickly your pass percentage and whether the failures are on critical items or less critical items that could be handled in a hot fix.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.