Test Suite vs. Test Plan

Question

I am having a difficult time pinning down an official description of what a test suite is compared to a test plan.

In Test Computer Software they don't even mention suites. (I assume because the book is rather dated)

From IBM they state that:

If each test case represents a piece of a scenario, such as the elements that simulate a user completing a transaction, use a test suite. For instance a test suite might contain four test cases, each with a separate test script: Test case 1: Login, Test case 2: Add New Products, Test case 3: Checkout, Test case 4: Logout

Test suites can identify gaps in a testing effort where the successful completion of one test case must occur before beginning the next test case.

which makes it sound like they are classifying a test suite as something to be run in succession.

Wikipedia states:

In software development, a test suite, less commonly known as a validation suite, is a collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours. A test suite often contains detailed instructions or goals for each collection of test cases and information on the system configuration to be used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.

Collections of test cases are sometimes incorrectly termed a test plan, a test script, or even a test scenario.

I also found this explanation:

Fundamentally Test Plan is a logical collection of Test Cases. Lets say you have some test cases that test the Authentication functionality of a website, then you can group all those Test Cases under an Authentication_Test_Plan.

Test Suite on the other hand is an execution unit such as Functional Test, User Acceptance Testing (UAT), Integration Test or Regression Test. These Test Suite can have any combination of Test Cases which are required for the corresponding Testing phase. So 2 Test Suites can actually have the same or widely different Test Cases.

I could continue to give more examples of dissenting opinions, but this has already gone pretty long. I am starting to get concerned that there isn't an actual definition, and everybody just makes up their own definition to fit their preconceived notions. If that is the case I'll just move on, but I'm hoping that isn't the case, and we can pin down what they actually are.

Answer 1

The traditional definitions would be something like this: A test suite is a collection of test cases related to the same test work. You might have a suite for regression, one for build verification tests, a suite that is specific for a component, and so on.

A test plan is generally a document which describes testing approach and methodologies being used for testing the project, risks, scope of testing, specific tools, and so on. It does not usually include specific test case information, as it is higher level than that.

Answer 2

If we refer to ISTQB glossary of terms then

test suite: A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one.

test plan: A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process.

Answer 3

From Test Management tools perspective a "Test Plan" is a collection of certain test cases to be executed for a certain build. For example in MSTF (VSTS) from Microsoft and ALM from HP the term 'Test Plan' is used. Other test management tools like SpiraTest uses term 'Test Set' to refer the collection of certain 'test cases' being 'planned' for 'Execution' for a certain 'build' testing purpose. In a similar way 'Suite' is also a name given to collection of test cases (action or steps based test cases representing single scenario like Login, end to end scenarios).

In general as well, when you are asked to test an application / module / feature, you may be asked to write or convey your 'Test Plan' that may contain your Test Mission, Test Strategy, Logistics and so on (this may be written and documented or may be not i.e. 'Indicative'). Other way to define a Test Plan is CMM/SEI way documenting test plan in terms of Scope, Resources, Time, Milestones, Entry / Exit criteria ideas of template based 'Test Plan' that fits into a certain projects (& serve a purpose indeed).

Answer 4

Certainly this is not official, but since a lot of people do work with Azure DevOps, for the sake of comparison, you may be interested to look at how Microsoft defines these terms. I would rephrase them below and add my own experience.

Test case is a collection of steps.
Test suite is a collection of test cases.
Test plan is a collection of test suites.

Test plans are meant to be created for each release or sprint (see the second link below: "For each development cycle and release of your product, you create a test plan and import the existing test cases into that plan"). Every time you plan a release, you copy the test cases from previous plans. You may include all cases, or just some. In case of our company, we may test Setup and Installation for all supported language versions, but not all software components. Test suites are just like folders - they are meant to help you organise the test cases. You can however just put the test cases directly into the plan. If you have many test cases then using suites makes life much easier.

Reference:

• https://docs.microsoft.com/en-us/azure/devops/test/test-objects-overview?view=azure-devops

• https://docs.microsoft.com/en-us/azure/devops/test/create-a-test-plan?view=azure-devops

Answer 5

In some special cases, what people are referring to as a test plan (https://en.wikipedia.org/wiki/Test_plan) can also be described as a test strategy (https://en.wikipedia.org/wiki/Test_strategy). Also, I have seen definitions of Test Plans that could be the same as a Test Suite. That being said, the definitions according to the ISTQB Glossary of Terms are typically what should be used as described above.

Answer 6

Sticking to the terms, a test suite is the collection of test cases that are coupled together to meet the defined short-term goals in a test plan. A test plan on the other hand is the complete set of information that testers need to proceed with while testing a technology. It usually includes all the test requirements, documents, test approach, methodology, tools, and technologies used.