Lets assume we have 5 modules in the application. Module 1 interacts with Module 2. We are in Sprint 15. In Sprint 15, there is new functionality added in Module 1. When the build of sprint 15 is deployed in QA, what must be the sequence of testing-

  1. Should we it be Smoke->Sanity->Functional->Regression? Considering, the answer for the above questions as yes,
  2. Smoke is to check the build stability where we will check crucial features of all the 5 modules. Lets say we have 3 crucial features in each module, so we will execute 15 test cases in smoke test cycle. Right?
  3. We will not test these 15 features in deep. Right?
  4. Since it is scripted, can we have smoke test suite as a standard one for each sprint? Of course, if there is any change in requirement or new feature in one sprint, then we will add/update this suite. So can we have smoke test suite as a standard one for each sprint?
  5. Generally how long should be a smoke test cycle? Should it be short like 3-4 Hours?
  6. As per point 1, we would start with Sanity testing after smoke. Sanity is not scripted and is testing the specific areas of application. So in this case of Sprint 15, we will test the major areas of Module 1 and Module 2. Right?
  7. So when we do Sanity testing, we will test major areas of Module 1 and 2 in deep. Right?
  8. Some say Sanity testing is done after retest. If that is correct, then after the bugs are fixed, we have to do a quick sanity test on areas of Module 1 and 2. As per this, the test cycle would be Smoke->Sanity->Functional->Sanity->Regression. Right?
  9. or based on if we have enough time we would do regression, so test cycle would be smoke->Sanity->Functional->Regression. If we do not have enough time then we would do Sanity. So test cycle would be smoke->Sanity->Functional->Sanity.

Please let me know the answers point wise and let me know if all the above points are right.

  • Please do two things in order to get you good answers: 1. One question at a time please, I see about 10 here :) 2. Generalize your question to that it will help others in similar situations. I think you have tried to do this already but that is still so much detail it feels tailored to your particular situation and its hard to see the forest for all the trees :) – Michael Durrant Jul 30 '17 at 13:54
  • One question per question also lets the voting system work much better – Michael Durrant Jul 30 '17 at 13:56
  • Please define smoke/sanity/functional/regression testing. They can be defined differently by different people/organizations. – Kevin McKenzie Jul 30 '17 at 15:03
  • The reason for asking all questions in a single question is because there are so many definitions and variations on these 4 types of testing and an answer to all questions in a single place will really help a lot of people. A kind of end to end answer. – user2356679 Jul 31 '17 at 23:21
  • Can anybody throw some light here? I certainly believe it will definitely help a lot of people. Again the reason for putting down all at one place is to have an end to end answer and arrive at a typical sequence of testing. – user2356679 Aug 4 '17 at 0:53

ok, so this question is really broad, but the underlying goal seems consistent in the order of testing within a sprint cycle. I'll try to address that with the understanding that the depth of testing is entirely dependent on analysis and what your application is and what was modified.

Beginning Point: Once Dev is complete with something (hopefully unit tested and in sprint fashion is a releasable feature - great scrum means QA will have already been involved and dev/QA are in sync)

  1. Build created - Smoke testing kicked off after install which will determine if the install was successful and the application is functioning. This should only take at most 30 minutes unless the app is humongous and then I would hope a team is smoke testing or parallel automation to verify and get the verification to 30 min or less.
  2. Functional testing on the new expected functionality happens. This is to ensure that the build is actually doing the improvements that were expected. No point in doing a whole bunch of other testing if the build doesn't even do what it's supposed to be improved in doing.
  3. Regression/Sanity testing is next. Regression is detailed testing targeting affected areas that would likely have been modified in some way which could lead to bugs. Sanity testing is the areas outside of this and is a cursory exploratory testing to verify that the rest of the application which should have been untouched is indeed "untouched" and functions the same.

Note: Analysis is needed for all testing as it really and truly depends on what code was modified, the confidence level of the team in a feature, the complexity and interactivity of a feature, usability, performance, security, user perspective (especially if you have a picky customer), budget constraints on hours, team size and skill sets, Automation, # of tests, size of tests, etc... There is a reason it's "engineering" and applying simple 1, 2, 3 steps is not going to fit every application the same.

P.S. To fit the above with your sprint example, Module 1 should be functional/regression tested, other modules should be regression tested where code connection with module 1 is concerned and sanity tested everywhere else. If you have the time more regression wouldn't hurt as it might find lingering bugs to work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.