Do I need to perform smoke testing again before sanity testing for every bug fix?

Question

let's say for the 1st time I have checked performed Smoke testing (application main feature) on the 1st day and then I report a Bug and developer fixed and deployed the app I test environment then do I have to do smoke testing again?
and let's say testing time is very less than do I have to do the sanity test+smoke testing again(for each bug i report)?

Answer 1

Smoke and Sanity tests are high level tests intended to main super major bugs like the page doesn't load at all, the URL is incorrect, the link to the main application form is broken. Stuff like that. It is used to provide quick and immediate feedback in seconds or minutes without having to wait hours to find out some basic and fundamental bugs that stop a lot of the rest of the functionality from working.
They are often used in deployments to get immediate feedback

When you find a specific bug and there are changes to code to fix it, you will normally want to do the following:

• Run the general application level smoke and sanity tests to make sure nothing high level is broken

• Run the specific test(s) for the area that was changed or affected. You may be running smoke, sanity, happy, sad and optional tests for this

• Write and run new tests that probe the bug fix area under change in more detail. These might include new smoke/sanity tests.

As you can see from the above, yes you might be running smoke and sanity tests again first. The point is that these would be helpful to you and the organization. If they find a bug in two minutes as opposed to 2 hours of full testing this is something that any good tester would welcome gladly. No good tester want to waste two hours when two minutes would do.

At the end of the day it really depends on what the problem is and what your smoke/sanity tests do.

For example:

• A workflow was changed and users start at a new page that is covered by smoke and sanity tests. Run them!

• A spelling typo was found in the footer disclaimers. The specific content of disclaimers is not checked in smoke test. Don't run them! Actually let me be a little more specific - don't run them to specifically check that the issue (spelling) was fixed. However do run them because SOMETIMES A SMALL CHANGE CAN BREAK THE ENTIRE SYSTEM and indeed this is a major part of the reason for smoke tests and why they should be run. I have learned this from several real world experiences. The lesson I learned was to be humble and just run them.

As to the distinction between 'smoke and 'sanity' and how they relate that is specific to your implementation. Generally in the industry 'smoke' and 'sanity' tests refer to the same sort of high level tests.

Answer 2

Not necessarily.

Smoke testing aims at ensuring that a new build is stable and has not broken some of the existing functionalities. So, you can limit it and decide it's scope on the basis of areas impacted as a consequence of bug fix:

If there is no other application area impacted then the scope of testing is limited to that bug fix but if there are other application areas impacted because of a fix then they should be included in the scope of testing.

So, for example, for a website, if you get a new build then you can check if login is working. Landing page and one or two important pages are displaying properly; as part of smoke testing. This won't take more than 2-3 minutes.

But if the developer feels that functionality "X" might have an impact because of the bug fix then that should be included in the scope of testing.

Generally, before marking the bug as "Ready for testing", the developers mention all the impacted areas in the bug itself (generally, as a comment). So, once the core bug is teated, the impacted areas will also be in the scope of testing.

Moreover, As you mentioned, it is not practical to go through the complete application after each build deployment. That's why there is at least one round of regression testing proposed after every phase of application development.

Answer 3

Use Continuous Integration and define your workflow.

Step1: Let the developer push his code in main dev branch and Let CI run suite of unit tests(very fast in few minutes) as smoke suite to accept/reject the check-in immediately.

Step2: Only if step 1 passes(having any successful check-ins), overnight let the smoke suite of integration tests run in Dev environment.

Step3: Only if step 2 passes,overnight let the CI deploy the same build in QA environment and run the full blown suite of regression tests.

Step4: Next day morning based on the overnight regression results, let the manual/ domain tester take the call to which tests to pick & run manually further.