Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

When there is a new build we perform smoke testing,in few projects they call it as sanity testing.Can someone please elaborate on the difference between the both.

share|improve this question

7 Answers 7

Just to muddy the waters even more I'm going to throw in "it depends". If you're looking for a practical answer within your own company then I'd ask them, as they are terms that change for different domains. Sometimes they are the same, sometimes they differ, and often they are defined at the whim of whomever writes the tests.

Stuartf's answer is as close to an "offical" definition as you will get. A little history and context will help display the difference:

Smoke tests refer orignally to hardware testing, wherein a device would be plugged in and if it started to smoke then it was unplugged as no further testing was required. It doesn't work, there is no point in testing it.

Sanity tests are, traditionally (in other realms of testing) tests that establish whether or not a result could be true, rather than if it is true, that is to say if the output is "sane". If you put "9" into the function "squareroot()" and get "badger" you have failed a sanity test. If you get, say, a positive number less than 9 then you have passed it, even if the number is not "3". This is why they refer to tests that ensure that the system is working enough to be tested - it is giving a sane response to basic tests, therefore it is not a waste of time to run full functional tests.

The confusion comes when lost in the complexity of software - at what point is the system testable? How can we see if smoke is coming out of it when it only breaks when a certian section of the code is run? It differs from project to project, evidently. Also in order to do a smoke test you need to use your testing framework (manual or otherwise) to actually check if the system is running and responsive; so in many cases they ARE a subset of functional tests. So the definitions are different, but are (often erroneously) used interchangably on a practical level. Hopefully knowing the context will give some meaning to the words, even if they both reduce (practically and contextually) to "give it a kick and see what happens".

Their purpose is similiar in software testing, to save time on running a full regression suite and finding out that the system falls over because of a base error. The reason being that a basic error in the system has to be fixed, and that fix affects the system, and the testing should then be restarted, so the quicker that larger, high priority errors can be found the better. For example a web application may be spidered for 404 errors and broken links before long, slow, functional tests are run.

Hope that's helpful!

share|improve this answer
    
+1 for an EXCELLENT analysis of the "It depends" aspect of things. On a side note, for the hardware smoke test, I've experienced this although, in our case, it was a flame test... old Leading Edge D2 PC, plugged it in, capacitor in the RAM circuits exploded (literally) and caught the motherboard on fire... Yup, that one doesn't work... –  TristaanOgre Jun 21 '11 at 14:45
2  
I envisiage a smoking wreck with a "no further testing required" sticker on the side :). I've suffered a capacitor explosion myself in a speaker system, it caked the inside of the casing with smoking dialetric, so I know from whence you come! Many thanks for your kind words –  kinofrost Jun 21 '11 at 16:14
    
Your mental image is accurate... mix in the smell of chemical fire-extinguisher and scorched silicon and you've pretty much described the situation... –  TristaanOgre Jun 21 '11 at 17:27
1  
it always depends... –  Joe Strazzere Jun 21 '11 at 19:24
1  
Heh you guys are making me remember an old article on new solid state devices that appeared in an april issue of popular electronics long ago (I'm ancient). Three were applicable to this discussion.. SSCG Solid State Confetti Generator (25uF electrolytic 5v cap, placed across 120vac source.. good for one use only. NED the Noise Emitting Diode (Place a normal LED across 120vac source, makes a loud noise, once.) and DED -Dark Emitting Diode, what you have after a NED has been used once. –  Chuck van der Linden Jun 23 '11 at 4:45

In most shops, there is no difference. Often, they are two names for the same process.

In both cases, they are a subset of all your test cases, covering the main functionality of your system, to ascertain that the most crucial features are working, but not bothering with finer details.

Many times the smoke test/sanity test is used as a gateway, through which the build must pass before it's deemed worthy of more in-depth testing. If the build fails this test, it goes back to the dev team for fixing. If it passes, the test team then digs in.

share|improve this answer
    
I concur with Joe, I've heard both used and when they are it's usually to refer to the same thing. –  MichaelF Jun 21 '11 at 12:18
1  
They are not the same thing, unfortunately this is common mistake in software quality assurance. They do share similar characteristics, however, they serve a different purpose. –  stuartf Jun 21 '11 at 12:52
    
We need a wiki page to cover all these terms and their most common definitions, because this question will come up again. –  user246 Jun 21 '11 at 13:05
    
Looking at the glossary in a textbook (The Testing Practitioner by Erik van Veenendaal), for Sanity Test it says "see smoke test". –  Tom77 Jun 21 '11 at 15:27
1  
In most shops, the terms are used interchangeably. In some shops they are distinct terms, as in the case of stuartf. Our profession has no real rules for terminology, and even the common usages are violated by some shops every day. I've encountered almost as many different phrases that mean something like "a gate to make sure that time isn't wasted on testing a build that is fundamentally flawed" as I've encountered companies. (Not to mention: strazzere.blogspot.com/2010/04/bug-by-any-other-name.html ) :) –  Joe Strazzere Jun 21 '11 at 16:39

Sanity and Smoke testing are conflated. They are not the same and they have a different purpose!

Smoke tests determines whether it is possible to "continue" testing and sanity tests determines whether it is "reasonable" to test.

A smoke test should only ensure that application launches and that all the interfaces are responsive. You do not test a subset of functionality in a smoke test.

where as

A sanity test should ensure that the application is in a state that it is ready for more rigorous testing. This is done by executing a small subset of you functional tests.

As Joe suggests, they are often used as a gate to make sure that time isn't wasted on testing a build that is fundamentally flawed.

share|improve this answer
    
I have also seen the 'sanity' test done as a quick test (in environments without good automated regression suites) ala your description, a random sampling of functional tests across the whole product, to ensure that a recent patch or 'hot fix' didn't break anything. That's a very different thing than 'is it good enough to test' it's closer to 'we don't have time to retest it all, does it look like nothing suddenly started behaving in a crazy way.' –  Chuck van der Linden Jun 23 '11 at 4:50

To remember easily,

Smoke test - you just test all pages are accessed without errors and whether input is submitted and you get results. You do this across the application.

Sanity test - You take 1 ore 2 modules. probably the more no. of fixes it has or the important one and test it deep by going through all functions.

You understand that you do this only to accept the build for further testing, ie. regression, system, etc.

share|improve this answer

Basically smoke testing is carried out once the product have been developed to ensure all the functionalities are working fine, whereas sanity testing somewhat for we ensure that the bugs/errors have been removed after the recent changes made in the code. I've been working on this topic and shared a guest post on it. You can get complete difference between them in this article. Hope this might be useful to your question. https://www.exploreb2b.com/articles/smoke-testing-vs-sanity-testing

share|improve this answer

As the name suggests->

Smoke testing-> Here you check your entire system and check if it works fine or has any issues (smoke). This is normally done on stage before deploying the code to prod.

Sanity Testing-> Here you do a last minute check. We only test the major functionality and see if the sanity is maintained.

Hope this helps !

share|improve this answer

sanity testing means we have to go through every main part in detail we can avoid parts who have low severity and in smoke testing we have to check each and every part in cursory way...each and every parts should be tested

share|improve this answer

protected by user246 Dec 17 '13 at 18:47

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.