When there is a new build we perform smoke testing,in few projects they call it as sanity testing. Can someone please elaborate on the difference between the both?
Just to muddy the waters even more I'm going to throw in "it depends". If you're looking for a practical answer within your own company then I'd ask them, as they are terms that change for different domains. Sometimes they are the same, sometimes they differ, and often they are defined at the whim of whomever writes the tests.
Stuartf's answer is as close to an "offical" definition as you will get. A little history and context will help display the difference:
Smoke tests refer orignally to hardware testing, wherein a device would be plugged in and if it started to smoke then it was unplugged as no further testing was required. It doesn't work, there is no point in testing it.
Sanity tests are, traditionally (in other realms of testing) tests that establish whether or not a result could be true, rather than if it is true, that is to say if the output is "sane". If you put "9" into the function "squareroot()" and get "badger" you have failed a sanity test. If you get, say, a positive number less than 9 then you have passed it, even if the number is not "3". This is why they refer to tests that ensure that the system is working enough to be tested - it is giving a sane response to basic tests, therefore it is not a waste of time to run full functional tests.
The confusion comes when lost in the complexity of software - at what point is the system testable? How can we see if smoke is coming out of it when it only breaks when a certian section of the code is run? It differs from project to project, evidently. Also in order to do a smoke test you need to use your testing framework (manual or otherwise) to actually check if the system is running and responsive; so in many cases they ARE a subset of functional tests. So the definitions are different, but are (often erroneously) used interchangably on a practical level. Hopefully knowing the context will give some meaning to the words, even if they both reduce (practically and contextually) to "give it a kick and see what happens".
Their purpose is similiar in software testing, to save time on running a full regression suite and finding out that the system falls over because of a base error. The reason being that a basic error in the system has to be fixed, and that fix affects the system, and the testing should then be restarted, so the quicker that larger, high priority errors can be found the better. For example a web application may be spidered for 404 errors and broken links before long, slow, functional tests are run.
Hope that's helpful!
Sanity and Smoke testing are conflated. They are not the same and they have a different purpose!
Smoke tests determines whether it is possible to "continue" testing and sanity tests determines whether it is "reasonable" to test.
A smoke test should only ensure that application launches and that all the interfaces are responsive. You do not test a subset of functionality in a smoke test.
A sanity test should ensure that the application is in a state that it is ready for more rigorous testing. This is done by executing a small subset of you functional tests.
As Joe suggests, they are often used as a gate to make sure that time isn't wasted on testing a build that is fundamentally flawed.
In most shops, there is no difference. Often, they are two names for the same process.
In both cases, they are a subset of all your test cases, covering the main functionality of your system, to ascertain that the most crucial features are working, but not bothering with finer details.
Many times the smoke test/sanity test is used as a gateway, through which the build must pass before it's deemed worthy of more in-depth testing. If the build fails this test, it goes back to the dev team for fixing. If it passes, the test team then digs in.
To remember easily,
Smoke test - you just test all pages are accessed without errors and whether input is submitted and you get results. You do this across the application.
Sanity test - You take 1 ore 2 modules. probably the more no. of fixes it has or the important one and test it deep by going through all functions.
You understand that you do this only to accept the build for further testing, ie. regression, system, etc.
Smoke testing is normal health check up of a build before starting the in-depth testing.
Sanity testing is to verify whether requirements are met or not, checking all features breadth-first. Depth of sanity testing is more than smoke testing. Sanity testing helps in identifying whether the test case results are consistent or not.
Some people consider both smoke and sanity testing as same.
Other books which discuss the difference:
- Software Testing by Limaye
- STRUCTURED SOFTWARE TESTING by Arunkumar Khannur
- Software Testing: Principles and Practice by Srinivasan Desikan, Gopalaswamy Ramesh
You can also have a look at Regression / Smoke / Sanity
As the name suggests->
Smoke testing-> Here you check your entire system and check if it works fine or has any issues (smoke). This is normally done on stage before deploying the code to prod.
Sanity Testing-> Here you do a last minute check. We only test the major functionality and see if the sanity is maintained.
Hope this helps !
Basically smoke testing is carried out once the product have been developed to ensure all the functionalities are working fine, whereas sanity testing somewhat for we ensure that the bugs/errors have been removed after the recent changes made in the code. I've been working on this topic and shared a guest post on it. You can get complete difference between them in this article. Hope this question might be useful to your question.