It is known that 100% test/code coverage does not mean your code "works" as expected.
What is generally meant by this statement and what use cases could demonstrate that 100% coverage is not enough?
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.Sign up to join this community
In addition to the other answers, a few extra thoughts:
Of course, these are all also reasons why it's impossible to test everything, which is at least in part the point. As I see it, 100% test coverage is like driving over every street in a town. You might be able to drive over each street at least once, but that doesn't mean you'll be able to find your way from where you are to a random address in the town, and every user will be going from a different starting point to a different end point, by a different route, at a different speed, in a different vehicle, with a different amount of fuel in the tank.
Thinking of 100% test coverage as a holy grail of testing is a common misconception which leads to over-confidence in the testing strategy and a false sense of security:
What else could you do?
instead of example-based approach to thinking about your test inputs, explore Property Based Testing where you could define the properties of the desired inputs and let the tools generate inputs:
Instead of asking “what is the expected result of this program given these example inputs?”, we ask the question: “what is the property of the result and program that doesn’t change given inputs?
for certain type of projects (e.g. compilers), ideas like Random Testing could help reveal difficult-to-find issues
Mutation testing is yet another technique to check the quality of your tests where the code under test is changed/mutated for the purpose of seeing if the tests would change its behavior
Other resources on the subject:
Normally the app is not just the code, but infrastructure, resources, protocols, databases, etc. Usability, performance, stability aspects are not usually a matter of code coverage as well.
Low quality test cases could result high coverage. One could write a test case to cover a piece of source code but this does not mean necessarily that this test case actually verifies the correct and expected behavior of that specific code part.
So unless a test case is specified, designed and then implemented thoroughly, the value of the coverage it actually introduces could be uncertain.
When you begin without unit tests, the inclusion metric can help. You'll feel a feeling of achievement as you cure the circumstance and go from zero percent to 25 percent to 70 percent. Stamp your advancement and praise your successes.
However, as you begin coming to the heart of the matter where you're discussing whether to stop at 85 percent, 95 percent, or 100 percent, you're beginning to ask the wrong inquiry. You're beginning to ask how far you ought to go to satisfy a measurement producing apparatus. Rather you ought to ask how you can legitimize each line of code you make and how you can keep a circumstance in which your clients pioneer striking new trails through your code. How would you in any event come to them with the affirmation that you've executed the majority of the code in your codebase? That is unquestionably not all that a lot to inquire.
On top of other excellent answers ,I want to share an real story depicting where 100% coverage is not enough:
I came across a situation in a start-up project, where a well tested(unit/integration) page was brought into consideration first time to Business owner.Due to some project constraints it was entirely designed by a Sr. developer.
The page was working fine as fully covered by detailed tests on multiple levels as per the functionality described by the developer.
However as discovered in business review ,that's not how a real user would use the application on the production floor in busy hours.Entire page was scrapped.
Moral of the story: When requirement itself is incorrect, no amount of coverage will suffice , even 100%.