If I have a test case that passes on one version of the code and a change is made to some lines in the code that are not hit by this test case, is there a possibility that the test case fails after this change? If yes, can you provide an example?
To clarify more: If I have for example a code block as follows


And I have a test case that only hits Statements 1 and 2. Now suppose a change is made to statement 3 and I need to run the regression to make sure nothing is broken. In that case, does it make sense to run the test case that does not hit Statement 3. Is there any possibility that the test case breaks after the modification?

  • Does this behaviour show itself when the test is executed on its own, or only when it is part of a larger test-set? Are you sure the changed code is also not executed as part of starting/setting-up the SUT? – Bart van Ingen Schenau Jul 10 '19 at 17:29

It depends on what Statemen3; is doing to the overall state of the software. Without you giving more details it is hard to say. At a high level: if Statement3; creates some situation where the behaviour of Statement1; or Statement2; might change, then your test case might fail.

That's why, if you are doing white-box testing (which seems to be the case), it is very important that you have a deep understanding of the code. If you don't, you should ask the developer who wrote (or who is responsible for) the code. They are the best people to answer this type of regression testing question.

| improve this answer | |

Depends with what you mean by "not hit by this test case".

If you mean traditional code coverage, yes, it is possible.

Metaprogramming allows you to change the structure of code at compile and run-time in ways that may create dependencies in good* unit checks.

Aspect-Oriented Programming changes behavior in a similar fashion.

Also, if you have checks that are integrate or bundle many components, data or expectations may cause mutations to appear in unexpected ways.

Note: These examples are more focused on OOP - other paradigms may have specific situations.

* Well defined boundaries and mocked dependencies.

| improve this answer | |
  • I would really appreciate it if you can provide a specific example. I have updated the description for more clarification – Abdo Saied Anwar Jul 11 '19 at 6:46

If your test case is not covering the new code, the testcase needs to be changed. Testers need to work closely with developer's to make sure all of the code is executed when the tests are performed.

| improve this answer | |

If I've understood what you're asking correctly, one would assume that provided the functionality of statement 3 remains unchanged but the code was just refactored. The test wont fail but it wont necessarily pass as it hasn't covered statement 3.

If you want it to pass for all then you will need to adjust the code accordingly (if required) to match the refactored statement 3. It comes down to how much coverage you want.

The code change would have to have changed the functionality of the test for it to fail. though it may not cover even if it doesn't.

TLDR Long and short. Make sure it hits all the statements and features required for coverage otherwise your test may pass with a false positive.

| improve this answer | |

Depends, If Your changes made effect statement 3 but has some dependency on testcase 1 and 2 , it might break. You need to cross check with the dependency of code for all the test cases.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.