8 votes

How can 100% test coverage be "not enough to prove working code"?

Thinking of 100% test coverage as a holy grail of testing is a common misconception which leads to over-confidence in the testing strategy and a false sense of security: code coverage metric only ...
alecxe's user avatar
  • 11.4k
7 votes
Accepted

How can 100% test coverage be "not enough to prove working code"?

In addition to the other answers, a few extra thoughts: 100% coverage does not mean it meets user expectations - Your tests could cover 100% of the code, but that does not mean that the application ...
Kate Paulk's user avatar
  • 31.5k
6 votes
Accepted

How would we know when we are over-testing?

We're over-testing when we are not adding value It's easy to test to feel good but it's important to do testing that truly adds value as the business defines it. This is a difficult thing to ...
Michael Durrant's user avatar
5 votes
Accepted

Working just on branch or line coverage could prevent us from finding which defect?

Achieving branch coverage is possible but it doesn't mean complete testing. Indeed, no amount of fact-checking can achieve complete testing, because fact-checking is only one of the activities of ...
João Farias's user avatar
  • 10.5k
4 votes

How should a Black-box Tester measure Test Coverage?

Black box approach in and of itself is specifically designed to come from the user perspective backwards. The difference between black box and white box testing is knowledge of the underlying code ...
mutt's user avatar
  • 1,693
4 votes

How would we know when we are over-testing?

I've not, yet, listened to the podcast so the what follows here are my initial, unfiltered, tuppence worth thoughts on the simple question regarding over-testing implied by coverage.py output so far ...
Martyn Walmsley's user avatar
3 votes

How would we know when we are over-testing?

Over-testing also hides defects in layers. I discovered in test code reviews & in executions analysis that if we repeat a basic initial assertion in multiple transactional flow tests(maybe due to ...
Vishal Aggarwal's user avatar
3 votes
Accepted

ISTQB question: Statement Coverage and Decision Coverage with loop

You have 4 possible decisions/branches: B-C B-D D-E D-A The 2 test cases A-B-D-E-F and A-B-C only cover 3 (1. – 3.) out of 4, which is why it's 75% decision/branch coverage.
beatngu13's user avatar
  • 2,132
3 votes

Is it a good practice to run functional tests with coverage?

First of all, coverage isn't coverage. There're several coverage criteria, but most used are: Function coverage – Has each function (or subroutine) in the program been called? Statement ...
beatngu13's user avatar
  • 2,132
3 votes

ISTQB Foundation Level Paper 3, no. 37

It's three paths. If I express it via a more c# syntax you get this: if (condition1) { statement = statement1; } else { statement = statement2; } That gives us 2 paths: condition1 == ...
Kate Paulk's user avatar
  • 31.5k
3 votes

How can 100% test coverage be "not enough to prove working code"?

Normally the app is not just the code, but infrastructure, resources, protocols, databases, etc. Usability, performance, stability aspects are not usually a matter of code coverage as well.
Alexey R.'s user avatar
  • 11.6k
3 votes
Accepted

How many test cases can I run if I'm using domain testing?

Domain testing is an umbrella term for Equivalent partition and boundary value analysis. Here, we try to cover all the available behaviours of a system by using the least number of inputs. Here domain ...
PDHide's user avatar
  • 11k
3 votes
Accepted

How can I measure the coverage of my E2E tests?

Michael Bolton proposes testing coverage as: “X coverage is how thoroughly we have examined the product with respect to some model of X”. And he completes: "Test coverage, like quality, is not ...
João Farias's user avatar
  • 10.5k
2 votes

Why do code coverage of integration test?

You typically exclude test code from code coverage reports. The entire goal is to learn which parts of production code are touched by your test code. If you run integration tests against a deployed ...
jpjwolli's user avatar
  • 487
2 votes

Testing approaches with temporal delays

Like Kate mentioned, these are easy to miss. Many systems have tasks that kick off at certain times. Its good to ask for, or check, the cron tables to see those jobs. Then you can run a "mid-night" ...
jruberto's user avatar
  • 1,304
2 votes

Testing approaches with temporal delays

This is the kind of potential problem that tends to be picked up by testers with a lot of in-depth and broad knowledge of the application when reviewing user stories/use cases/requirements - and it's ...
Kate Paulk's user avatar
  • 31.5k
2 votes
Accepted

Comparing test coverage metrics to identify an increase/decrease after implementing automation

Test coverage and implementation of automation framework are 2 different things. According to me they are not linked directly. Automation frameworks won't write the test scripts on their own. People ...
IAmMilinPatel's user avatar
2 votes
Accepted

Examples for LOC figures, of test suites, and software projects?

So you need real-world examples of LOC metrics in a software sources and a test suite LOC volume, right? I can share some rough metrics for one real-world project. Never heard about representative ...
Vasily Ryabov's user avatar
2 votes

What are some good examples of pure Computer Science and Mathematics applications in software QA?

Hoare Logic - Automated testing. Computational complexity Big-O Halting Problem Sampling theory Automata Theory Finite-state machines
Martin Spamer's user avatar
2 votes

How to document the paths of my windows service in an easy to read diagram?

I am not aware of a specific type/name for this documentation, but you can use draw.io to create a flow/path diagram as you mentioned. You can also explore the different types of diagrams they have ...
Maira Bay's user avatar
  • 194
2 votes

How many test cases required to achieve 100 % statement coverage

There is exactly one executable statement in the example, so if the question is not asking for branch coverage, one test case will give 100% coverage. An if statement is a flow, or branching ...
Kate Paulk's user avatar
  • 31.5k
2 votes
Accepted

Why Phantomjs got crashed while running more test cases?

PhantomJS notoriously has a lot of memory issues and will crash after a certain amount of memory use. I have dealt with this issue by splitting up the tests into separate phantom processes rather than ...
Beetny's user avatar
  • 136
2 votes

ISTQB question: Statement and Decision coverage

This becomes more readable with indentation: READ A READ B READ C IF C>A THEN IF C>B THEN PRINT "C must be smaller bla bla.." ELSE PRINT "Proceed to next stage" ...
beatngu13's user avatar
  • 2,132
2 votes

How would we know when we are over-testing?

The biggest problem with over testing is maintenance of tests. First you are allocating resources to create the tests, then your tests require resources to maintain. Since your man power is finite, ...
newsn31's user avatar
  • 609
2 votes

How can I measure the coverage of my E2E tests?

I am afraid there is no good enough answer for that. There are tools and methods but they all have major disadvantages that the people you report to should be aware of. Test coverage, how many tests/...
Rsf's user avatar
  • 7,089
2 votes

How to calculate condition coverage?

With your reference to your test case what you have provided is correct but think how the application/function will take the input. It can take only three inputs so when your conditions are split ...
Mohamed Sulaimaan Sheriff's user avatar
2 votes
Accepted

What will be the statement coverage, decision coverage and branch coverage of these following problem?

Statement coverage means every statement had to be executed. Decision coverage measures the coverage of conditional branches. Branch coverage measures the coverage of conditional and unconditional ...
pavelsaman's user avatar
  • 4,538
2 votes
Accepted

Given the following fragment of code, how many tests are required for 100% decision coverage?

I hope this will help you understand why the answer is 4. 100% Decision coverage The code traverses 4 paths/decisions for 100% Decision coverage. This happens when the decision statements are nested. ...
IAmMilinPatel's user avatar
2 votes
Accepted

How many more test cases are required to achieve provide 100% statement and 100% decision coverage?

Considering the given piece of code following cases can given 100% Decision and 100% Statement Coverage. Case 1 Width = 10 Length = 5 Case 2 Width = 5 Length = 10 Now, you add the three statements ...
IAmMilinPatel's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible