8
votes
How can 100% test coverage be "not enough to prove working code"?
Thinking of 100% test coverage as a holy grail of testing is a common misconception which leads to over-confidence in the testing strategy and a false sense of security:
code coverage metric only ...
- 11.4k
7
votes
Accepted
How can 100% test coverage be "not enough to prove working code"?
In addition to the other answers, a few extra thoughts:
100% coverage does not mean it meets user expectations - Your tests could cover 100% of the code, but that does not mean that the application ...
- 31.5k
6
votes
Accepted
How would we know when we are over-testing?
We're over-testing when we are not adding value
It's easy to test to feel good but it's important to do testing that truly adds value as the business defines it.
This is a difficult thing to ...
- 25.1k
5
votes
Accepted
Working just on branch or line coverage could prevent us from finding which defect?
Achieving branch coverage is possible but it doesn't mean complete testing.
Indeed, no amount of fact-checking can achieve complete testing, because fact-checking is only one of the activities of ...
- 10.5k
4
votes
How should a Black-box Tester measure Test Coverage?
Black box approach in and of itself is specifically designed to come from the user perspective backwards. The difference between black box and white box testing is knowledge of the underlying code ...
- 1,693
4
votes
How would we know when we are over-testing?
I've not, yet, listened to the podcast so the what follows here are my initial, unfiltered, tuppence worth thoughts on the simple question regarding over-testing implied by coverage.py output so far ...
3
votes
How would we know when we are over-testing?
Over-testing also hides defects in layers.
I discovered in test code reviews & in executions analysis that if we repeat a basic initial assertion in multiple transactional flow tests(maybe due to ...
- 5,552
3
votes
Accepted
ISTQB question: Statement Coverage and Decision Coverage with loop
You have 4 possible decisions/branches:
B-C
B-D
D-E
D-A
The 2 test cases A-B-D-E-F and A-B-C only cover 3 (1. – 3.) out of 4, which is why it's 75% decision/branch coverage.
- 2,132
3
votes
Is it a good practice to run functional tests with coverage?
First of all, coverage isn't coverage. There're several coverage criteria, but most used are:
Function coverage – Has each function (or subroutine) in the program been called?
Statement ...
- 2,132
3
votes
ISTQB Foundation Level Paper 3, no. 37
It's three paths. If I express it via a more c# syntax you get this:
if (condition1)
{
statement = statement1;
}
else
{
statement = statement2;
}
That gives us 2 paths:
condition1 == ...
- 31.5k
3
votes
How can 100% test coverage be "not enough to prove working code"?
Normally the app is not just the code, but infrastructure, resources, protocols, databases, etc. Usability, performance, stability aspects are not usually a matter of code coverage as well.
- 11.6k
3
votes
Accepted
How many test cases can I run if I'm using domain testing?
Domain testing is an umbrella term for Equivalent partition and boundary value analysis. Here, we try to cover all the available behaviours of a system by using the least number of inputs. Here domain ...
- 11k
3
votes
Accepted
How can I measure the coverage of my E2E tests?
Michael Bolton proposes testing coverage as:
“X coverage is how thoroughly we have examined the product with respect to some model of X”.
And he completes:
"Test coverage, like quality, is not ...
- 10.5k
2
votes
Why do code coverage of integration test?
You typically exclude test code from code coverage reports. The entire goal is to learn which parts of production code are touched by your test code.
If you run integration tests against a deployed ...
- 487
2
votes
Testing approaches with temporal delays
Like Kate mentioned, these are easy to miss.
Many systems have tasks that kick off at certain times. Its good to ask for, or check, the cron tables to see those jobs. Then you can run a "mid-night" ...
- 1,304
2
votes
Testing approaches with temporal delays
This is the kind of potential problem that tends to be picked up by testers with a lot of in-depth and broad knowledge of the application when reviewing user stories/use cases/requirements - and it's ...
- 31.5k
2
votes
Accepted
Comparing test coverage metrics to identify an increase/decrease after implementing automation
Test coverage and implementation of automation framework are 2 different things. According to me they are not linked directly.
Automation frameworks won't write the test scripts on their own. People ...
- 8,921
2
votes
Accepted
Examples for LOC figures, of test suites, and software projects?
So you need real-world examples of LOC metrics in a software sources and a test suite LOC volume, right? I can share some rough metrics for one real-world project. Never heard about representative ...
- 268
2
votes
What are some good examples of pure Computer Science and Mathematics applications in software QA?
Hoare Logic - Automated testing.
Computational complexity
Big-O
Halting Problem
Sampling theory
Automata Theory
Finite-state machines
- 335
2
votes
How to document the paths of my windows service in an easy to read diagram?
I am not aware of a specific type/name for this documentation, but you can use draw.io to create a flow/path diagram as you mentioned. You can also explore the different types of diagrams they have ...
- 194
2
votes
How many test cases required to achieve 100 % statement coverage
There is exactly one executable statement in the example, so if the question is not asking for branch coverage, one test case will give 100% coverage.
An if statement is a flow, or branching ...
- 31.5k
2
votes
Accepted
Why Phantomjs got crashed while running more test cases?
PhantomJS notoriously has a lot of memory issues and will crash after a certain amount of memory use. I have dealt with this issue by splitting up the tests into separate phantom processes rather than ...
- 136
2
votes
ISTQB question: Statement and Decision coverage
This becomes more readable with indentation:
READ A
READ B
READ C
IF C>A THEN
IF C>B THEN
PRINT "C must be smaller bla bla.."
ELSE
PRINT "Proceed to next stage"
...
- 2,132
2
votes
How would we know when we are over-testing?
The biggest problem with over testing is maintenance of tests. First you are allocating resources to create the tests, then your tests require resources to maintain. Since your man power is finite, ...
- 609
2
votes
How can I measure the coverage of my E2E tests?
I am afraid there is no good enough answer for that.
There are tools and methods but they all have major disadvantages that the people you report to should be aware of.
Test coverage, how many tests/...
- 7,089
2
votes
How to calculate condition coverage?
With your reference to your test case what you have provided is correct but think how the application/function will take the input. It can take only three inputs so when your conditions are split ...
2
votes
Accepted
What will be the statement coverage, decision coverage and branch coverage of these following problem?
Statement coverage means every statement had to be executed.
Decision coverage measures the coverage of conditional branches. Branch coverage measures the coverage of conditional and unconditional ...
- 4,538
2
votes
Accepted
Given the following fragment of code, how many tests are required for 100% decision coverage?
I hope this will help you understand why the answer is 4.
100% Decision coverage
The code traverses 4 paths/decisions for 100% Decision coverage.
This happens when the decision statements are nested. ...
- 8,921
2
votes
Accepted
How many more test cases are required to achieve provide 100% statement and 100% decision coverage?
Considering the given piece of code following cases can given 100% Decision and 100% Statement Coverage.
Case 1
Width = 10
Length = 5
Case 2
Width = 5
Length = 10
Now, you add the three statements ...
- 8,921
Only top scored, non community-wiki answers of a minimum length are eligible