Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [coverage]

Coverage measures the amount of testing done of a certain type. It is always in relation to some kind of measure, for example: statement coverage, branch coverage, etc. (Not restricted to code coverage).

Filter by
Sorted by
Tagged with
12 votes
4 answers
2k views

How can I report coverage on an n-dimensional test matrix?

I have a test matrix for my product's installation, which is n-dimensional, where n is about 11 or 12 at this point. For instance, the OS is one dimension, and the DB is another, and the Reporting ...
Carmi's user avatar
  • 1,203
11 votes
3 answers
11k views

Measuring code coverage in end-to-end tests?

I've been hearing about ideas for measuring code coverage in end-to-end tests (e.g., Selenium tests) and I really don't see a reason for that, even if this is technically feasible. There's a concept ...
dzieciou's user avatar
  • 10.5k
11 votes
3 answers
2k views

Python coverage and external executables

I have a project that has almost complete unit test coverage. There's only one statement that isn't covered and I can't work out a good way to do so. Say the project is called foo, I have foo/commands....
Soren's user avatar
  • 213
10 votes
5 answers
3k views

QA Methodology: Do you retest every change on a new build or only changed code?

I am leading a new QA team at my company and we are small (Myself and one other person). We are currently testing releases compiled by over 10 developers, so the ratio of developers to QA is skewed. ...
Frank's user avatar
  • 143
10 votes
2 answers
15k views

Why do code coverage of integration test?

in my project I started code coverage for unit and integration test with gradle + jacoco. It's already working and generating reports. I saw a lot of questions and tutorials about how to do code ...
Thiago Fioravante's user avatar
9 votes
3 answers
682 views

How should I test the output of image renderers thoroughly, efficiently and in a branchable way

I am trying to improve the tests for a bunch of renderers that generate 2D images, and I am looking for advice and ideas for testing that are both efficient and thorough but also branch friendly. All ...
Matt Bond's user avatar
  • 191
9 votes
1 answer
1k views

Continuous Integration and testing environment and configuration for Android

I am an Android developer but I am not satisfied with my testing and Continuous Integration environment and configuration. Someone could recommend me how to configure the best setup and workflow for ...
Edu Zamora's user avatar
8 votes
1 answer
15k views

How does Bullseye code coverage tool work?

I'm using Bullseye code coverage tool. It's great. I'm interested in knowing it's internal functioning. How does it actually work?
Abhishek's user avatar
  • 183
7 votes
4 answers
293 views

Should coverage analysis be performed on test code?

I'm of the opinion that it should. Test code is meant to be deterministic. This means that 100% of it should be executed. If it's not 100% (and assuming the analysis tool isn't under-reporting), ...
Noel Yap's user avatar
  • 211
7 votes
2 answers
640 views

C++ coverage tool for QA personnel

Background: Our QA organization is considering using NCover Collector to gather code coverage information for manual and automated GUI tests (this is a desktop application). NCover will only work on ...
user1825216's user avatar
6 votes
4 answers
2k views

How would we know when we are over-testing?

I've recently heard the idea of over-testing for the first time while listening to the interview of the coverage.py maintainer on the Talk Python to Me podcast show. coverage.py has a new feature to ...
alecxe's user avatar
  • 11.4k
6 votes
4 answers
5k views

What to call a matrix with all possible combinations?

For example: The application show a list of results. Now we have a new feature: The list can be filtered. The user can use a search input field and a date range to filter the results. To cover all ...
Twaldigas's user avatar
  • 1,654
6 votes
2 answers
476 views

Is it a good practice to run functional tests with coverage?

Does it make sense to run once in a while, say, on a weekly basis, integration/functional tests against special build to have a coverage report. If anyone here has some practical experience of using ...
shabunc's user avatar
  • 361
6 votes
2 answers
612 views

Tool to minimize tests number and maximize code coverage

I am currently undergoing black-box testing campaigns for applications that require file input in linux environments. For that I need sets of test files of different formats that can be used to be ...
biggdman's user avatar
  • 161
6 votes
1 answer
1k views

Why Phantomjs got crashed while running more test cases?

I have written 751 test cases in Jasmine JavaScript. 116 test cases for Mobile View and 645 test cases on Desktop view. While executing test cases, most of the times, test case execution was not ...
Thiru Arasu's user avatar
6 votes
4 answers
530 views

Can code coverage be used for Java tests that work against a Python code base?

Development stack is in Python (flask) and our automation suite(API) is coded in Java. Can we do code coverage and if Yes, which plugin to use?
Aneesh Goel's user avatar
6 votes
3 answers
1k views

Measuring feature coverage with BDDs

I'm looking for approaches to measuring feature coverage (however you want to define that), and how BDDs can help (or be a neutral, or even negative factor) in such measurement. My question is: do ...
Marcin's user avatar
  • 191
5 votes
6 answers
2k views

How can 100% test coverage be "not enough to prove working code"?

It is known that 100% test/code coverage does not mean your code "works" as expected. What is generally meant by this statement and what use cases could demonstrate that 100% coverage is not enough?
alecxe's user avatar
  • 11.4k
5 votes
3 answers
539 views

What types of testing are LEAST suited to using pairwise testing (and similar test design approaches)?

For those of you who have used pairwise test design methods (or similar combinatorial software testing methods) in an attempt to zero in on a manageable number of unusually powerful tests from among a ...
Justin's user avatar
  • 2,357
5 votes
6 answers
3k views

Recommendations for code coverage tools?

I have a wishlist for code coverage tools and am wondering if people could recommend some tools that fulfill all or many of my wishes. I am starting my search for tools that work specifically with ....
Sam Woods's user avatar
  • 8,569
4 votes
2 answers
13k views

What are basic metrics of coverage with functional UI automation testing

There is web application product which is tested by test automation (selenium webdriver based test project in scope of the whole project) Client is asking what is the test coverage by automation of ...
user1829657's user avatar
4 votes
1 answer
2k views

What is a good code coverage tool for Ruby?

I would really like to see which bits of code/functionality I've missed testing in my application, which is a mix of Ruby and JQuery and Java applets. Years ago I used a nice code coverage tool for C ...
Jennifer Davis's user avatar
4 votes
6 answers
8k views

How can I measure the coverage of my E2E tests?

I'm looking for a way to measure the coverage of my end-to-end tests. I have some existing E2E tests already and I'm adding more test scenarios to it. But I would like to have a way to quantify the ...
xenon's user avatar
  • 151
4 votes
2 answers
4k views

How should a Black-box Tester measure Test Coverage?

I work in Product based company as a QA/Test Engineer. For ex:I have been assigned a new project/enhancement that would be delivered to the clients in the next production release. The Business Analyst ...
user avatar
4 votes
3 answers
125 views

Determine most depended-on code

Another title might be: determine strongest candidates for 100% test coverage. I'm looking for a tool that will display the methods/functions that are most depended on by my code base. To be clear, I'...
Philip's user avatar
  • 141
4 votes
4 answers
923 views

How to interpret / analyse code coverage results and provide suggestions for improvements?

Knowledge level I am looking for a how-to for a beginner, not so much for a expert or consultant answer like "it depends". Feel free to add the "it depends" parts nevertheless. Question How would ...
Lernkurve's user avatar
  • 187
4 votes
2 answers
5k views

Code Coverage for Web Applications?

Does anyone have any experience using code coverage tools for web applications? I primarily work with Webdriver and JUnit, but I am not sure if we are covering all the possible cases. This may ...
Ben's user avatar
  • 83
4 votes
1 answer
232 views

Does Emma provide true block coverage?

While trying to find the block coverage from Emma, I came across this curious observation. See the method below public double getBalance() { return this.balance; } The coverage summary from ...
Rahul Gopinath's user avatar
3 votes
5 answers
201 views

What are some good examples of pure Computer Science and Mathematics applications in software QA?

I am looking for some good examples of pure Computer Science and Mathematics applications in software QA. The objective is to create a course for CS and Applied Math students with applications, that ...
faleichik's user avatar
  • 131
3 votes
4 answers
1k views

Good resource on how to write dot net unit test cases?

I am looking for good resources on how to write good unit test cases in C#/Dot Net. Books, resources on the net or simply good advises. In particular the unit cases should give good code coverage and ...
Mikael Gislen's user avatar
3 votes
4 answers
132 views

Testing approaches with temporal delays

I am working with a system that has fairly complicated business logic. It has reasonably good unit, and integration tests. We recently ran into a bug which we had missed. This bug only became ...
Jeremy French's user avatar
3 votes
1 answer
2k views

ISTQB question: Statement Coverage and Decision Coverage with loop

I'm learning ISTQB and encountering a question about Statement and Decision coverage in a sample exam of ISTQB org. Evaluate the following control flow diagram and determine the ...
LearningandWorking's user avatar
3 votes
2 answers
338 views

Dynamic whitebox testing and coverage metrics for Java systems?

What coverage measures/metrics are useful for unit testing and for integration testing of Java or other object-oriented systems? Context: We would like to establish a better test process. Our ...
prockel's user avatar
  • 273
3 votes
2 answers
3k views

How to get Code coverage via external rest assured API automation using jacoco?

I have a multi module application "Project A". This application is hosted in clustered app nodes using tomcat as web server. We are using Spring, Web Services as primary among many other technologies ...
Prabhudatta Choudhury's user avatar
3 votes
1 answer
2k views

How to merge different karma coverage html reports into single html file?

I am new to TDD. we have already executed the test cases through karma-coverage.. If the test case count increases, the code coverage took more time to complete the job. so i had switched into ...
Thiru Arasu's user avatar
2 votes
4 answers
1k views

Is unit and component testing sufficient?

If you can test every line of [your product's] code via unit tests, wouldn't unit testing alone (theoretically) be sufficient? Are there other "necessary" benchmarks of proper test coverage other ...
ManRow's user avatar
  • 121
2 votes
1 answer
138 views

Working just on branch or line coverage could prevent us from finding which defect?

Achieving branch coverage is possible but it doesn't mean complete testing. What kind of defect you can find with this technique? what kind of risk you could have with this technique? what possible ...
Emna Ayadi's user avatar
2 votes
2 answers
207 views

What non-structural coverage criteria are there?

There are huge amounts of structural coverage criteria (see e.g. ISTQB-Standard or ISO26262-Standard), ranging from control-flow to data-flow coverage, applied to source code or to specifications. ...
DaveBall aka user750378's user avatar
2 votes
1 answer
3k views

ISTQB question: Statement and Decision coverage

READ A READ B READ C IF C>A THEN IF C>B THEN PRINT "C must be smaller bla bla.." ELSE PRINT "Proceed to next stage" ENDIF ELSE PRINT "B CAN BE SMALLER THEN C" ENDIF Could someone explain to me ...
Alexander Tunick's user avatar
2 votes
2 answers
64 views

Is running tests with coverage functionally the same as running without?

This is a follow-up to Is it a good practice to run functional tests with coverage? which made me confused a little bit. I was under the impression that typically a test run with enabled coverage ...
alecxe's user avatar
  • 11.4k
2 votes
1 answer
126 views

Comparing test coverage metrics to identify an increase/decrease after implementing automation

I would like to compare the levels of test coverage for projects before and after implementing an automation framework. I would like to know if there has been an increase or decrease in test coverage. ...
Pilauricer's user avatar
2 votes
1 answer
810 views

Examples for LOC figures, of test suites, and software projects?

To learn what kind of "software" project my automated testing team faces (I consider test automation a certain kind of software project), I assessed the size of our automated (functional, GUI-focussed)...
TheBlastOne's user avatar
2 votes
1 answer
877 views

Test coverage tool that shows call graph of untested new code from a merge

I'm looking for a test coverage tool that can show the call path/call graph of new untested code that came from a merge. Example: I have a git repo (on github or bitbucket) and a test suite that ...
Fladson Gomes's user avatar
2 votes
1 answer
1k views

Combining coverage from unit tests and functional tests/Selenium

Has anyone had experience getting a combined code coverage measure across both unit tests and functional/UI automation tests? I'm in a Java stack with Cobertura, Jenkins and Ant. I can successfully ...
wrschneider's user avatar
1 vote
4 answers
1k views

How to calculate condition coverage?

I am trying to prepare to test and I see the next exercise in internet: How many test cases are needed to achieve 100 % condition coverage? if ((temperature < 0) or (temperature > 100)) { ...
Dem0n13's user avatar
  • 121
1 vote
3 answers
212 views

Difference between Test Coverage AND code coverage?

My team asked me to measure the Test coverage percentage via a tool i choose ( how many tests from the app features are covered by automated testing ) , so i added the gitlab coverage badge and now i ...
USER_1234's user avatar
1 vote
3 answers
10k views

ISTQB Foundation Level Paper 3, no. 37

How many test cases are necessary to cover all the possible sequences of statements (paths) for the following program fragment? if (condition 1) then statement 1 else statement 2 fi if ...
Kristina's user avatar
1 vote
1 answer
2k views

How many more test cases are required to achieve provide 100% statement and 100% decision coverage?

You have designed test cases to provide 100% statement and 100% decision coverage for the following fragment of code. if width > length then biggest_dimension = width else biggest_dimension = ...
user16780334's user avatar
1 vote
1 answer
42 views

How to document the paths of my windows service in an easy to read diagram?

I have a windows service that issues thousands of android ADB commands to devices, I plan to go through the source for it and document all possible scenarios/paths, there are a lot of if x do y ...
symon's user avatar
  • 934
1 vote
2 answers
47 views

Should most of the Regression testing items be automated?

I work on a very complex financial product. We have very few Regression items/test cases automated. Like 20% is automated. Now, should I automate more and reach till 70-80 % coverage. Is it necessary ...
automation78's user avatar