Questions tagged [flaky]

"Flaky" term is used to describe tests which produce unstable results

Filter by
Sorted by
Tagged with
0
votes
0answers
203 views

Transient Fix Detection Problem

If a fault, f_1 is fixed by a change c_1 it mean that the failure F(f_1) associated with the fault, f_1, disappears (i.e. ¬F (f_1)). However, to be more precise, we should take account of time: it ...
1
vote
1answer
52 views

Is there any tool to determine if a test is flaky?

I'm thinking about new approach to include autotests to CI/CD pipeline and my thought is to determine if a test is flaky before they are in pipeline. In this case an only way to do that is to setup ...
6
votes
5answers
374 views

Balancing readability and reliability in end-to-end testing

One of the things that often helps to make tests more reliable and to reduce flakiness is Explicit Waits - waiting for an element to be visible or clickable before interacting with it, waiting for a ...
6
votes
1answer
170 views

Test Quarantine in Protractor

Follow-up to How to deal with flaky tests? One of the approaches to tackle flaky tests is to temporarily quarantine the tests - move them to a special "bucket" with other flaky tests: I offer one ...
18
votes
6answers
7k views

How to deal with flaky tests that have intermittent failures?

Having flaky tests that sometimes fail and sometimes pass is very frustrating: there is no single reason each of the flaky tests fail the failures don't seem to happen to real users using the ...
2
votes
1answer
3k views

Rerun failed (flaky) tests with Jest

We are using Jest as our test runner for our JavaScript (React and Selenium) tests. We have one or two flaky tests (as TeamCity marks them flaky and we process them as technical-debt), although we ...