Skip to main content
18 votes
Accepted

How to deal with flaky tests that have intermittent failures?

Here's the general approach we're currently implementing in our team: Measure flakiness to identify unstable tests. One way is to move suspected tests from the main deployment pipeline into ...
dzieciou's user avatar
  • 10.5k
18 votes
Accepted

How do you determine that your project's quality has increased over time?

A good starting point is to define what quality means in your context. And then find out ways how to measure it. It seems you want to measure something without saying what that something is. Having ...
pavelsaman's user avatar
  • 4,548
17 votes

Good practices to audit test automation code

As others have said: code review. It is not uncommon for code like assert true == true to be used as a placeholder during test automation development (I personally would use assert true == false or ...
Kate Paulk's user avatar
  • 31.5k
7 votes
Accepted

Pair-testing with a candidate during an interview

Yes, it would. I like teams that execute exploratory testing in pairs, I think it is easier to write charters, create mind-maps and decide on heuristics together than alone. It will keep you sharper, ...
Niels van Reijmersdal's user avatar
6 votes

Good practices to audit test automation code

Review test automation code. Run each automated test at least once before you write the code it will test. (Of course, this means you'll have to write the test code before you write the code it will ...
Dale Emery's user avatar
  • 4,485
6 votes
Accepted

Detecting complicated CSS selectors

What parameters would you consider to determine if a CSS selector is resistant to a change? What makes a reliable CSS locator? Some factors I would consider: To start with, the basics are: ...
Michael Durrant's user avatar
6 votes

How to deal with flaky tests that have intermittent failures?

Here is a good general purpose article that deals with flaky tests: https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests What we find is that with image-driven ...
stevenz's user avatar
  • 111
6 votes

Pair-testing with a candidate during an interview

Great idea. Some things that might help: Instead of pair testing, just explain the basic context and let the candidate figure it out. A big advantage here is you will get a feel of: How many ...
FDM's user avatar
  • 5,894
6 votes

How do you determine that your project's quality has increased over time?

For me it is about measurable faster delivery of valuable features, while reducing downtime of functionality. I think good products and teams go faster and faster overtime. This might be different ...
Niels van Reijmersdal's user avatar
5 votes

How to deal with flaky tests that have intermittent failures?

I normally follow this flowchart when dealing with a flaky test in general.
Yu Zhang's user avatar
  • 9,942
5 votes
Accepted

Good practices to audit test automation code

Code review will do it. Management, senior managers need to be aware of the value of having a review. They have to actively allow effort and time for reviews. This is normally the hardest part. Some ...
Yu Zhang's user avatar
  • 9,942
4 votes

Good practices to audit test automation code

I didn't see this one, so I wanted to add it "Management Issue". Having someone who willingly left without ever doing real work is a management issue. They should have been fired by management for ...
mutt's user avatar
  • 1,693
4 votes

Detecting tests that will never fail

Another idea which we have not yet applied in practice is Mutation Testing. Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the ...
alecxe's user avatar
  • 11.4k
4 votes

Certify in writing that your software will work flawlessly

It should be clearly communicated that failure is an option to avoid the client having any misconceptions. There is no flawless or perfect software. There always be bugs, unexpected behaviors, and ...
alecxe's user avatar
  • 11.4k
4 votes
Accepted

As functionality is not a quality requirement, can quality of software be judged based on functionality only?

Software quality measures whether requirements are satisfied or not. Requirements can be both functional or non-functional. Functional requirements meaning what software should do. It would include ...
JAINAM's user avatar
  • 1,835
3 votes

Detecting tests that will never fail

Your situation is there because of Technical Debt. Everything you pointed out is controlled easily enough via Continual Review of your test suites. The programming world has words for this "...
John Peters's user avatar
3 votes

Categorizing possible improvements

I would like to mention two hints. I never got in touch with planning tasks in a professional way, so my ideas are more like how I would do it. Thus, its difficult to give you a concrete list of ...
Sascha Kretzschmann's user avatar
3 votes

How quality relates to functionality? (it is not considered by quality models)

To improve quality for a software product we are splitting up quality into internal and external quality. Spread over three quality groups. It gives a more detailed view on the different quality ...
Niels van Reijmersdal's user avatar
3 votes

How to deal with flaky tests that have intermittent failures?

I like to setup a re-run strategy, where we rerun any tests that has failed, until it fails three times in a row. If the test is still failing after two times it will be a real failing test and not ...
Niels van Reijmersdal's user avatar
3 votes

Pair-testing with a candidate during an interview

I have done something like this for a manual tester position. I whipped up a quick MVC site with 3 pages and put "issues" into each page (something like 11 issues total). I made it clear that I wasn't ...
kirbycope's user avatar
  • 4,090
3 votes

Pair-testing with a candidate during an interview

Quite interesting topic and although this question has been raised two years ago it is still a current topic! Hence I would like to share our experience / thoughts. Preparation from our side as ...
Daniel Boehm's user avatar
  • 1,183
3 votes

Certify in writing that your software will work flawlessly

Our job as testers is to assure quality and highlight any differences between the requirements and the final product... which means you can't (and shouldn't) guarantee that there are no defects. This ...
dvniel's user avatar
  • 2,548
3 votes
Accepted

Certify in writing that your software will work flawlessly

A client pays you for a job and on delivery he wants you to assure him in writing that your software will work flawlessly. It is not possible. Quality is a subjective characteristic - it depends on ...
João Farias's user avatar
  • 10.9k
3 votes

How do you determine that your project's quality has increased over time?

These are the metrics our company uses in priority order: Number of support cases per number of active users Number of errors generated and logged by our application This covers a number of ...
anichols's user avatar
3 votes
Accepted

preparation for technical/practical assessment both manual and automated testing for an SDET role at a dream company

This is a nasty interview process. Some things I'd suggest you consider: Be honest. You obviously want to work with this company, so make sure you can give them good reasons why they want to hire you....
Kate Paulk's user avatar
  • 31.5k
2 votes

Latest Research Topics in Software Quality Assurance

Testing usually falls to a tester/ test engineer, the designing and maintenance of test cases, writing necessary testing code and all the meta-work usually falls to a team lead such as an SQA engineer....
Ravi Ojha's user avatar
  • 395
2 votes

How can ISO 9000 and ISO 25000 Quality definition differ - parts included

The main difference it that requirements are what has been written down in the requirements documents, both the top level requirements document as supplied by the customer and in the derived ...
Steve Barnes's user avatar
2 votes
Accepted

Judge the project rating sonarQube

There is no golden standard for code quality statistics. Also older Stackoverflow questions about code duplication shows these topics are very opinionated. What good or bad statistics are will also ...
Niels van Reijmersdal's user avatar
2 votes

Suggest corner cases to test online assessment platform for slot booking

You can also test below cases. In our QA services company we create test cases in below manner: If the application used in countries observing Daylight saving, validate the slot booking time when ...
Anand's user avatar
  • 785
2 votes

Good practices to audit test automation code

Although code-reviews seems the most obvious one. I would like to vote for test-first and pair-testing. Test-first: Making sure you write a failing test first will go a long way preventing ...
Niels van Reijmersdal's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible