34 votes
Accepted

How can developers be assured that their software is fault free?

They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of ...
  • 1,453
29 votes

How can developers be assured that their software is fault free?

NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, ...
21 votes

How can developers be assured that their software is fault free?

All non-trivial software has bugs. Risk is part of life. It is foolhardy to attempt to remove all possible risk, because in the process, you introduce the risk that you will accomplish nothing. ...
  • 23k
18 votes
Accepted

How do you determine that your project's quality has increased over time?

A good starting point is to define what quality means in your context. And then find out ways how to measure it. It seems you want to measure something without saying what that something is. Having ...
  • 4,358
17 votes

Good practices to audit test automation code

As others have said: code review. It is not uncommon for code like assert true == true to be used as a placeholder during test automation development (I personally would use assert true == false or ...
  • 30.8k
17 votes
Accepted

How to deal with flaky tests that have intermittent failures?

Here's the general approach we're currently implementing in our team: Measure flakiness to identify unstable tests. One way is to move suspected tests from the main deployment pipeline into ...
  • 10.4k
7 votes

How can developers be assured that their software is fault free?

Software isn't fault free. When dealing with mission critical software, sure, you need to invest a lot more in quality assurance, and you need the best developers you can get, and you need to get ...
  • 181
7 votes
Accepted

Pair-testing with a candidate during an interview

Yes, it would. I like teams that execute exploratory testing in pairs, I think it is easier to write charters, create mind-maps and decide on heuristics together than alone. It will keep you sharper, ...
6 votes

Good practices to audit test automation code

Review test automation code. Run each automated test at least once before you write the code it will test. (Of course, this means you'll have to write the test code before you write the code it will ...
  • 4,460
6 votes
Accepted

Detecting complicated CSS selectors

What parameters would you consider to determine if a CSS selector is resistant to a change? What makes a reliable CSS locator? Some factors I would consider: To start with, the basics are: ...
6 votes

How to deal with flaky tests that have intermittent failures?

Here is a good general purpose article that deals with flaky tests: https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests What we find is that with image-driven ...
  • 111
6 votes

Pair-testing with a candidate during an interview

Great idea. Some things that might help: Instead of pair testing, just explain the basic context and let the candidate figure it out. A big advantage here is you will get a feel of: How many ...
  • 5,834
6 votes

How do you determine that your project's quality has increased over time?

For me it is about measurable faster delivery of valuable features, while reducing downtime of functionality. I think good products and teams go faster and faster overtime. This might be different ...
5 votes

How can developers be assured that their software is fault free?

Everybody keeps repeating that all non-trivial programs have bugs. That just shows how far from mature is software engineering as a field of study, and how big gap there is between computer science (...
5 votes
Accepted

What is a reasonable expectation for bugs upon release?

That's not really answerable. What's reasonable depends on the expected use of the software, and the time/money that's being invested. For example, for aeronautic software, what's viewed as ...
5 votes

What is a reasonable expectation for bugs upon release?

Role of QA is not to find all the bugs. QA is there to establish a process of finding bugs, collecting and prioritizing them, to be fixed fixed by developers in order of business priorities. And ...
5 votes
Accepted

Good practices to audit test automation code

Code review will do it. Management, senior managers need to be aware of the value of having a review. They have to actively allow effort and time for reviews. This is normally the hardest part. Some ...
  • 9,740
4 votes

What is a reasonable expectation for bugs upon release?

There is an old gag about acceptable failure rates. The joke goes that an american company ordered some manufactured parts from a company in (a foreign country) The spec called for there to be "no ...
  • 41
4 votes

What is a reasonable expectation for bugs upon release?

The most straightforward answer is: those that were found and were not fixed, those that were found and were labeled as features plus all that were not found. The ones in the first and second group ...
  • 241
4 votes

Good practices to audit test automation code

I didn't see this one, so I wanted to add it "Management Issue". Having someone who willingly left without ever doing real work is a management issue. They should have been fired by management for ...
  • 1,663
4 votes

Detecting tests that will never fail

Another idea which we have not yet applied in practice is Mutation Testing. Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the ...
  • 11.4k
4 votes

How to deal with flaky tests that have intermittent failures?

I normally follow this flowchart when dealing with a flaky test in general.
  • 9,740
4 votes

Certify in writing that your software will work flawlessly

It should be clearly communicated that failure is an option to avoid the client having any misconceptions. There is no flawless or perfect software. There always be bugs, unexpected behaviors, and ...
  • 11.4k
4 votes
Accepted

As functionality is not a quality requirement, can quality of software be judged based on functionality only?

Software quality measures whether requirements are satisfied or not. Requirements can be both functional or non-functional. Functional requirements meaning what software should do. It would include ...
  • 1,795
3 votes

Categorizing possible improvements

I would like to mention two hints. I never got in touch with planning tasks in a professional way, so my ideas are more like how I would do it. Thus, its difficult to give you a concrete list of ...
3 votes

How quality relates to functionality? (it is not considered by quality models)

To improve quality for a software product we are splitting up quality into internal and external quality. Spread over three quality groups. It gives a more detailed view on the different quality ...
3 votes
Accepted

Requirements confusion - Functional suitability as Non-functional characteristic (ISO25010)

At first let's look at the definition of functional requirement term from ISO/IEC 27000:2014 standard: functional requirement requirement that specifies a function that a system or system ...
3 votes

What is a reasonable expectation for bugs upon release?

There is, of course, a substantial difference between a customer finding a defect that the Test team had previously found (but which Management were happy to release into Live) and those which were ...
  • 608
3 votes

Detecting tests that will never fail

Your situation is there because of Technical Debt. Everything you pointed out is controlled easily enough via Continual Review of your test suites. The programming world has words for this "...
3 votes

How to deal with flaky tests that have intermittent failures?

I like to setup a re-run strategy, where we rerun any tests that has failed, until it fails three times in a row. If the test is still failing after two times it will be a real failing test and not ...

Only top scored, non community-wiki answers of a minimum length are eligible