34 votes
Accepted

How can developers be assured that their software is fault free?

They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of ...
user avatar
  • 1,453
29 votes

How can developers be assured that their software is fault free?

NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, ...
user avatar
21 votes

How can developers be assured that their software is fault free?

All non-trivial software has bugs. Risk is part of life. It is foolhardy to attempt to remove all possible risk, because in the process, you introduce the risk that you will accomplish nothing. ...
user avatar
  • 22.8k
18 votes
Accepted

How do you determine that your project's quality has increased over time?

A good starting point is to define what quality means in your context. And then find out ways how to measure it. It seems you want to measure something without saying what that something is. Having ...
user avatar
  • 4,278
17 votes

Good practices to audit test automation code

As others have said: code review. It is not uncommon for code like assert true == true to be used as a placeholder during test automation development (I personally would use assert true == false or ...
user avatar
  • 30.7k
17 votes
Accepted

How to deal with flaky tests that have intermittent failures?

Here's the general approach we're currently implementing in our team: Measure flakiness to identify unstable tests. One way is to move suspected tests from the main deployment pipeline into ...
user avatar
  • 10.3k
8 votes

Typical localization of bugs

Some of the things I've run into: Currency formats are locale dependent. You can also have issues with different locale-based keyboards. I've seen issues where a problem only occurs when the language ...
user avatar
  • 30.7k
7 votes

How can developers be assured that their software is fault free?

Software isn't fault free. When dealing with mission critical software, sure, you need to invest a lot more in quality assurance, and you need the best developers you can get, and you need to get ...
user avatar
  • 181
7 votes
Accepted

Pair-testing with a candidate during an interview

Yes, it would. I like teams that execute exploratory testing in pairs, I think it is easier to write charters, create mind-maps and decide on heuristics together than alone. It will keep you sharper, ...
user avatar
6 votes

Good practices to audit test automation code

Review test automation code. Run each automated test at least once before you write the code it will test. (Of course, this means you'll have to write the test code before you write the code it will ...
user avatar
  • 4,430
6 votes
Accepted

Detecting complicated CSS selectors

What parameters would you consider to determine if a CSS selector is resistant to a change? What makes a reliable CSS locator? Some factors I would consider: To start with, the basics are: ...
user avatar
6 votes

How to deal with flaky tests that have intermittent failures?

Here is a good general purpose article that deals with flaky tests: https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests What we find is that with image-driven ...
user avatar
  • 111
6 votes

Pair-testing with a candidate during an interview

Great idea. Some things that might help: Instead of pair testing, just explain the basic context and let the candidate figure it out. A big advantage here is you will get a feel of: How many ...
user avatar
  • 5,834
6 votes

How do you determine that your project's quality has increased over time?

For me it is about measurable faster delivery of valuable features, while reducing downtime of functionality. I think good products and teams go faster and faster overtime. This might be different ...
user avatar
5 votes
Accepted

What is a reasonable expectation for bugs upon release?

That's not really answerable. What's reasonable depends on the expected use of the software, and the time/money that's being invested. For example, for aeronautic software, what's viewed as ...
user avatar
5 votes

What is a reasonable expectation for bugs upon release?

Role of QA is not to find all the bugs. QA is there to establish a process of finding bugs, collecting and prioritizing them, to be fixed fixed by developers in order of business priorities. And ...
user avatar
5 votes

How can developers be assured that their software is fault free?

Everybody keeps repeating that all non-trivial programs have bugs. That just shows how far from mature is software engineering as a field of study, and how big gap there is between computer science (...
user avatar
5 votes
Accepted

Good practices to audit test automation code

Code review will do it. Management, senior managers need to be aware of the value of having a review. They have to actively allow effort and time for reviews. This is normally the hardest part. Some ...
user avatar
  • 9,640
4 votes

What is a reasonable expectation for bugs upon release?

There is an old gag about acceptable failure rates. The joke goes that an american company ordered some manufactured parts from a company in (a foreign country) The spec called for there to be "no ...
user avatar
  • 41
4 votes

What is a reasonable expectation for bugs upon release?

The most straightforward answer is: those that were found and were not fixed, those that were found and were labeled as features plus all that were not found. The ones in the first and second group ...
user avatar
  • 241
4 votes

Typical localization of bugs

You can save yourself a lot of time by knowing more about your application. First, many of the issues dealing with currency formats, number formats, measurement system, long and short date formats, ...
user avatar
  • 5,137
4 votes

Typical localization of bugs

We got burned by a limited input field for income. Because I live in germany, values up to 100000 € where perfectly ok for our testers. But the website got translated for countries like russia, where ...
user avatar
  • 41
4 votes

Good practices to audit test automation code

I didn't see this one, so I wanted to add it "Management Issue". Having someone who willingly left without ever doing real work is a management issue. They should have been fired by management for ...
user avatar
  • 1,663
4 votes

Detecting tests that will never fail

Another idea which we have not yet applied in practice is Mutation Testing. Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the ...
user avatar
  • 11.4k
4 votes

How to deal with flaky tests that have intermittent failures?

I normally follow this flowchart when dealing with a flaky test in general.
user avatar
  • 9,640
4 votes

Certify in writing that your software will work flawlessly

It should be clearly communicated that failure is an option to avoid the client having any misconceptions. There is no flawless or perfect software. There always be bugs, unexpected behaviors, and ...
user avatar
  • 11.4k
4 votes
Accepted

As functionality is not a quality requirement, can quality of software be judged based on functionality only?

Software quality measures whether requirements are satisfied or not. Requirements can be both functional or non-functional. Functional requirements meaning what software should do. It would include ...
user avatar
  • 1,795
3 votes
Accepted

Requirements confusion - Functional suitability as Non-functional characteristic (ISO25010)

At first let's look at the definition of functional requirement term from ISO/IEC 27000:2014 standard: functional requirement requirement that specifies a function that a system or system ...
user avatar
3 votes

Typical localization of bugs

Typical bugs which described in many qa forums are: text expansion, resulting in truncated strings GUI alterations, resulting in overlaps of GUI elements and controls or their misalignment automatic ...
user avatar
  • 769
3 votes

What is a reasonable expectation for bugs upon release?

There is, of course, a substantial difference between a customer finding a defect that the Test team had previously found (but which Management were happy to release into Live) and those which were ...
user avatar
  • 608

Only top scored, non community-wiki answers of a minimum length are eligible