34
votes
Accepted
How can developers be assured that their software is fault free?
They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of ...
- 1,453
30
votes
How can developers be assured that their software is fault free?
NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, ...
- 451
18
votes
Accepted
How to deal with flaky tests that have intermittent failures?
Here's the general approach we're currently implementing in our team:
Measure flakiness to identify unstable tests. One way is to move suspected tests from the main deployment pipeline into ...
- 10.5k
18
votes
Accepted
How do you determine that your project's quality has increased over time?
A good starting point is to define what quality means in your context. And then find out ways how to measure it. It seems you want to measure something without saying what that something is.
Having ...
- 4,538
17
votes
Good practices to audit test automation code
As others have said: code review.
It is not uncommon for code like assert true == true to be used as a placeholder during test automation development (I personally would use assert true == false or ...
- 31.5k
7
votes
How can developers be assured that their software is fault free?
Software isn't fault free.
When dealing with mission critical software, sure, you need to invest a lot more in quality assurance, and you need the best developers you can get, and you need to get ...
- 181
7
votes
Accepted
Pair-testing with a candidate during an interview
Yes, it would. I like teams that execute exploratory testing in pairs, I think it is easier to write charters, create mind-maps and decide on heuristics together than alone. It will keep you sharper, ...
- 32.5k
6
votes
Good practices to audit test automation code
Review test automation code.
Run each automated test at least once before you write the code it will test. (Of course, this means you'll have to write the test code before you write the code it will ...
- 4,485
6
votes
Accepted
Detecting complicated CSS selectors
What parameters would you consider to determine if a CSS selector is resistant to a change? What makes a reliable CSS locator? Some factors I would consider:
To start with, the basics are:
...
- 25.2k
6
votes
How to deal with flaky tests that have intermittent failures?
Here is a good general purpose article that deals with flaky tests:
https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests
What we find is that with image-driven ...
- 111
6
votes
Pair-testing with a candidate during an interview
Great idea. Some things that might help:
Instead of pair testing, just explain the basic context and let the candidate figure it out. A big advantage here is you will get a feel of:
How many ...
- 5,904
6
votes
How do you determine that your project's quality has increased over time?
For me it is about measurable faster delivery of valuable features, while reducing downtime of functionality. I think good products and teams go faster and faster overtime. This might be different ...
- 32.5k
5
votes
Accepted
What is a reasonable expectation for bugs upon release?
That's not really answerable. What's reasonable depends on the expected use of the software, and the time/money that's being invested. For example, for aeronautic software, what's viewed as ...
- 3,370
5
votes
What is a reasonable expectation for bugs upon release?
Role of QA is not to find all the bugs. QA is there to establish a process of finding bugs, collecting and prioritizing them, to be fixed fixed by developers in order of business priorities. And ...
5
votes
Accepted
Good practices to audit test automation code
Code review will do it.
Management, senior managers need to be aware of the value of having a review. They have to actively allow effort and time for reviews. This is normally the hardest part. Some ...
- 9,932
5
votes
How can developers be assured that their software is fault free?
Everybody keeps repeating that all non-trivial programs have bugs. That just shows how far from mature is software engineering as a field of study, and how big gap there is between computer science (...
5
votes
How to deal with flaky tests that have intermittent failures?
I normally follow this flowchart when dealing with a flaky test in general.
- 9,932
4
votes
Good practices to audit test automation code
I didn't see this one, so I wanted to add it "Management Issue".
Having someone who willingly left without ever doing real work is a management issue. They should have been fired by management for ...
- 1,693
4
votes
What is a reasonable expectation for bugs upon release?
There is an old gag about acceptable failure rates. The joke goes that an american company ordered some manufactured parts from a company in (a foreign country) The spec called for there to be "no ...
- 41
4
votes
What is a reasonable expectation for bugs upon release?
The most straightforward answer is: those that were found and were not fixed, those that were found and were labeled as features plus all that were not found. The ones in the first and second group ...
- 241
4
votes
Detecting tests that will never fail
Another idea which we have not yet applied in practice is Mutation Testing.
Mutation testing (or Mutation analysis or Program mutation) is used to
design new software tests and evaluate the ...
- 11.4k
4
votes
Certify in writing that your software will work flawlessly
It should be clearly communicated that failure is an option to avoid the client having any misconceptions. There is no flawless or perfect software. There always be bugs, unexpected behaviors, and ...
- 11.4k
4
votes
Accepted
As functionality is not a quality requirement, can quality of software be judged based on functionality only?
Software quality measures whether requirements are satisfied or not. Requirements can be both functional or non-functional.
Functional requirements meaning what software should do. It would include ...
- 1,835
3
votes
Categorizing possible improvements
I would like to mention two hints.
I never got in touch with planning tasks in a professional way, so my
ideas are more like how I would do it.
Thus, its difficult to give you a concrete list of ...
3
votes
How quality relates to functionality? (it is not considered by quality models)
To improve quality for a software product we are splitting up quality into internal and external quality. Spread over three quality groups. It gives a more detailed view on the different quality ...
- 32.5k
3
votes
What is a reasonable expectation for bugs upon release?
There is, of course, a substantial difference between a customer finding a defect that the Test team had previously found (but which Management were happy to release into Live) and those which were ...
- 618
3
votes
Detecting tests that will never fail
Your situation is there because of Technical Debt. Everything you pointed out is controlled easily enough via Continual Review of your test suites. The programming world has words for this "...
- 521
3
votes
How to deal with flaky tests that have intermittent failures?
I like to setup a re-run strategy, where we rerun any tests that has failed, until it fails three times in a row. If the test is still failing after two times it will be a real failing test and not ...
- 32.5k
3
votes
Pair-testing with a candidate during an interview
I have done something like this for a manual tester position. I whipped up a quick MVC site with 3 pages and put "issues" into each page (something like 11 issues total). I made it clear that I wasn't ...
- 4,090
Only top scored, non community-wiki answers of a minimum length are eligible