They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of estimating how many bugs remain in a system. There are at least two forms of static analysis.
If the system goes wrong then it goes wrong. There are assorted methods ...
NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, with multiple levels of review, using very small functions to minimize the risk in each, they managed to get it down to 1 defect per 10,000 lines of code (might'...
All non-trivial software has bugs. Risk is part of life. It is foolhardy to attempt to remove all possible risk, because in the process, you introduce the risk that you will accomplish nothing.
Even if your algorithm and your implementation are perfect, your software does not live in a vaccuum: it depends on compilers, interpreters, libraries, operating ...
As others have said: code review.
It is not uncommon for code like assert true == true to be used as a placeholder during test automation development (I personally would use assert true == false or assert fail or similar as my placeholder so my code could not pass until I was ready to write the correct assert, and so there was no chance I would forget the ...
Here's the general approach we're currently implementing in our team:
Measure flakiness to identify unstable tests. One way is to move suspected tests from the main deployment pipeline into quarantine, repeat execution of those tests multiple times for the same environment conditions and choose tests that were producing mixed results (see Martin Fowler's ...
Some of the things I've run into:
Currency formats are locale dependent.
You can also have issues with different locale-based keyboards. I've seen issues where a problem only occurs when the language and the keyboard are configured for a specific locale (this was Turkish language and Turkish keyboard).
For German, you'll want to watch your line breaks - ...
There are many ways to have someone "throw some testing" at your site. Which way is best for you depends on
Your ability to provide input to the process
Some folks just post something on a forum where professional testers hang out (like here, or sqaforums.com) and ask for people to "test" their site. The good: You get ...
Yes, it would. I like teams that execute exploratory testing in pairs, I think it is easier to write charters, create mind-maps and decide on heuristics together than alone. It will keep you sharper, you will go faster in executing and analysing the results.
I would like to find answers on the following questions during an interview:
Will your team like ...
If we agree in principle that
validation is regarded as "building the right software," and
verification is "building the software right," and
QC is quality control helps identify defects in the product via testing, and
QA is quality assurance intended to help prevent defects in the product through process management
then, one position is that both ...
Software isn't fault free.
When dealing with mission critical software, sure, you need to invest a lot more in quality assurance, and you need the best developers you can get, and you need to get them all the fancy tools and use them (including picking the best language for the job). With all that, you're still going to get bugs.
The main solution is ...
Review test automation code.
Run each automated test at least once before you write the code it will test. (Of course, this means you'll have to write the test code before you write the code it will test.)
Great idea. Some things that might help:
Instead of pair testing, just explain the basic context and let the candidate figure it out. A big advantage here is you will get a feel of:
How many questions does this person ask?
What type of questions? Relevant or not?
How independent and confident is this tester?
Quantifying the result isn't hard science, but:
That's not really answerable. What's reasonable depends on the expected use of the software, and the time/money that's being invested. For example, for aeronautic software, what's viewed as reasonable is perfection, and they spend a huge amount of time, and money, and engineering resource, on making the flight software perfect. And they fail. Same thing ...
Role of QA is not to find all the bugs. QA is there to establish a process of finding bugs, collecting and prioritizing them, to be fixed fixed by developers in order of business priorities. And provide business side with info about current and projected stability of the codebase, and if release is stable enough to be released withing the deadlines.
Here is a good general purpose article that deals with flaky tests:
What we find is that with image-driven testing tools (Sikuli, Kantu, Testplant...) flaky tests are easier to diagnose, as the screenshots tell you more easily what goes wrong, as opposed to dissecting ...
I'd suggest setting up a ipv6 only network (you'll need an ipv6 router and a DNS server - more if you want to test across subnets). Dual Stack can be a challenge to test, as fallback to v4 may mask v6 issues. An app (assuming it uses the network) should behave the same running on ipv4 or ipv6 networks.
You can also simply review the code to see if/where ip ...
You can save yourself a lot of time by knowing more about your application.
First, many of the issues dealing with currency formats, number formats, measurement system, long and short date formats, long and short time formats, calendar, first day of week, and also sort order are handled by the operating system if the developer is utilizing the OS APIs (...
We got burned by a limited input field for income. Because I live in germany, values up to 100000 € where perfectly ok for our testers. But the website got translated for countries like russia, where 100000 rubel is not really much....
There is an old gag about acceptable failure rates. The joke goes that an american company ordered some manufactured parts from a company in (a foreign country) The spec called for there to be "no more defective parts than 5 in 1000"
Some time later the parts arrive at the american company. 5 containers of 1000 finished parts each. then the driver of one ...
The most straightforward answer is: those that were found and were not fixed, those that were found and were labeled as features plus all that were not found. The ones in the first and second group should be documented and the customers should be aware of them and the ones in the last group should not be the elephant in the room.
QA is supposed to find bugs,...
Everybody keeps repeating that all non-trivial programs have bugs. That just shows how far from mature is software engineering as a field of study, and how big gap there is between computer science (solving theoretical problems right way) and software engineering (solving practical problems in a practical way).
There is whole body of research on formal ...
Code review will do it.
Management, senior managers need to be aware of the value of having a review. They have to actively allow effort and time for reviews. This is normally the hardest part. Some senior managers consider reviews to be waste of time.
Team leader, they need to supervise reviews to be carried out.
Team members need to perform technical ...
I didn't see this one, so I wanted to add it "Management Issue".
Having someone who willingly left without ever doing real work is a management issue. They should have been fired by management for not doing their job. Management should eliminate silos and enforce bad practices as well while creating an environment for success which would prevent this ...
Another idea which we have not yet applied in practice is Mutation Testing.
Mutation testing (or Mutation analysis or Program mutation) is used to
design new software tests and evaluate the quality of existing
software tests. Mutation testing involves modifying a program in small
Which in practice means that if there was a mutation/change (...
What parameters would you consider to determine if a CSS selector is resistant to a change? What makes a reliable CSS locator? Some factors I would consider:
To start with, the basics are:
Locator should not include page layout structure
Don't base selector on the actual page text
Work within your existing framework(s)
Prefer css over xpath for ...
It should be clearly communicated that failure is an option to avoid the client having any misconceptions. There is no flawless or perfect software. There always be bugs, unexpected behaviors, and corner cases.
how do you respond to a client request like that
You could give a test summary report where you may show what was tested, what activities were ...
At first let's look at the definition of functional requirement term from ISO/IEC 27000:2014 standard:
requirement that specifies a function that a system or system component must be able to perform
— So functional requirements are what software should do.
Then, coming back to
degree to which a ...
Typical bugs which described in many qa forums are:
text expansion, resulting in truncated strings
GUI alterations, resulting in overlaps of GUI elements and
controls or their misalignment
automatic hotkeys assignment, resulting in duplicated hotkey
hard-coded strings, resulting in untranslated strings
unsupported code pages, resulting in garbage
You didn't mention platform. On the process side for windows apps, you can strive to create IP agnostic applications See here or here for some more ideas / elaboration.
For other platforms, I would imagine there is an equally limited number of networking functions that can tie an application to IP versions - a simple script could help identify exactly where ...
There is, of course, a substantial difference between a customer finding a defect that the Test team had previously found (but which Management were happy to release into Live) and those which were missed by the Test team. As a tester, it's the latter which most concern me.
"Reasonable" is inherently subjective. For the sake of argument, I take "...