Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

For example when creating software for critical real time applications, like those used in aviation, how certain are you that it's going to run without problems? Are you 100% certain or are you >90% certain?

We see that bug fixes are continuously added in every software application. What if the bug bites back before there is a chance to fix it,especially in mission critical applications?

So is a fault free application really possible and is exception handling the solution to uncertainty? How far can exception handling go?

share|improve this question
6  
"What if the bug bites back before there is a chance to fix it,especially in mission critical applications?" - then you hope the application has a safe failure mode. – Joe Strazzere Mar 16 at 12:46
2  
I hate this title, and love these answers below. – Baronz Mar 16 at 15:46
1  
Not going to add an answer since the ones already there are great. But as a curiosity, have you heard about Therac-25? It's the canonical example of a software bug with catastrophical consequences. – Marc.2377 Mar 17 at 12:40
    
An account of Therac 25 I once read ended by saying the company had given up making medical machines and gone back to their core business of nuclear reactor controllers. Gulp. – RedSonja Mar 17 at 14:47
    
Exception handling is required even if there aren't bugs. For example, you'd want to handle uncorrectable memory errors gracefully: lwn.net/Articles/348886 – gmatht Mar 18 at 5:46

10 Answers 10

up vote 30 down vote accepted

They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of estimating how many bugs remain in a system. There are at least two forms of static analysis.

If the system goes wrong then it goes wrong. There are assorted methods to reduce the impact of failures. Having code that checks for consistency and then abandoning a transaction or even restarting a process or the entire system is common One way of doing this uses assertions. To ensure systems restart after some types of failure Watchdog timers are frequently used.

share|improve this answer
1  
Of course, static analysis can be buggy :P In fact, this applies to any form of testing and analysis - in the end, someone wrote that software, someone assembled the proofs, someone forgot to account for this one platform where 1 + 1 doesn't always equal 2... there's plenty of opportunities for bugs. Manual testing isn't exempt, of course. – Luaan Mar 16 at 17:11
1  
Yes it can be bug-free see wikipedia article from my answer (for some definition of "bug" and "free"). Proving system correct is jut lots of work, buggy systems are cheaper. – Peter Masiar Mar 16 at 22:27
8  
@PeterMasiar If you define bug as not-meeting-the-specification, then you can with great effort prove that there are no areas where the program doesn't conform to the specification. What you can not do is prove that the specification itself does not contain any errors. This is an epistemological constraint, not a technical one. The observation that this is the case goes back to (at least) Wittgenstein's Philosophical Investigations, published in 1953. – Noah Sussman Mar 17 at 21:04
1  
@immibis OK, thank you. Please see the link to "bebugging" in my answer. See also this page that suggests "rebugging" is a derogatory term applied to work done by some software developers, urbandictionary.com/define.php?term=rebugging – AdrianHHH Mar 18 at 10:15
1  
@immibis Please accept my apologies. For many people on Stack Overflow English is a difficult language and not their first language. I try and write clearly and to use appropriate words. You used a word that appears reasonable, I did not remember its definition so I looked it up and found that it expressed something unexpected. Given that Urban Dictionary would you like one of your co-workers to say "Hey Immibis are you rebugging that program?" I would not like that statement said to me. – AdrianHHH Mar 19 at 17:32

NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, with multiple levels of review, using very small functions to minimize the risk in each, they managed to get it down to 1 defect per 10,000 lines of code (might've been 100k, I forget). This is, to this day, held as the most defect-free code ever written.

Their ultimate decision was that this mode of software production was not useful or usable to anyone on the planet: they couldn't write all their code like this, nor even all their mission-critical code, and even after the huge financial and time investment in perfectionism, they only managed to reduce the defects, not eradicate them.

In fact, they never again developed code to this standard, as it simply was not economical.

[Edit: I am wrong. In comments, @Benjaminssp provides a link ( http://www.fastcompany.com/28121/they-write-right-stuff) showing that the SATC has continued in its efforts towards zero-defect code, and has achieved an error rate that looks to be about 2 errors per MLoc - this is frankly mindblowing to me, but they show that it can be achieved! However, I feel my point still stands, though greatly weakened: even NASA cannot get complete freedom from defects.]

So: if you attempt to reassure a developer that their software is 100% defect free, then the developer will typically ask for another QA person. The best you can say is "Your code performs to spec, for the inputs we have tested it with, which achieved 100% code coverage... assuming our tests are correct."

Always remember that your testing code itself will have a few defects per thousand lines.

share|improve this answer
3  
+1 For tests also have faults if you check them in large. People just make mistakes :) – Niels van Reijmersdal Mar 16 at 13:54
1  
do you have a reference for this? – rdans Mar 16 at 15:27
1  
Here’s a source, though it doesn't say anything about the program's discontinuation: fastcompany.com/28121/they-write-right-stuff – Benjaminssp Mar 16 at 15:44
1  
@DewiMorgan And it's still absurdly expensive. Probably worth it dealing with something where a single error can mean losing a billion dollars in equipment, but oh well :) – Luaan Mar 16 at 17:07
1  
@Luaan Forget a billion dollars in equipment, how about half a dozen lives in a very high publicity accident? – Michael Kjörling Mar 17 at 12:40

All non-trivial software has bugs. Risk is part of life. It is foolhardy to attempt to remove all possible risk, because in the process, you introduce the risk that you will accomplish nothing.

Even if your algorithm and your implementation are perfect, your software does not live in a vaccuum: it depends on compilers, interpreters, libraries, operating systems, and other systems over which you have little or no control. On top of that, people need to use, configure, and manage your software in prescribed ways that you have probably not communicated adequately. Any Operations person will tell you that everything your system depends on is buggy, imperfect, and only sometimes reliable.

You may reply, "I live in a world where none of those things are my responsibility. All I care about is eliminating all bugs from my software." If that's really true, congratulations, and I hope you are still in business a year from now. More likely, those things are your problem too. Your customers/users do not care whether your system is broken because of a faulty algorithm or a poorly communicated configuration setting. If your job is to improve quality, your time may be better spent looking at this bigger context.

If you want to accomplish things, to get things done, you cannot eliminate all risk. Instead, you need to manage risk: to make trade-offs between how likely something is to go wrong, how much damage that thing will do, and how you recover from it. People who are good at building things that last -- including entire businesses -- get to be really good that that.

share|improve this answer
1  
I know we're not supposed to comment "me too!", but ME TOO! -- you're line "All non-trivial software has bugs" is exactly the line I find myself saying in meeting at least twice a week. It's an important mantra, and way to help focus testers on what faults are important to remediate because they fall within the standard operational profile. – Baronz Mar 16 at 15:45
2  
This also points the way to reducing bugs, of course: write more trivial code. The greater the complexity, the greater the error density. – Dewi Morgan Mar 16 at 17:05
2  
And trivial software has the biggest bug of them all: it doesn't solve a problem. Once you get to the point where your trivial software solves someone's problem, it's already at the point where it has bugs. At the very least, some user will think some behaviours are wrong (while others will assert the opposite) :P And don't even get me started on "environmental bugs". Good luck getting even the trivial program working correctly in all the environments where your users are going to run it. After all those years, a lot of american software still simply doesn't work in most european countries. – Luaan Mar 16 at 17:08
    
This is an excellent answer. I would like to add that even if the software was perfect, it is run on hardware. Hardware bugs and hardware errors exist. – emory Mar 16 at 19:03

Software isn't fault free.

When dealing with mission critical software, sure, you need to invest a lot more in quality assurance, and you need the best developers you can get, and you need to get them all the fancy tools and use them (including picking the best language for the job). With all that, you're still going to get bugs.

The main solution is basically a form of defense in depth.

Redundancy is a great thing. In widest sense, you want multiple independent systems that use multiple separate ways to get to the same answer. This way, if some path fails, you'll notice the error, and you will have a way to choose the correct answer, if you're lucky. This is used everywhere from the Shuttle to Nuclear power plants, and not just with software.

You also need to understand your failure modes. There might be a failure that doesn't have any long-reaching effects - that's a pretty mild failure. If there's a temporary issue that causes your program to crash "safely", restarting might be good enough. In many mission critical scenarios, this is perfectly fine, especially combined with redundancy above. If there's a possibility to corrupt saved state in a way that cannot be resolved with a restart, or if the same error can cause your system to fail repeatedly in a scale that doesn't fit the mission requirements, you're in way more trouble. The worst case of all is a silent failure - basically, something went wrong, but it didn't raise any alarms. This can cause long-term corruption of the data, and quite a bit of harm before being detected. Again, see redundancy above. There's a bit of overlap between the different failure modes - for example, a probe that did a longer burn than expected might be discovered as soon as we get the data back (light is slow), but by then it might have already been lost. Still, you want to know about failures, so silent failures are often very ugly.

Unsurprisingly, mission critical systems tend to be rather expensive for both development and deployment, and they have many additional constraints and usually a limited speed (for example, you may have three different algorithms cross-checking each other, but one of them is faster than others and so you could get a faster result by only checking that one). As always, engineering is about trade-offs.

share|improve this answer

Everybody keeps repeating that all non-trivial programs have bugs. That just shows how far from mature is software engineering as a field of study, and how big gap there is between computer science (solving theoretical problems right way) and software engineering (solving practical problems in a practical way).

There is whole body of research on formal program verification - which whole point is to prove that program is right (has no bugs).

Similar question was asked at math SO Why can't programs be proven?

Answer is: yes they can be proven correct but is is a lot of work. I recall that Dijkstra wrote simple OS and proven it works according to requirements but I cannot find a link.

Programs can be assured that they are bug free, if they will be proven by design (derived from requirements). Hardly anyone is doing it in commercial application because of the cost. It is cheaper to deal with the bugs than derive program (and prove it correct).

Of course testing cannot prove absence of bugs, only their presence.

share|improve this answer
6  
Formal program verification and proving properties can only verify that a program matches some particular formal requirements. This doesn't in any way ensure that this program is bug free, as a large class of real world bugs is program behavior that does exactly match the specified requirements, but in some particular situation these requirements don't match the actual business needs or user expectation - thus the system is buggy, broken and needs to be fixed despite theoretically being "correct". In essence, you simply go from worrying about bugs in code to bugs in the formal requirements. – Peteris Mar 16 at 20:17
1  
"Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth – Mark Mar 16 at 22:22
2  
If your requirements are detailed enough to allow deriving final code, then the form of those requirements is essentially just a high-level programming language. For a nontrivial system, the size and complexity of any somewhat detailed requirements will be huge, comparable to the resulting codebase, and the process of developing it is as bug-prone as developing normal code. – Peteris Mar 17 at 10:08
1  
A civil engineer can't prove that his bridge won't fall down either - all he can show is that it passes tests for a range of scenarios. Failing to grasp that software engineering is engineering, and applying the same standards to it, is a common failure amongst people who don't themselves work in software. Yes, there's a body of work on formal proof - and it is a classic example of ivory-tower academics losing track of the real world to produce a lovely theory which is unuseable in practise. – Graham Mar 17 at 10:51
3  
While you can use formal verification to prove that the program conforms to the specification, you cannot prove by any formal method that the specification does not contain any errors. You can prove conformance but you cannot prove correctness. – Noah Sussman Mar 17 at 21:10

Bug Free is a term that is thrown around loosely.

I am aware of one proven "bug free" product ever created, the seL4 microkernel. Someone managed to do an automated proof of the software which demonstrates that, if compiled by a standards-conforming compiler, the microkernel would function as described in its API documentation.

"High reliability software" strives to get close to bug free. They rely heavily on things like

  • Restriction of language features to those which are most easily verified
  • Strong peer review process
  • Rigorous testing

As you approach higher levels of reliability, you may also find that the needs of your particular field start to specify particular behaviors which must be more reliable than others. For example, I have seen software which needed to be resilient against random bit flips in memory. As a result, in some especially critical locations, they used a 32-bit word for a boolean, and carefully structured the values, neither of which was zero, to have certain properties that made it easier to detect bit flips.

In the end, the limit for ultra high reliability software is the hardware it runs on. As you approach that limit, reliability is better gained by mitigating hardware unreliability, rather than trying to develop software which works more reliably on ideal abstract machines.

share|improve this answer

If you are in the business of producing safety critical code, which I am, you learn how to:

  • write and interpret proper requirements (most bugs are caused by requirements)
  • write code following international coding standards
  • test your code rigorously; syntax, coverage, error handling, graceful degradation and many others
  • integrate your code rigorously
  • refactor mercilessly
  • conduct reviews
  • test it again
  • follow correct practices; planning, versioning, reviews, bug reports...
  • keep testing

There are many tools available for all these things. You can go on lots of courses. You use consultants if you don't know the answer. It's not impossible. Of course you never find all the bugs, but you can make sure your software doesn't kill anyone.

In my experience safety critical software takes about twice as long to produce as the other stuff.

share|improve this answer
    
As little as twice? – AdrianHHH Mar 17 at 20:12
    
"Of course you never find all the bugs, but you can make sure your software doesn't kill anyone." - so you cannot assure developers that their software is fault free. How can you know that your software won't kill anyone? – Joe Strazzere Mar 20 at 15:46
    
@Joe Strazzere Well, if it fails it stops gracefully with a message, instead of firing missiles randomly about the place. That sort of thing. – RedSonja Mar 21 at 12:00
    
@RedSonja - sure, but if you can "never find all the bugs", how do you know that one of these "unfound" bugs won't fire missiles randomly? I would assert that you actually cannot be sure. – Joe Strazzere Mar 21 at 22:22
    
@Joe Strazzere Well, I have to test every line and every possible combination of input data. I can state that it fulfils the requirements. I have signed off the test reports, and by German law if I let a bug through and someone dies, it's me who goes to prison. I had to be there when they did actual live firing tests. I am as sure as I can be. There may be bugs in the requirements - I found and reported quite a few inconsistencies. It won't fire rockets randomly. If you offend it it just does nothing until you talk nicely to it again. – RedSonja Mar 22 at 6:51

I don't think anyone can guarantee their software to be bug-free without lying to you.

However, there is something called formal verification where one can get a proof of correctness. When a piece of software is defined as "correct" in formal verification, that means the software conforms to the specifications that are set to it. See: https://en.wikipedia.org/wiki/Formal_verification


Example:

Take a simple state machine with two states: on and off. The specification for this state machine is that it can only go from on to off and vice-versa. In other words, there should not be any loops on either on or off.

Using a formal verification system, you can model your state machine and have a search space for every possible initial condition. For this example: the initial condition can be on or off. From here, you can search the search space for both starting conditions to verify that there is no string of events possible that will break your specification.

Note: This is NOT the same

share|improve this answer

There's no such a fault free software. Actually, there's one...it's your Hello World program, but you know it, nobody uses such program.

With that said, there's no way to remove all defects from application. However, there are practices to help prevent or detect problem as much as we can such as:

  • Code review
  • Unit test
  • Requirement analysis
  • Test driven development
share|improve this answer

Fault free programs exist, but they require bugixes as well

The question is logical and coherent, but it's based on a wrong assumption "You see there are no fault free programs, because bugfixes are being continuously made for any program". In reality a fault-free program still can cause problems and might require bugfixes. The program can be 100% validated to get only expected results, but you can't expect all the possible results in a real environment, nevertheless some of these results remain more desirable than others.

An example: 2 + 2 = ?

#include <iostream>

void main()
{
    int result;

    result = 2+2;
    std::cout << "The answer is " << result;
}

This program seems too simple to be bugged. However, there are still two potential problems to be fixed. What exactly? Try to figure out it yourself before reading the anwser.

.

The answer

The first one is that our program isn't standards-compliant. ANSI/ISO Standard C actually requires main() to return a value. This is primarily a maintainability and compatibility issue.

The second one is a functional issue though. Being launched from the command line in Unix it returns "The answer is 4" without a newline character:

~/program
The answer is 4$ _

Interaction with other programs, that relies on that, can cause problems. As you can see, it isn't about exception handling, it is about possible differences between desirable and actual results.

The code after all the bugfixes:

#include <iostream>

int main()
{
    int result;

    result = 2+2;
    std::cout << "The answer is " << result << '\n';
    return(0);
}
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.