How can developers be assured that their software is fault free?

Question

For example when creating software for critical real time applications, like those used in aviation, how certain are you that it's going to run without problems? Are you 100% certain or are you >90% certain?

We see that bug fixes are continuously added in every software application. What if the bug bites back before there is a chance to fix it,especially in mission critical applications?

So is a fault free application really possible and is exception handling the solution to uncertainty? How far can exception handling go?

Answer 1

They cannot be 100% sure of a bug free system. They can increase their confidence that the system is bug-free by use of static analysis and testing. Some people advocate bebugging as a way of estimating how many bugs remain in a system. There are at least two forms of static analysis.

If the system goes wrong then it goes wrong. There are assorted methods to reduce the impact of failures. Having code that checks for consistency and then abandoning a transaction or even restarting a process or the entire system is common One way of doing this uses assertions. To ensure systems restart after some types of failure Watchdog timers are frequently used.

Answer 2

NASA's Software Assurance Technology Center at Goddard once did a test to see how few defects they could get in some code for the shuttle. After a truly rigorous and vastly, vastly expensive process, with multiple levels of review, using very small functions to minimize the risk in each, they managed to get it down to 1 defect per 10,000 lines of code (might've been 100k, I forget). This is, to this day, held as the most defect-free code ever written.

Their ultimate decision was that this mode of software production was not useful or usable to anyone on the planet: they couldn't write all their code like this, nor even all their mission-critical code, and even after the huge financial and time investment in perfectionism, they only managed to reduce the defects, not eradicate them.

In fact, they never again developed code to this standard, as it simply was not economical.

[Edit: I am wrong. In comments, @Benjaminssp provides a link ( http://www.fastcompany.com/28121/they-write-right-stuff) showing that the SATC has continued in its efforts towards zero-defect code, and has achieved an error rate that looks to be about 2 errors per MLoc - this is frankly mindblowing to me, but they show that it can be achieved! However, I feel my point still stands, though greatly weakened: even NASA cannot get complete freedom from defects.]

So: if you attempt to reassure a developer that their software is 100% defect free, then the developer will typically ask for another QA person. The best you can say is "Your code performs to spec, for the inputs we have tested it with, which achieved 100% code coverage... assuming our tests are correct."

Always remember that your testing code itself will have a few defects per thousand lines.

Answer 3

All non-trivial software has bugs. Risk is part of life. It is foolhardy to attempt to remove all possible risk, because in the process, you introduce the risk that you will accomplish nothing.

Even if your algorithm and your implementation are perfect, your software does not live in a vaccuum: it depends on compilers, interpreters, libraries, operating systems, and other systems over which you have little or no control. On top of that, people need to use, configure, and manage your software in prescribed ways that you have probably not communicated adequately. Any Operations person will tell you that everything your system depends on is buggy, imperfect, and only sometimes reliable.

You may reply, "I live in a world where none of those things are my responsibility. All I care about is eliminating all bugs from my software." If that's really true, congratulations, and I hope you are still in business a year from now. More likely, those things are your problem too. Your customers/users do not care whether your system is broken because of a faulty algorithm or a poorly communicated configuration setting. If your job is to improve quality, your time may be better spent looking at this bigger context.

If you want to accomplish things, to get things done, you cannot eliminate all risk. Instead, you need to manage risk: to make trade-offs between how likely something is to go wrong, how much damage that thing will do, and how you recover from it. People who are good at building things that last -- including entire businesses -- get to be really good that that.

Answer 4

Software isn't fault free.

When dealing with mission critical software, sure, you need to invest a lot more in quality assurance, and you need the best developers you can get, and you need to get them all the fancy tools and use them (including picking the best language for the job). With all that, you're still going to get bugs.

The main solution is basically a form of defense in depth.

Redundancy is a great thing. In widest sense, you want multiple independent systems that use multiple separate ways to get to the same answer. This way, if some path fails, you'll notice the error, and you will have a way to choose the correct answer, if you're lucky. This is used everywhere from the Shuttle to Nuclear power plants, and not just with software.

You also need to understand your failure modes. There might be a failure that doesn't have any long-reaching effects - that's a pretty mild failure. If there's a temporary issue that causes your program to crash "safely", restarting might be good enough. In many mission critical scenarios, this is perfectly fine, especially combined with redundancy above. If there's a possibility to corrupt saved state in a way that cannot be resolved with a restart, or if the same error can cause your system to fail repeatedly in a scale that doesn't fit the mission requirements, you're in way more trouble. The worst case of all is a silent failure - basically, something went wrong, but it didn't raise any alarms. This can cause long-term corruption of the data, and quite a bit of harm before being detected. Again, see redundancy above. There's a bit of overlap between the different failure modes - for example, a probe that did a longer burn than expected might be discovered as soon as we get the data back (light is slow), but by then it might have already been lost. Still, you want to know about failures, so silent failures are often very ugly.

Unsurprisingly, mission critical systems tend to be rather expensive for both development and deployment, and they have many additional constraints and usually a limited speed (for example, you may have three different algorithms cross-checking each other, but one of them is faster than others and so you could get a faster result by only checking that one). As always, engineering is about trade-offs.

Answer 5

Everybody keeps repeating that all non-trivial programs have bugs. That just shows how far from mature is software engineering as a field of study, and how big gap there is between computer science (solving theoretical problems right way) and software engineering (solving practical problems in a practical way).

There is whole body of research on formal program verification - which whole point is to prove that program is right (has no bugs).

Similar question was asked at math SO Why can't programs be proven?

Answer is: yes they can be proven correct but is is a lot of work. I recall that Dijkstra wrote simple OS and proven it works according to requirements but I cannot find a link.

Programs can be assured that they are bug free, if they will be proven by design (derived from requirements). Hardly anyone is doing it in commercial application because of the cost. It is cheaper to deal with the bugs than derive program (and prove it correct).

Of course testing cannot prove absence of bugs, only their presence.

Answer 6

If you are in the business of producing safety critical code, which I am, you learn how to:

• write and interpret proper requirements (most bugs are caused by requirements)
• write code following international coding standards
• test your code rigorously; syntax, coverage, error handling, graceful degradation and many others
• integrate your code rigorously
• refactor mercilessly
• conduct reviews
• test it again
• follow correct practices; planning, versioning, reviews, bug reports...
• keep testing

There are many tools available for all these things. You can go on lots of courses. You use consultants if you don't know the answer. It's not impossible. Of course you never find all the bugs, but you can make sure your software doesn't kill anyone.

In my experience safety critical software takes about twice as long to produce as the other stuff.

Answer 7

Bug Free is a term that is thrown around loosely.

I am aware of one proven "bug free" product ever created, the seL4 microkernel. Someone managed to do an automated proof of the software which demonstrates that, if compiled by a standards-conforming compiler, the microkernel would function as described in its API documentation.

"High reliability software" strives to get close to bug free. They rely heavily on things like

• Restriction of language features to those which are most easily verified
• Strong peer review process
• Rigorous testing

As you approach higher levels of reliability, you may also find that the needs of your particular field start to specify particular behaviors which must be more reliable than others. For example, I have seen software which needed to be resilient against random bit flips in memory. As a result, in some especially critical locations, they used a 32-bit word for a boolean, and carefully structured the values, neither of which was zero, to have certain properties that made it easier to detect bit flips.

In the end, the limit for ultra high reliability software is the hardware it runs on. As you approach that limit, reliability is better gained by mitigating hardware unreliability, rather than trying to develop software which works more reliably on ideal abstract machines.

Answer 8

I don't think anyone can guarantee their software to be bug-free without lying to you.

However, there is something called formal verification where one can get a proof of correctness. When a piece of software is defined as "correct" in formal verification, that means the software conforms to the specifications that are set to it. See: https://en.wikipedia.org/wiki/Formal_verification

---

Example:

Take a simple state machine with two states: on and off. The specification for this state machine is that it can only go from on to off and vice-versa. In other words, there should not be any loops on either on or off.

Using a formal verification system, you can model your state machine and have a search space for every possible initial condition. For this example: the initial condition can be on or off. From here, you can search the search space for both starting conditions to verify that there is no string of events possible that will break your specification.

Note: This is NOT the same

Answer 9

There's no such a fault free software. Actually, there's one...it's your Hello World program, but you know it, nobody uses such program.

With that said, there's no way to remove all defects from application. However, there are practices to help prevent or detect problem as much as we can such as:

• Code review
• Unit test
• Requirement analysis
• Test driven development

Answer 10

Fault free programs exist, but they require bugixes as well

The question is logical and coherent, but it's based on a wrong assumption "You see there are no fault free programs, because bugfixes are being continuously made for any program". In reality a fault-free program still can cause problems and might require bugfixes. The program can be 100% validated to get only expected results, but you can't expect all the possible results in a real environment, nevertheless some of these results remain more desirable than others.

An example: 2 + 2 = ?

#include <iostream>

void main()
{
    int result;

    result = 2+2;
    std::cout << "The answer is " << result;
}

This program seems too simple to be bugged. However, there are still two potential problems to be fixed. What exactly? Try to figure out it yourself before reading the anwser.

.

The answer

The first one is that our program isn't standards-compliant. ANSI/ISO Standard C actually requires main() to return a value. This is primarily a maintainability and compatibility issue.

The second one is a functional issue though. Being launched from the command line in Unix it returns "The answer is 4" without a newline character:

~/program
The answer is 4$ _

Interaction with other programs, that relies on that, can cause problems. As you can see, it isn't about exception handling, it is about possible differences between desirable and actual results.

The code after all the bugfixes:

#include <iostream>

int main()
{
    int result;

    result = 2+2;
    std::cout << "The answer is " << result << '\n';
    return(0);
}