54

I call them masked bugs. Bug 1 (the erroneous subtraction) is masked by bug 2 (the erroneous addition). Ideally, they would be caught by unit testing - since unit tests should be reporting whether or not each module returns the correct value, and both units are behaving incorrectly. Since it's not an ideal world, the next best thing is to step through the ...


28

An auditor or accountant would describe this as an incidence of "Compensating Errors" - it's probably something which is much more common in both of those fields. I think the term describes the situation and implies the challenge in trouble-shooting and resolution quite nicely.


22

I've always said such code "works by coincidence". If you say the code is broken or has a bug, people tend to push back because it appears to work to a casual observer. "Works by coincidence" acknowledges that the code does provide the intended result, but for the wrong reasons. People generally understand that a "coincidence" is a not something that can ...


17

You have intermittent failures Welcome to the crowd. In my experience they are the norm and the bane of Quality Engineering. So first off - accept that and start making resources available for it. Some techniques that you can use: Allow sufficient resources. Be prepared to need to take days/weeks to fix intermittent failures Collect examples. Some ...


13

"How to Investigate Intermittent Problems" by James Bach provides a very thorough treatment of the subject. Here's an outline of his 92 suggestions: If the bug exists, it has a cause: Possibility 1: The system is NOT behaving differently. The apparent intermittence is an artifact of the observation. Possibility 2: The system behaved differently ...


11

I probably first encountered this in the perl community, where the advice to use warnings "all"; and use strict; is so common. My guess is that I first read this in "effective perl programming" by Randal Schwartz - looks like it is on page 146. That book was published in 1998. I have had a lot of success with that approach for pointed and small programs, ...


8

An early tool which generated warnings, with the intention to have them fixed by the programmer as if they were errors, was Lint. Stephen Johnson. Lint, a C program checker. Computer Science Technical Report 65, Bell Laboratories, December 1977. Reference shamelessly copied from: https://en.wikipedia.org/wiki/Lint_(software) Note that the mentioned ...


8

I totally agree with Neils, and I'd like to address the question of whether or not this post belongs on this Testing site. Good testing starts long before any code is written, especially when it comes to the job of preventing error and simplifying future testing. Some basic testing principle apply regarless of the language used or application developed, ...


6

I would suggest also: Try logging in on your friends computer and seeing if you can upload from there. Try having your friend log in on your computer and uploading a pic from there Try using (send to other person actual file) the exact same image for both accounts Check browser is supported including version Compare image file types to make sure they are ...


6

Analyse from different perspectives to find the exact cause I am assuming that you are trying to debug the issue in the same environment where it was reported. Few things that should be considered while reproducing the issues: Pile on more logs: Try to capture as much as you can. This is the first thing you can rely on to get some valuable input on such ...


5

Apart from the technological aspects, you might need to consider organizational aspects as well. You will, most likely, have a ticket (a bug report) which remains open/assigned (unresolved) for quite a long time (like several months). In some organizations, this may make you look like a bad performer. In such a case, find a suitable solution with your boss ...


5

Here is what I instruction my developers to do: 1) add logging; allow the user to select advanced logging. If turning on logging "solves" the problem then it is a timing issue. We add logging to the location in the code that we suspect is involved in the problem the user is seeing. 2) have user take a video of what they are doing when the problem occurs. ...


5

Achieving branch coverage is possible but it doesn't mean complete testing. Indeed, no amount of fact-checking can achieve complete testing, because fact-checking is only one of the activities of testing, which also includes modeling, learning, experimentation, etc. (Reference) What kind of defect you can find with this technique? Code and mutation ...


5

I simply call them bugs that cancel each other, and I doubt if there is any special way to find or avoid them, and whether any special research was done on it (I tried Google Scholar and haven't found). The only place where it might have some importance is for building statistical models of the product, they usually assume a single point of failure. But ...


4

If anything, it was the reverse of this -- the culture started letting errors and warnings stay in, more and more. Back in punch card days, access to the punch card machines to read your deck of cards -- program -- was limited, often very expensive, and the deck often had to go through "verifiers," people who looked it over for errors. Even with a lightly ...


4

There are two that come to mind: Therac-25 and the Northeast Blackout of 2003. The Therac-25 was a radiation therapy machine; there was a race condition that could cause the high-power electron beam to trigger instead of the low power beam. Three people died as a result. When I was in college, the Therac-25 bug was brought up when we were learning about ...


4

Excellent question. Debugging is a basic skill programmers should have. Many testers lack programming skills because companies throw people with no or only little programming experience into test automation and expect them to learn such skills themselves. Debugging is then one of many programming skills they lack. For instance, the problem with ...


4

In addition to the great answer Michael provided I would say the following. Frankly speaking the question misses some important specific like "on which exactly step does the process fail". Since you described some high-level e2e case, the failure might appear not only as the submittion result. For example the button might not work because you are using ...


4

I've called this "an even number of bugs". I believe that the term can be attributed to my Calculus professor, who once made "an even number of mistakes" on the board. Examples of "an even number of mistakes" could include: Suggesting that the driver turn left when you meant right, but she turns right by mistake because she gets her directions confused. ...


4

In quantum chemistry, a related phenomenon is called "cancellation of errors"


3

Here are some ideas: Strongly agree with logging! With time stamps including milliseconds (big difference between events A at 01:23:45.999 and B at 01:23:46.000, vs. A at 01:23:45.000 and B at 01:23:46.999). Add bug-specific logging: if you can detect when a bug has occurred, log stuff. Circular buffers provide low-cost storage for history. Object self-...


3

I wonder if it is relevant who said it first. I expect it historically to be more in line with many different teams running into similar issues with warnings and errors before they wrote it down as a rule in a book. Who said it first will probably be a lot of separate people in different programming teams around the globe. I think this is a logical rule each ...


3

One part of the Ruby on Rails history over the last ten years is that frequently changes are marked with deprecation warning messages in one version before backward compatibility is removed altogether in the next version. So experienced folks in this community have learn to ignore those warnings at their peril. Over 30 years my personal experience has been ...


3

If you are dealing with the case @jonsharpe described above, you can create examples with the components in isolation. If they are not decoupled, you probably have a testability problem that is hiding these bugs.


3

Two errors which perfectly cancel each other out are possible only in the most theoretical sense. On any real computation machine, such a perfect cancellation is not possible. Consider the very simple example of a function f(x) which takes an integer as input and doubles it. The domain of this function is [-INT_MAX/2, INT_MAX/2]. Now, consider another ...


3

Most of the times that I have met this sort of thing it has not been bug in function Y cancels but in function X - it has been bug in function X compensated for by incorrect patch in function Y, i.e. somebody notice that the output of Y was wrong so fixed the symptom rather than realising that the problem was in X. This gets to be a problem when one of: ...


2

Practices of an Agile Developer, Subramaniam & Hunt, 2006. Page 137 has this... Just because your compiler treats warnings lightly doesn't mean you should. Treat warnings as errors. Checking in code with warnings is just as bad as checking in code with errors or code that fails its tests. No checked-in code should produce any warnings from the build ...


2

Steve McConnell writes in Code Complete (1993): p69: Eliminate the causes of all compiler errors and warnings. Pay attention to what the compiler tells you about your code. A lot of warnings often indicates low-quality code, and you should try to understand each warning you get. In practice, warnings you've seen again and again have one of two possible ...


2

Since you asked for anecdotal, this is about as anecdotal as it gets. This is a very opinion based question. First: Fix all warnings and errors. Do not tolerate any error messages Can be impractical, depending on the size/scope of an application. Some applications have many, many pieces, and you/your team are not in control of all of them. They will ...


2

There is no simple answer to this, and no generic solution. A few heuristic I have used over the years are: Lion in the desert algorithm- assuming that the results are reproducible, remove parts of the environment until you can no longer see the problem. Repeat until you isolate the source. Possible problems: removing parts of the environment can still ...


Only top voted, non community-wiki answers of a minimum length are eligible