What is it called when several bugs cancel each other out?

Question

Is there any name for a situation where two bugs cancel each other out and in the result, the software works as expected? For example, suppose software A calculates a value and mistakenly value X is subtracted from the input somewhere and exactly the same value (X) is added to value again mistakenly. So we have two independent bugs, but magically, they cancel out their side-effect and the result of calculation looks correct in the end.

I'm asking this question because I've found this type of bugs one of the most confusing and the hardest to find. It has happened to me a few times when I was working on something else and accidentally spotted an error/bug/mistake. At first, you scratch your head asking yourself how it has been working so far then? It took a while until you realized another bug has been cancelling out this one. I was wondering if there is a category for this type of bugs and they have been studied methodically. I like to learn more about them and how to avoid/find them.

Answer 1

I call them masked bugs. Bug 1 (the erroneous subtraction) is masked by bug 2 (the erroneous addition).

Ideally, they would be caught by unit testing - since unit tests should be reporting whether or not each module returns the correct value, and both units are behaving incorrectly.

Since it's not an ideal world, the next best thing is to step through the process as much as the tools available to you permit. For instance, when I'm dealing with a new feature, I will usually start with entering basic data, then check that the data I enter is correctly saved to the database. Then I start exploring boundaries and validation scenarios. I don't try to run the full end-to-end scenario until I've stepped through and covered as much as I can in the smallest steps the application.

Masked bugs are not rare. They're not common, but they happen often enough that most testers have run into quite a few of them. Masked bugs that are cancelled out (as opposed to just hidden) by the masking bug are a lot rarer, but they still happen. I recall (not fondly) a set of bugs that I ran into once, where each more specific instance was masked by the general instance that I discovered first. I think there ended up being 4 separate bugs that got found and fixed, each one more specific than the previous masking bug.

Answer 2

An auditor or accountant would describe this as an incidence of "Compensating Errors" - it's probably something which is much more common in both of those fields. I think the term describes the situation and implies the challenge in trouble-shooting and resolution quite nicely.

Answer 3

I've always said such code "works by coincidence".

If you say the code is broken or has a bug, people tend to push back because it appears to work to a casual observer. "Works by coincidence" acknowledges that the code does provide the intended result, but for the wrong reasons. People generally understand that a "coincidence" is a not something that can safely be relied on and they tend to be more likely to devote the resources to fix the problem (in my experiences, at least).

A lot of times, these sorts of bugs are the result of testing things at too high of a level. If you break up your tests into more narrow, focused tests that isolate specific units of code, then your two offsetting bugs are more likely to end up in separate units/modules and will, therefore, be more visible.

That won't catch all of them, though. If the bugs are close enough together (as in your case, where they're both parts of the implementation of a single calculation) then testing alone may never be able to catch them individually. That's where things like code review come into play. A fresh pair of eyes tend to be fairly good at spotting things that are out of place. For your particular example, a static analyzer might be able to flag the -X and +X as self-defeating and able to be optimized away.

Answer 4

I simply call them bugs that cancel each other, and I doubt if there is any special way to find or avoid them, and whether any special research was done on it (I tried Google Scholar and haven't found).

The only place where it might have some importance is for building statistical models of the product, they usually assume a single point of failure. But again I doubt if it is important since this kind of bugs is usually rare.

Answer 5

I've called this "an even number of bugs". I believe that the term can be attributed to my Calculus professor, who once made "an even number of mistakes" on the board.

Examples of "an even number of mistakes" could include:

• Suggesting that the driver turn left when you meant right, but she turns right by mistake because she gets her directions confused.
• Not noticing the message that company is bringing their spouse, so you do not prepare enough food, then somebody cancels.

I think that this happens a bit too often in my life!

^{Image: https://www.dreamstime.com}

Answer 6

In quantum chemistry, a related phenomenon is called

"cancellation of errors"

Answer 7

If you are dealing with the case @jonsharpe described above, you can create examples with the components in isolation.

If they are not decoupled, you probably have a testability problem that is hiding these bugs.

Answer 8

Two errors which perfectly cancel each other out are possible only in the most theoretical sense. On any real computation machine, such a perfect cancellation is not possible.

Consider the very simple example of a function f(x) which takes an integer as input and doubles it. The domain of this function is [-INT_MAX/2, INT_MAX/2].

Now, consider another function g(x) which does the same thing, but buried somewhere in the code of g(x) are two bugs: the first bug adds 1000 to the result for no reason, and then the second bug subtracts away 1000 from the result for no reason.

You might say that these two bugs cancel each other out, and therefore function f(x) and function g(x) are identical. But they are not identical functions. There is a subtle difference. The two compensating errors change the domain of function g(x). With the addition of 1000, it can no longer handle the upper bound INT_MAX/2 of the domain without numerical overflow error. The domain of this function is actually [-INT_MAX/2, INT_MAX/2-500].

So the two functions are not exactly the same.

I don't think you could come up with an example of two functions, one with a pair of compensating errors and the other without the pair of compensating errors, which are exactly the same in all respects.

The difference introduced by a pair of compensating errors might be subtle, but it is detectable.

---

For those saying, "That's nice. So what?"
Consider: A careful, exhaustive analysis of function g(x) would reveal that its domain is [-INT_MAX/2, INT_MAX/2-500] which is not what we would expect for an integer doubling function. Thus, it is still "buggy" despite the pair of (supposedly) mutually-canceling errors.

Answer 9

Most of the times that I have met this sort of thing it has not been bug in function Y cancels but in function X - it has been bug in function X compensated for by incorrect patch in function Y, i.e. somebody notice that the output of Y was wrong so fixed the symptom rather than realising that the problem was in X.

This gets to be a problem when one of:

1. Somebody spots and fixes one but not the other(s)
2. It turns out that the "fix" is partial and doesn't cover all of the issues
3. Code reuse - somebody re-uses X and "knows" it is OK because of all the prior use
4. There is too much of this you can end up with a very slow system
5. Porting to another system/language/platform
6. You finally get around to unit testing
7. The Auditor or Trainee is looking over your shoulder and wants to know why the magic/complexity.

Answer 10

No: because it would subtract from, not add to, the clarity of the description of the issue.

If the phrase or word is not obvious and you need to ask others that's feedback that you should just 'call a spade a spade' and not an 'earth and ground removal tool'

Answer 11

I think this particular situation runs under the title "bug parity". Of course, the implied suggestion that one just needs another bug to fix an existing one is somewhat tongue-in-cheek.

Answer 12

Luck?

Reading about the situation raises all kinds of alarms / questions regarding the testsuite being used and/or the way the devs "fix" bugs. Many companies, it seems, have those incompetent coders who are apparently so used to wading through muddy water that they don't understand, that they also don't care to really tackle a bug when one is found: instead of going to the bottom with full analysis and a robust fix, they take the much shorter route of coming up with a "hack" or "kludge" that isn't understood but (seems to) work(s); aka, the testsuite stopped complaining. (Note: only when then resulting code is 1. actually correct, 2. nobody understands it, you could speak of "black magic" or "voodoo coding" (waving a dead chicken), but that is clearly not what we're talking about here).

The picture here is that "random" things are tried until the testsuite doesn't give an error anymore. This sounds stupid, and it is, but it happens sadly a lot. (Note: the result is they make the deadline every time, but create a source tree like a house of cards. Unmaintainable, because nothing makes sense or does what it says. Unflexible, because any change reveals a dozen or so bugs that before the change were cancelling eachother out mostly (in the case of two bugs, two easy to find bugs (or rather, one bug and one "fix") are turned into one hard-to-find and obscure bug: so hard that Q&A and the testsuite don't run into it, it can't be reproduced and only happens in rare conditions in the production environment. I know, because then the good coders (or coder if said company only has one) are asked to look into it :/.

If such a top coder exists, and he is able to find the problem, and good version control exists, then in the end the problem will be described as: "there was (obscure, whatever) bug in xyz, which was partly fixed in commit ef576c2 but did not really address the real issue".

In the meantime a LOT time was wasted, but it isn't the place of this good coder to describe it as: "A Johnson fuckup as usual. When is that guy finally going to be fired?"

Evolutionary this seems to be something that keeps itself alive however: making deadlines and providing a product with cheap coders (that aren't offered high salaries elsewhere anyways) for a while, and then ending up with a product that constantly needs "maintenance", aka keeps bringing in money after the client has already become dependent on it, or invested too much into, seems to work better then the same application that costs four times as much and takes twice as long to deliver (but never fails).

Hence, reading your description: "Luck" (jokingly) when it was the same coder, or else I'd call it "someone's stupidity".

Answer 13

In the world of Thermodynamics, this would be considered an "entropic state". It might seem stable and balanced now, but eventually, it'll all come crashing down! By the sound of it, they're not actually bugs and perhaps are intended for different situations. In calculus, what you have are two expressions that evaluate to the same result with three values: x=y-z, y=x+z

Answer 14

I would refer to them as "zero-sum bug set", or ZeSBugS, which is then pronounced "zes bugs".

These zes bugs went unnoticed for 2 years!

Disclaimer: I've actually never used this expression! :)

Answer 15

It might well be called a kludge, if it happens to be one. (But "compensating" seems like the best word in general.) And here is a real world example--although I admit I'm making it up--but you decide if it could happen.

Someone reads a spec for a GPIO and fails to notice that a signal is asserted active low. They write dummy code to assert it with a function they expect ORs a bit into a register, but don't complete it at the time. Later, someone writes the code for the bit-wise OR and checks the operation. They notice the polarity of the signal is wrong. To prove that is the problem, they insert code to AND the bit-wise inverse. They test the code, it works perfectly, because it so happens no one else is calling the OR function at the time. They go to lunch. When they come back, they are distracted by another project. The project gets handed to someone else who starts by writing a test of what works.

Answer 16

In the world of particle physics and geophysics they might be called "annihilators". For instance, when trying to model an observed magnetic field, any pair of magnetically susceptible bodies whose signal cancels each other out are called annihilators. The principle of model parsimony (occam's razor: keep things as simple as possible) means these annihilators should be removed.

I think the analogy is quite apt for your situation.

Answer 17

In a testing campaign, if the software returns the expected results, then there is no bug. So this particular test case would be "OK" at the functional level.

You would find the bugs by doing unit testing, which means testing one function at a time. That would help you identify 2 unit bugs and fix them.

Answer 18

'Isn't this just code that works?' No, because:

1. Code is fresh vegetables: it goes stale and needs to be refreshed. Code is only valuable if it is maintainable. Mystery bugs makes it unmaintainable.

2. What if an exception happens between the first and second bug? The first bug is active after the exception!

Answer 19

I've always referred to this type of thing as a "false positive".

In your situation where underlying math is wrong, it could be a problem if these values are stored & used again later, you could see the bug somewhere else, but maybe not, depending on the tests being run.

How do you catch it depends on how/where it happens? Could you output the values as it goes through each step of processing? Run through the calculations individually & verify as you go? Review the values stored in the database? I'm sure there are many other ideas on how to catch this.