56

Is there any name for a situation where two bugs cancel each other out and in the result, the software works as expected? For example, suppose software A calculates a value and mistakenly value X is subtracted from the input somewhere and exactly the same value (X) is added to value again mistakenly. So we have two independent bugs, but magically, they cancel out their side-effect and the result of calculation looks correct in the end.

I'm asking this question because I've found this type of bugs one of the most confusing and the hardest to find. It has happened to me a few times when I was working on something else and accidentally spotted an error/bug/mistake. At first, you scratch your head asking yourself how it has been working so far then? It took a while until you realized another bug has been cancelling out this one. I was wondering if there is a category for this type of bugs and they have been studied methodically. I like to learn more about them and how to avoid/find them.

  • 7
    Isn't that just not a bug? It might be a sub-optimal implementation, but if the result is correct the software is working as required. – jonrsharpe Jun 1 at 18:59
  • 4
    I guess you could call this defect masking, although it usually says that one bug is masked by the other because the first bug didn't allow you to execute the code that contains the other bug. But I can't come up with/remember anything better right now. @jonrsharpe: perhaps it's just one path through the algorithm. another time one of the two (addition or subtraction) could be left out, leaving you with the wrong value at the end. – pavelsaman Jun 1 at 19:09
  • 2
    What are you hoping to achieve by asking? Has there been a problem communicating this situation to someone as "several bugs that cancelled each other out"? – jonrsharpe Jun 1 at 19:11
  • 12
    There is a gray area where the second bug could be a workaround. Sometimes the specification is so unclear that e.g. you may think the data you receive is according to spec though it isn't, so you compensate for that sending-end bug on the receiving end. Neither of the two components will work with a compliant partner though... – Peter - Reinstate Monica Jun 2 at 16:38
  • 1
    Even if the second bug is not a workaround, @Peter-ReinstateMonica is right that breaking the two modules apart and matching them with another is likely to reveal that bug. Unit testing, as someone else mentioned, will help prevent this issue. – Robert Columbia Jun 3 at 12:23

18 Answers 18

54

I call them masked bugs. Bug 1 (the erroneous subtraction) is masked by bug 2 (the erroneous addition).

Ideally, they would be caught by unit testing - since unit tests should be reporting whether or not each module returns the correct value, and both units are behaving incorrectly.

Since it's not an ideal world, the next best thing is to step through the process as much as the tools available to you permit. For instance, when I'm dealing with a new feature, I will usually start with entering basic data, then check that the data I enter is correctly saved to the database. Then I start exploring boundaries and validation scenarios. I don't try to run the full end-to-end scenario until I've stepped through and covered as much as I can in the smallest steps the application.

Masked bugs are not rare. They're not common, but they happen often enough that most testers have run into quite a few of them. Masked bugs that are cancelled out (as opposed to just hidden) by the masking bug are a lot rarer, but they still happen. I recall (not fondly) a set of bugs that I ran into once, where each more specific instance was masked by the general instance that I discovered first. I think there ended up being 4 separate bugs that got found and fixed, each one more specific than the previous masking bug.

| improve this answer | |
  • 11
    Masked bugs that cancel each other can be somewhat common in embedded and real time systems, for example a race condition that is hidden by incorrectly specified delays on two real time interrupt handlers, but may surface in unusual conditions like a failed external device or when someone changes the timing. Do you have any insight on bugs that are masked potentially based on environment conditions (be it environment of your server like network load or physical environment of an embedded device interacting with the firmware) – crasic Jun 2 at 17:45
  • 5
    To me, a "masked bug" is when one bug keeps another from being hit. For example, you've got some code that lets any logged-in user perform administrative actions, but it's masked by another bug that keeps users from logging in. – Mark Jun 2 at 21:25
  • 3
    Masked bug are frequent - bugs exactly canceling each other are rarer. Life being what it is, it's not unusual for a minor bug to mask a critical, high priority one... Beware of cosmetic fixes. – ptyx Jun 2 at 22:33
  • 2
    Whatever it's called, it definite is a manifestation of a lack of unit testing. – corsiKa Jun 3 at 6:15
  • 2
    I don't think "masked" is the right term here, either. It might depend on the circumstances though. Normally I'd agree that masked means hide by avoiding it: one bug hides another, but it can not be masked at the same time by the bug that it masks. – Carlo Wood Jun 3 at 17:06
28

An auditor or accountant would describe this as an incidence of "Compensating Errors" - it's probably something which is much more common in both of those fields. I think the term describes the situation and implies the challenge in trouble-shooting and resolution quite nicely.

| improve this answer | |
23

I've always said such code "works by coincidence".

If you say the code is broken or has a bug, people tend to push back because it appears to work to a casual observer. "Works by coincidence" acknowledges that the code does provide the intended result, but for the wrong reasons. People generally understand that a "coincidence" is a not something that can safely be relied on and they tend to be more likely to devote the resources to fix the problem (in my experiences, at least).

A lot of times, these sorts of bugs are the result of testing things at too high of a level. If you break up your tests into more narrow, focused tests that isolate specific units of code, then your two offsetting bugs are more likely to end up in separate units/modules and will, therefore, be more visible.

That won't catch all of them, though. If the bugs are close enough together (as in your case, where they're both parts of the implementation of a single calculation) then testing alone may never be able to catch them individually. That's where things like code review come into play. A fresh pair of eyes tend to be fairly good at spotting things that are out of place. For your particular example, a static analyzer might be able to flag the -X and +X as self-defeating and able to be optimized away.

| improve this answer | |
  • 3
    +1 for 'works by coincidence' – Neil_UK Jun 4 at 4:41
  • What can be more annoying are imperfectly cancelling bugs. I worked on one unit that had a sign error in one place that was fixed in another place. It worked about 98% of the time, but there were certain circumstances where it would generate large errors, and they were of course difficult to replicate as they were partially dependent on sensor inputs which would read slightly differently each experiment. (It was in the field before we even realized there was a problem.) – Cristobol Polychronopolis Jun 4 at 19:52
  • I am calling it “wrong”, but I will try to change that to “working by coincidence“. I like the difference in reception that you mention. – Oliver Meyer Jun 14 at 6:18
5

I simply call them bugs that cancel each other, and I doubt if there is any special way to find or avoid them, and whether any special research was done on it (I tried Google Scholar and haven't found).

The only place where it might have some importance is for building statistical models of the product, they usually assume a single point of failure. But again I doubt if it is important since this kind of bugs is usually rare.

| improve this answer | |
4

I've called this "an even number of bugs". I believe that the term can be attributed to my Calculus professor, who once made "an even number of mistakes" on the board.

enter image description here

Examples of "an even number of mistakes" could include:

  • Suggesting that the driver turn left when you meant right, but she turns right by mistake because she gets her directions confused.
  • Not noticing the message that company is bringing their spouse, so you do not prepare enough food, then somebody cancels.

I think that this happens a bit too often in my life!

Image: https://www.dreamstime.com

| improve this answer | |
4

In quantum chemistry, a related phenomenon is called

"cancellation of errors"

| improve this answer | |
3

If you are dealing with the case @jonsharpe described above, you can create examples with the components in isolation.

If they are not decoupled, you probably have a testability problem that is hiding these bugs.

| improve this answer | |
  • Actually, my problem is not explaining the problem, but I'm curious if there is any study/work about this type of bugs. You mentioned decoupling/testability. Is there any metric that helps us to avoid this type of bugs? I guess the scope of this type of bug is as big as the most granular type of test you have. Any resource/study that addresses this kind of question. – CoyBit Jun 1 at 21:29
  • 1
    TDD addresses this issue with the design. Tools like JDepend gather metrics like Afferent Couplings, Efferent Couplings, and Distance from the Main Sequence. More info here: dzone.com/articles/jdepend-design-metrics-ci – João Farias Jun 2 at 19:30
3

Two errors which perfectly cancel each other out are possible only in the most theoretical sense. On any real computation machine, such a perfect cancellation is not possible.

Consider the very simple example of a function f(x) which takes an integer as input and doubles it. The domain of this function is [-INT_MAX/2, INT_MAX/2].

Now, consider another function g(x) which does the same thing, but buried somewhere in the code of g(x) are two bugs: the first bug adds 1000 to the result for no reason, and then the second bug subtracts away 1000 from the result for no reason.

You might say that these two bugs cancel each other out, and therefore function f(x) and function g(x) are identical. But they are not identical functions. There is a subtle difference. The two compensating errors change the domain of function g(x). With the addition of 1000, it can no longer handle the upper bound INT_MAX/2 of the domain without numerical overflow error. The domain of this function is actually [-INT_MAX/2, INT_MAX/2-500].

So the two functions are not exactly the same.

I don't think you could come up with an example of two functions, one with a pair of compensating errors and the other without the pair of compensating errors, which are exactly the same in all respects.

The difference introduced by a pair of compensating errors might be subtle, but it is detectable.


For those saying, "That's nice. So what?"
Consider: A careful, exhaustive analysis of function g(x) would reveal that its domain is [-INT_MAX/2, INT_MAX/2-500] which is not what we would expect for an integer doubling function. Thus, it is still "buggy" despite the pair of (supposedly) mutually-canceling errors.

| improve this answer | |
  • 1
    "I don't think you could come up with an example": I want to number each day of the week, starting on Sunday = 0. But I accidentally define Sunday to be 6! Oops! I also accidentally define Saturday to be 0! Double oops! But because I only use it to work out whether it is a weekday or not (day != Saturday and day != Sunday) it never gets noticed. Fix ONE of the errors and the code will break. – Oddthinking Jun 3 at 4:23
  • @Oddthinking Bad example: No bug at all. Just a different and unusual (but completely valid in the internal scope of the functon) internal representation of day-numbers. It doesn't invalidate the functionality at all. Now, if you change one, you introduce a new bug. – Tonny Jun 3 at 14:08
  • 1
    @Tonny: We seem to be quibbling about the definition of "bug" rather than addressing anything of substance. If another developer noticed the error in one of the names and they filed a bug report, would it raise eye-brows? – Oddthinking Jun 3 at 14:29
  • 2
    I still say "That's nice. So what?", because coming up with a highly pedantic theoretical frame challenge doesn't help the OP. For practical purposes such scenarios (setting aside whether or not they are not possible according to the strictest of definitions) can occur and can be dealt with more effectively than the exhaustive analysis you propose (e.g. by unit testing the individual functions). In any case, your assertion that it is not possible is wrong. Consider for example big number data fields which do not have upper bounds. – JBentley Jun 4 at 9:33
  • 1
    @UnLogicGuys By this kind of pedantic logic you could also argue equally well that there is no such thing as a bug. Only software that works exactly as written and hardware that behaves exactly as put together. That is beside the point. – EntangledLoops Jun 5 at 3:13
3

Most of the times that I have met this sort of thing it has not been bug in function Y cancels but in function X - it has been bug in function X compensated for by incorrect patch in function Y, i.e. somebody notice that the output of Y was wrong so fixed the symptom rather than realising that the problem was in X.

This gets to be a problem when one of:

  1. Somebody spots and fixes one but not the other(s)
  2. It turns out that the "fix" is partial and doesn't cover all of the issues
  3. Code reuse - somebody re-uses X and "knows" it is OK because of all the prior use
  4. There is too much of this you can end up with a very slow system
  5. Porting to another system/language/platform
  6. You finally get around to unit testing
  7. The Auditor or Trainee is looking over your shoulder and wants to know why the magic/complexity.
| improve this answer | |
2

No: because it would subtract from, not add to, the clarity of the description of the issue.

If the phrase or word is not obvious and you need to ask others that's feedback that you should just 'call a spade a spade' and not an 'earth and ground removal tool'

| improve this answer | |
1

I think this particular situation runs under the title "bug parity". Of course, the implied suggestion that one just needs another bug to fix an existing one is somewhat tongue-in-cheek.

| improve this answer | |
1

Luck?

Reading about the situation raises all kinds of alarms / questions regarding the testsuite being used and/or the way the devs "fix" bugs. Many companies, it seems, have those incompetent coders who are apparently so used to wading through muddy water that they don't understand, that they also don't care to really tackle a bug when one is found: instead of going to the bottom with full analysis and a robust fix, they take the much shorter route of coming up with a "hack" or "kludge" that isn't understood but (seems to) work(s); aka, the testsuite stopped complaining. (Note: only when then resulting code is 1. actually correct, 2. nobody understands it, you could speak of "black magic" or "voodoo coding" (waving a dead chicken), but that is clearly not what we're talking about here).

The picture here is that "random" things are tried until the testsuite doesn't give an error anymore. This sounds stupid, and it is, but it happens sadly a lot. (Note: the result is they make the deadline every time, but create a source tree like a house of cards. Unmaintainable, because nothing makes sense or does what it says. Unflexible, because any change reveals a dozen or so bugs that before the change were cancelling eachother out mostly (in the case of two bugs, two easy to find bugs (or rather, one bug and one "fix") are turned into one hard-to-find and obscure bug: so hard that Q&A and the testsuite don't run into it, it can't be reproduced and only happens in rare conditions in the production environment. I know, because then the good coders (or coder if said company only has one) are asked to look into it :/.

If such a top coder exists, and he is able to find the problem, and good version control exists, then in the end the problem will be described as: "there was (obscure, whatever) bug in xyz, which was partly fixed in commit ef576c2 but did not really address the real issue".

In the meantime a LOT time was wasted, but it isn't the place of this good coder to describe it as: "A Johnson fuckup as usual. When is that guy finally going to be fired?"

Evolutionary this seems to be something that keeps itself alive however: making deadlines and providing a product with cheap coders (that aren't offered high salaries elsewhere anyways) for a while, and then ending up with a product that constantly needs "maintenance", aka keeps bringing in money after the client has already become dependent on it, or invested too much into, seems to work better then the same application that costs four times as much and takes twice as long to deliver (but never fails).

Hence, reading your description: "Luck" (jokingly) when it was the same coder, or else I'd call it "someone's stupidity".

| improve this answer | |
  • I do not like your harsh words. But the core of your statement ist right: it remains a bug although the code does as it should. Those who claim there is no bug ignore maintainability of the code. The reasons people do this can be manifold: the concept of a code owner might keep me from changing YOUR code. Too little testing or specification to tell correct behaviour from incorrect behaviour. Being the hero programmer who thrives on the code complexity. Wrong goals: Make it work! Simple stupidity rarely is the reason. – Oliver Meyer Jun 14 at 6:31
1

In the world of Thermodynamics, this would be considered an "entropic state". It might seem stable and balanced now, but eventually, it'll all come crashing down! By the sound of it, they're not actually bugs and perhaps are intended for different situations. In calculus, what you have are two expressions that evaluate to the same result with three values: x=y-z, y=x+z

| improve this answer | |
1

I would refer to them as "zero-sum bug set", or ZeSBugS, which is then pronounced "zes bugs".

These zes bugs went unnoticed for 2 years!

Disclaimer: I've actually never used this expression! :)

| improve this answer | |
1

It might well be called a kludge, if it happens to be one. (But "compensating" seems like the best word in general.) And here is a real world example--although I admit I'm making it up--but you decide if it could happen.

Someone reads a spec for a GPIO and fails to notice that a signal is asserted active low. They write dummy code to assert it with a function they expect ORs a bit into a register, but don't complete it at the time. Later, someone writes the code for the bit-wise OR and checks the operation. They notice the polarity of the signal is wrong. To prove that is the problem, they insert code to AND the bit-wise inverse. They test the code, it works perfectly, because it so happens no one else is calling the OR function at the time. They go to lunch. When they come back, they are distracted by another project. The project gets handed to someone else who starts by writing a test of what works.

| improve this answer | |
1

In the world of particle physics and geophysics they might be called "annihilators". For instance, when trying to model an observed magnetic field, any pair of magnetically susceptible bodies whose signal cancels each other out are called annihilators. The principle of model parsimony (occam's razor: keep things as simple as possible) means these annihilators should be removed.

I think the analogy is quite apt for your situation.

| improve this answer | |
0

In a testing campaign, if the software returns the expected results, then there is no bug. So this particular test case would be "OK" at the functional level.

You would find the bugs by doing unit testing, which means testing one function at a time. That would help you identify 2 unit bugs and fix them.

| improve this answer | |
0

'Isn't this just code that works?' No, because:

  1. Code is fresh vegetables: it goes stale and needs to be refreshed. Code is only valuable if it is maintainable. Mystery bugs makes it unmaintainable.

  2. What if an exception happens between the first and second bug? The first bug is active after the exception!

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.