Posted about this just recently. Reference to the original is at the bottom.
Key Idea
Suppose you want to know how long a task will take. You take a look at the problem and estimate that it will be one hour's worth of difficulty/complexity. At the end of the hour, it's not done. You've realized something about the problem and it's as if you are starting over. What is a reasonable/minimal estimate of the time it will now take? Thinking for a moment, you realize it should be the amount of complexity you already knew about plus the complexity you just discovered. At this point, the old complexity was zero, so you add 1 and 0 to get your new estimate of 1. Continuing in this fashion you obtain the following Fibonacci Sequence:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, ...
Each step satisfies the following golden ratio since the sequence yields the most efficient next estimate
a+b / a = a / b
Or more explicitly
new_estimate_of_complexity = new_complexity + old_known_complexity
new_estimate_of_complexity / new_complexity = new_complexity / old_known_complexity
Further Issues
But then you ask, does it mean that it is always as if I'm starting over? People will get upset if I have to tell them that I thought it was going to take 8 hours but now it will take 13 more. Thinking for a moment one realizes that if we only update the time, so that estimating 13 when we had 8 only adds 5 more hours, then it is as if the sequence is reversed.
..., 55, 34, 21, 13, 8, 5, 3, 2, 1, 1, 0
The same logic still applies. It's as if we started with one of these higher numbers and we're giving smaller estimates as time went on.
Furthermore, we can scale each number in the sequence by the same amount, say 8.
0, 8, 8 (2 days), 16 (four days), 24, 40 (one work week), 64, 104 (~ 2.5 weeks), 168 (~ one month), 272, 440 (a sprint), ...
This scaling number accounts for how long the problem solver needs to understand and reason about the problem. On average, per individual and across individuals, 8 hours is usually the best starting point, since it strikes a balance between all of the following issues:
- Risk of giving estimates that are too short.
- Risk of giving estimates that are too long.
- Accounting for interruptions.
- Error in estimating the complexity of a problem.
- The problem statement changing midstream.
- Etc.
What it is not
It is not about uncertainty in the task. Quite the opposite, it's about what we know. Known complexity.
It's not about human perception, but it does have the side effect of influencing our perceptions. Yes, 13 really does seem bigger than 8 without being too much bigger.
It's not about guessing, which people commonly assume when they guess at the scaling factor. Clearly, identifying this number could be made more rigorous.
Scaling is not the same as using a different multiple such as 2. This would effectively subsume the Fibonacci Sequence, which we know is accounting for what is known about the complexity of the problem. Other sequences are making assumptions about the unknown as well as the known, which is why they are inefficient.
Splitting into smaller stories in order to reset the clock, is not necessary nor beneficial. Can anyone make this problem smaller than what it really is? Does anyone have a time machine so I can go back and make up my hours? No on both counts. But when you reset the clock, you are hiding how risky of a problem it happens to be. A problem, that is legitimately estimated to be 8 hours, is the same as one with 8 hours after resetting from 440 hours.
