Bernhardsson on why projects run late
📚SOURCE
💎PERMANENT NOTES
🧵SYNTHESIS
The essay offers a statistical explanation for something everyone in software knows and nobody budgets for. Bernhardsson looks at the ratio of actual time to estimated time for a task and observes that it is not spread evenly around one. A task can finish in half the time. It can also take ten times as long. It cannot take less than no time. The distribution of that ratio has a hard floor and a long right tail, and he models it as log-normal.
The consequence is the insight. In such a distribution the median and the mean are far apart. People are, he argues, fairly good at estimating the median: about half of tasks come in under the estimate and half over. But a project is a sum of tasks, and sums are governed by the mean. The mean is pulled to the right by the few tasks that blow up. So a project built from individually fair estimates will still overrun, and by more as the uncertainty per task grows.
He adds a second point: the total is dominated by the tasks with the widest uncertainty. One poorly understood item can outweigh a dozen well-understood ones, so the plan's risk lives where the plan is vaguest.
✨KEY IDEAS
Overruns are a ratio. A task is late by a factor and never by a fixed amount.
An estimate can be right as a median and still far too low as an expectation.
Add up skewed tasks and the total drifts towards the tail, because the blow-ups are never cancelled by the early finishes.
The few most uncertain tasks decide the project. The well-understood ones barely move it.
With moderate uncertainty per task the mean lands well above the median, and it climbs steeply as uncertainty widens.
📝READING NOTES
Paraphrase, not quotation. A blog essay from 2019 with a simulation attached. Read on Friday night and Saturday morning, 21 and 22 February, nine days after I recommended slipping the launch.
Why it mattered that week
On 12 February I told a room that the Exceptions Inbox would not ship on the 18th. The new date was 11 March. I spent the following week asking how I had been so wrong, and the question assumed I had made an error, some single thing I should have caught.
This essay reframes it. My estimate was a decent median. If we had run that project a hundred times, perhaps half would have made the 18th. I had reported the median as if it were a commitment.
The model in three lines
- Actual over estimated is skewed: floor at zero, long tail to the right.
- People estimate the middle of that distribution.
- Projects are sums, sums follow the mean, and the mean sits to the right of the middle.
What I checked against my own work
- The load test failing at a third of volume was one item, and it accounted for most of the slip. That is his second point.
- I went back through the Halden launches I could find dates for. Every one was late. None was early. A floor and a tail.
- It fits what Hale said in the lecture on base rates, which I had sat through on the evening of the failed test without connecting the two.
Where I push back
- Log-normal is an assumption. He fits it to a modest set of data and it fits well. That does not make it the law. The qualitative claim, skew with a long tail, is what I rely on.
- He treats tasks as independent. Ours were not. When the load test failed, three other items slipped because of it. Correlated tasks make the tail fatter than his model says, so the essay if anything understates.
- It explains and does not prescribe. Knowing the mean exceeds the median does not tell me what number to give Sales. My working answer: give a range, and say which end is the median.
Where it went
- 📝 Issue 34 - The estimate I gave and the one I should have, a week late, on this.
- 💎 Late projects are late by a multiple and not by a margin.
- 💎 My launch estimates ignore every launch I have seen, which needed 📖 Kahneman on the planning fallacy a month later to be finished. Bernhardsson says why the inside view comes out low. Kahneman says what to use in its place.
- Concept note: 🧩 Planning fallacy.
The arithmetic
My own check, in a spreadsheet on the Saturday. Ten tasks, each estimated at one week, each with the same skew.
- If overruns were symmetric: total about ten weeks, give or take.
- With a long right tail: the typical total was nearer fourteen to sixteen, and one run in ten went past twenty.
- With one very uncertain task among the ten: that single task was most of the difference.
Rough, and the exact figures depend on a spread I guessed. The shape did not depend on it.
🗣️IN SHORT
- In : Why software projects take longer than you think: a statistical model, Erik Bernhardsson, 2019
- Bernhardsson argues that : projects overrun because task durations are skewed, so honest per-task estimates add up to a total that is too low
- by stating that : the ratio of actual to estimated time has a long right tail, people estimate its median, and a sum of such tasks is governed by the larger mean and dominated by the most uncertain items