“We should just be more careful at the specification stage”
Coding, Fast and Slow: Developers and the Psychology of Overconfidence
“… If you do “fully understand” something, you’ve got a library or existing piece of software that does that thing, and you’re not writing anything. Otherwise, there is uncertainty, and it will often blow up. And those blow ups can take anywhere from one day to one year to beyond the heat death of the universe to resolve.
When you first hit this pain, you think “We should just be more careful at the specification stage”. But this turns out to fail, badly. Why? The core reason is that, as you can see from the examples above, if you were to write a specification in such detail that it would capture those issues, you’d be writing the software. And there is really just no way around this. (if, as you read this, you’re trying to bargain this one away, I have to tell you — there is really really really no way around this. Full specifications are a terrible economic idea. Some ways below I’m going to lay out better economic choices)
The real trouble here is the interplay between the two sources of estimation error: the human bias towards overconfidence, and the inherent uncertainty involved in any real software project. That uncertainty is severe enough that even the careful, rational SystemII is unable to come up with accurate predictions.