...I think that a better analogy is a large construction project like an
office building or a large bridge. They often run over budget and
schedule, and often develop unforeseen problems. Some of them are
almost exactly like previous ones, and some have many new elements.
During construction, unexpected problems arise, and once completed,
usage patterns may turn out to be quite different from those
anticipated. Many of the implementation details are routine and
implemented by tradesmen who don't know much about the overall
architecture.
I've operated with that idea for many years, but I'm coming to believe
it's actively destructive because it leads to pure waterfall thinking:
design, architect, engineer, build. Where is there a place for an
iterative design, build, test cycle in building construction?
Well, that's a good point, and one I've yet to think through. You know,
I think of some kinds of industrial plants - chemical or steel plants,
say. When a new process is developed, one often builds a pilot plant.
The pilot plant is on an intermediate scale, larger than lab but smaller
than the eventual full scale. The pilot plant is where the key tricks
of the trade are learned, and process problems are solved. It's where
you learn what scales and what doesn't. You don't generally go from a
lab-scale nuclear reactor directly to a 1000 megawatt generating plant
without building a 100 MW intermediate plant first.