Why Should We Iterate?
Published: 2025-10-24Why do we iterate? What is the benefit of breaking a piece of work into smaller pieces? How is it better? Is the reason that you think about actually the whole reason?
If you are in the agile community you probably give an answer along the benefits of fast feedback, "always working software" and prioritization. These are all valid. Small, incremental work does mean that customers don't have to wait until all the features are ready (and some of them will be more valuable to the customer than others), and that as engineers we can also use this cadence to measure and adjust as we build out the full solution.
But as I discovered recently, there is also more to it; it is also about our ability to make decisions, make them quickly, and generally be happier.
Making Decisions
As humans, we make decisions. Often complex ones. As professionals, we make many complex and impactful decisions. But we are also very bad at making decisions, and even worse at admitting it.
Why do we make these decisions? Usually it is to get to an end goal. As a professional, that goal could be to do with profit, market share or efficiency. As humans, the goal could be to do with health, wealth or happiness. But I think that most of the goals can actually be boiled down to happiness.
If we can accept this, then it leads to the next question: why do we deliberate so much over decisions? The typical answer is because decisions require information, and the best way to make a good decision is to gather as much information as we can and use it to inform us.
This however leads us to an important fact about that information: most of the information we gather is in the form of predictions, and we are probably worse at making predictions as we are at making decisions.
Predicting
When we are choosing between two solutions to a problem, we may "gather information" like:
- how complex the problem is,
- how many changes it is likely to take,
- how long it is likely to take, and
- how many people it is likely to need
If you look closely however, most of these are predictions: How do you really know how complex the problem is? Maybe you can gather information which will give you a decent starting point, but at some point you will need to extrapolate because most software work is novel. The same goes for information about effort and resources; they are really mostly predictions. How many times have you finished a complex piece of work which has matched the information you gathered at the start?
It has been shown many times that humans are bad at predicting time and effort, even when we have good information; according to the "Planning Fallacy", we are demonstrably bad at estimating how long a task will take even if we had completed a very similar task not long before! Based on this, how can we expect that gathering more information will make this any better?
Getting Stressed
Another reason we gather this information is to reduce our own stress. Humans don't like uncertainty; they usually like to be in control. According to Naval Ravikant:
"Uncertainty, not outcome, is the root of stress."
and according to popular philosophical thought:
"Stress is the gap between our expectations and reality"
When we gather information, we believe we are reducing uncertainty and therefore reducing that gap, ultimately reducing our stress. But is this really just a placebo? If this information that we gather is usually unreliable, are we really just delaying our stress until a later date?
Another reason why we get stressed over large decisions is the fear of making mistakes. If we keep our decisions large and filled with uncertainty, the risk of making a mistake is larger and that gap between our expectations and reality is not only psychological but also temporal; the expectations we have now will not be resolved with reality for a long time. I'm sure at one time you have had to sit nervously waiting for a test result or a piece of feedback, perhaps biting your nails and tapping your foot. Not only did you need to wait for this resolution, but it was also out of your control. All of these add up to more stress: uncertainty, time and control.
Thinking Small
These, I think, are core reason why we need to iterate in small steps. Some people call it experimenting - making small changes to your system and checking what happens.
The core effects of this are:
- we really are reducing uncertainty by gathering information. The difference is that the information gathered this way is no longer a prediction. It is real. You have thrown out a small experiment and gathered actual data: new knowledge about your project, the customer or something else.
- the time between starting the experiment and getting the results is reduced. We no longer have to wait months or even years to receive information we can actually act on. It can come in weeks or days.
- small, iterative experiments allows us to take back control. Each iteration allows us to make an adjustment, perhaps to correct a small assumption that we now know to be wrong. Is the customer reaction good or bad? What was the actual cost of this iteration? Is the next priority actually the next priority?
How does this affect our decision-making? Two-fold: they will be more informed (based on the more experiments you make) and they will be smaller. Smaller decisions are easier to make, and you are more likely to accept a 'just do it' mindset because there is less risk.
Harvard professor Dr Ellen Langer teaches us it's better to make decisions quickly and deal with the consequences. You usually can't really know if one decision will be better than its alternative. But you can make a small decision quickly, measure it, learn from it and make a better decision next time, all the while feeling happier about it.
Experiment First
Hopefully this gives you a new perspective about decisions and information. Information is still important, but we need to realise that we should be gathering the right information; facts and feedback, and not predictions. The information-gathering therefore should not be dammed up behind the wall of one huge decision; it should form a cycle of experiments which continually refine the decisions you need to make while still flowing in the direction of your goal.
In this way, you are also narrowing your gap between expectation and reality by making your expectations smaller, and measuring reality faster. By doing so you reduce both your uncertainty and your stress. And who doesn't want to do that?