Have you ever been surprised when it took you much longer to complete a task than you expected or you overestimated your knowledge or ability in a situation? This happens to me all the time with any home maintenance task. I end up spending more time watching DIY YouTube videos trying to solve the problem than I actually do working on the task because I don’t have the knowledge to complete the task. Half the time, I come to the realization that I need to outsource the task if I want it done right.
In a business sense, I often surprise myself when I scope a client project and underestimate how long it will take my team to complete a task we’ve done many times before. How did this happen? What can I learn? Do I go back to the client and tell them the project is out of scope or do I just chalk it up to a leaning opportunity? I try to backtrack and figure out why there is such a discrepancy between the estimated and actual time. One of the reasons I underestimate the home improvement and client project tasks is called the overconfidence bias. I’m overly confident in my knowledge and ability to complete the home improvement and client tasks. This type of overconfidence is called the planning fallacy. The planning fallacy happens when we underestimate the length of time it will take to complete a task, often ignoring past experience. In business, we do this all the time and that is why we end up behind schedule and over budget.
We are often overconfident in our estimation because of an availability bias (or the recency effect). That is, we tend to place too much emphasis on the information and evidence most readily available when we’re making a decision. We depend on the most recent events because they are fresh in our minds, leading us to rely too strongly on them when estimating or thinking about what might happen in the future. Think about the gambler who is on a hot streak. He wants that streak to continue, so he ignores past losses and keeps playing. We focus on recent causes and tend to ignore the full range of experience, including previous failures. The gambler does not think about his grand total of losses over time. We too can become caught up in the hot streak of success and forget failures of the past.
Research shows that humans, by nature, tend to be overconfident in their ability to make judgments about their own performance and their performance as compared to others. The overconfidence effect applies to our forecasting ability. We systematically overestimate our knowledge and our ability to predict—on a massive scale. The overconfidence effect is the difference between what people really know and what they think they know. I tend to be overconfident in my abilities to complete a client task (e.g., draft a survey, create a measurement strategy, analyze data, generate a report, etc.) because it is something I have done many times and under various circumstances. Yes, even when I have tracked hours and use that past record as a baseline. Surprisingly, experts suffer even more from the overconfidence bias than laypeople do. In addition, the bias is stronger in men than women. I overcome the overconfidence effect by having my team review my scoping and time estimates. When the project is finished, I find my team is usually right that it took longer than I originally estimated!
There are a couple of points in this cognitive process that lead to overconfidence:
- People tend to focus on that first guess. They mostly ignore the other alternative. This is called “option fixation.”
- People are too easily satisfied with their explanations. When we keep them to ourselves, our explanations tend to be far shallower than we think they are.
How do you overcome overconfidence bias?
- Be aware of your bias to overestimate your knowledge and ability.
- If you employ only internal Subject Matter Experts (SMEs), you may be suffering from overconfidence bias. You need to consider blending internal and external expertise.
- Build in a margin (cushion) to your original prediction or estimation to compensate for the overconfidence bias.
- Use the ORAPAPA questioning framework – Opportunities, Risks, Alternatives & Improvements, Past Experience, Analysis, People, and Alignments & Ethics. The exercise of formally outlining and documenting your decision process and potential outcomes will offer you a broader perspective.
- When possible, get feedback on your decision options so you don’t repeat past mistakes. Also build in a mechanism for rapid feedback on decisions you make in the future.
- Employ unbiased, third-party experts. Outsiders bring a fresh perspective. They don’t have the same reference points or emotional attachments to investments and projects. They also lack your preexisting hypotheses and the political egocentrism.
- Encourage effective group dynamics with rigorous dialogue and debate.
Smarter People Planning protects you from falling into cognitive traps by offering an objective perspective and alerting you to potential pitfalls.
What business decision are you going to make differently now that you are aware of the overconfidence effect?