Philip Tetlock and the Art of Better Forecasting

Trying to reason about the future under uncertainty is at the heart of every high-stakes decision, from public policy and national security to personal careers and philanthropy. Yet decades of research suggest that unaided human intuition is often much less accurate than we would like to believe.

This article draws on the work of psychologist Philip Tetlock, whose forecasting research has transformed how experts, governments, and ordinary people think about prediction, accuracy, and good judgment. It is an original synthesis inspired by an in-depth interview with Tetlock, organized to show both what his findings mean and how you can apply them.

Why Forecasting Accuracy Matters More Than Confidence

Public debate often rewards the wrong qualities in forecasters. Commentators who speak in bold, absolute terms attract attention, but their track record is rarely checked carefully. Tetlock’s early research tracked more than 28,000 political predictions from hundreds of experts and found that, on average, their accuracy was barely better than chance.

What did matter was not ideology, seniority, or fame, but judgment style—how people think about uncertainty and evidence.

• Confident pundits tend to rely on a single big idea, explain everything through that lens, and rarely revisit their mistakes.
• Careful forecasters gather diverse clues, update beliefs when news arrives, and quantify their uncertainty in probabilities.

In practical terms, this means:

• Organizations that track forecast accuracy can identify who actually adds value.
• Citizens and leaders should weigh predictive track records more heavily than rhetorical skill.

The Fox and the Hedgehog: Two Mindsets About the Future

Tetlock popularized a modern version of an old metaphor: the hedgehog and the fox.

Across multiple forecasting competitions, fox-like thinkers systematically outperformed hedgehogs, particularly on medium-term political and economic questions. Being a fox does not require encyclopedic knowledge; it requires intellectual humility and an active search for disconfirming evidence.

What Forecasting Tournaments Revealed

Tetlock’s most famous work, including the Good Judgment Project, used large, structured forecasting tournaments to evaluate real-time predictions about global events. Participants were asked to assign numerical probabilities to thousands of questions like election outcomes, policy changes, and conflict risks.

These tournaments reached several important findings:

• Accuracy can be measured. By scoring probabilistic forecasts with proper scoring rules, researchers could distinguish better and worse forecasters over time.
• Some people are consistently better. A small subset of participants—later called superforecasters—dramatically outperformed both chance and most professionals, year after year.
• Skill is partially learnable. Training, feedback, and team-based forecasting improved performance, even among already strong participants.

The results were striking enough that the U.S. intelligence community funded and studied these tournaments as part of a broader effort to improve geopolitical forecasting.

Inside the Mindset of a Superforecaster

Superforecasters are not mystical seers. On standard intelligence measures, they are smart but not superhuman, and they come from a wide variety of educational backgrounds. Their advantage lies in their habits of thinking, especially when dealing with uncertainty.

Common traits include:

• Active open-mindedness. They seek out conflicting evidence and enjoy having their minds changed by strong arguments.
• Probabilistic thinking. Instead of saying something will or won’t happen, they assign and refine percentage chances—30%, 60%, 85%, and so on.
• Frequent updating. As new information arrives, they adjust probabilities in small, sensible steps rather than swinging from zero to one.
• Decomposition. They break complex questions into simpler components that can be estimated separately.
• Relentless feedback use. Forecasts are recorded, scored, and reviewed, which creates a loop of learning and calibration.

In tournament settings, elite forecasters were able to meaningfully discriminate between events that were 56% versus 57% likely—subtle distinctions far beyond ordinary “possible/impossible/certain” folk categories.

From Folk Probability to Precise Probabilities

Most people use a rough mental scheme for uncertainty, built around a few vague bins such as “never going to happen,” “might happen,” and “almost sure.” Tetlock and other psychologists showed that this folk probability system leads to systematic errors.

For example:

• People treat very small probabilities as if they were zero, especially for slow-moving, hard-to-picture risks.
• They treat very high probabilities as if they were certainties, underweighting the remaining risk that things go wrong.

By contrast, forecasters in tournaments are rewarded for being well-calibrated—that is, when they say 70% across many events, about 70% of those events must occur. Over time, practice and scoring help them shift from fuzzy labels to precise, meaningful percentages.

How Forecasting Improvements Help Real Decisions

Forecasting tournaments might sound abstract, but their lessons carry over to everyday and institutional choices:

• Policy planning. Governments can use quantified forecasts to weigh the expected value of interventions, such as pandemic preparedness or climate adaptation, instead of relying only on narratives and worst-case scenarios.
• Security and intelligence. Intelligence agencies can track which analytic methods and teams yield the most accurate geopolitical forecasts, improving resource allocation.
• Career and philanthropy. Individuals and foundations can explicitly forecast outcomes for different paths—such as projects, grants, or job moves—and then compare them after the fact.

In all these domains, the combination of explicit probabilities, clear time frames, and post-hoc evaluation makes it easier to learn from experience rather than repeat the same mistakes.

Working with Low-Probability, High-Impact Risks

Some of the hardest forecasting problems concern very rare but extremely consequential events: global pandemics, catastrophic wars, or technological accidents. Because these tail risks rarely occur, it is difficult to validate any numerical probability estimate with direct experience.

Tetlock and colleagues suggest several strategies:

• Use structured categories. Group very rare events into broader classes (e.g., major pandemics, regional armed conflicts) where there is at least some historical data.
• Check logical consistency. Compare your probability for a subset versus a broader set. If you think dying in a car crash is more likely than dying from a car crash plus all other causes combined, your numbers need revision.
• Complement forecasts with principles. For extremely deep uncertainty, societies may rely partly on precautionary principles and robustness—ensuring that plans can tolerate surprises rather than trying to predict every detail.

These tools do not remove uncertainty, but they help decision-makers reason more coherently about it.

Why Famous Experts Are Not Always the Best Forecasters

A recurring theme in Tetlock’s work is the gap between public visibility and predictive accuracy. Interviews and subsequent commentary on his research have noted that highly visible experts often perform no better—and sometimes worse—than less prominent peers when their predictions are scored objectively.

Several forces drive this disconnect:

• Media incentives favor strong, simplified claims over nuanced uncertainty.
• Experts may feel pressure to defend their previous public positions rather than update in light of new evidence.
• Audiences often seek reassurance and clear stories, not probability distributions.

Tetlock’s tournaments highlight that, if we care about accuracy, we need systems that measure performance and reward calibration, not just confident storytelling.

How You Can Practice Better Forecasting

You do not need to join a formal tournament to benefit from these insights. Many of Tetlock’s findings translate into simple practices for personal and professional decision-making:

• Write down your predictions. Before key decisions, record what you think will happen and by when, using explicit percentages and dates.
• Break problems into parts. For example, a prediction about a technology’s adoption might be decomposed into technical feasibility, regulatory approval, cost trends, and social acceptance.
• Search for counterarguments. Deliberately look for evidence and perspectives that challenge your current view.
• Update incrementally. When relevant news appears, adjust your probabilities in proportion to how surprising the information is.
• Review outcomes periodically. Compare what you expected with what occurred, and look for patterns in your overconfidence or pessimism.

Over time, these habits can sharpen your intuitive sense of likelihoods and help you identify situations where your judgment is systematically biased.

Common Pitfalls in Thinking About the Future

Tetlock’s research intersects with broader findings in behavioral economics and cognitive psychology about how people systematically err when reasoning under uncertainty. Some frequent pitfalls include:

• Overconfidence. Assigning probabilities that are too extreme (very close to 0% or 100%) when evidence is limited.
• Motivated reasoning. Letting identity or ideology determine what we expect to happen, rather than weighing evidence neutrally.
• Hindsight bias. After an event occurs, convincing ourselves we “knew it all along,” which prevents learning from mistakes.
• Neglect of base rates. Ignoring how often similar events have occurred in the past when forming predictions.

Superforecasters work to counter these tendencies by systematically considering base rates, exploring multiple scenarios, and explicitly quantifying their uncertainty.

Forecasting as a Tool for Better Institutions

The implications of Tetlock’s work go beyond individual skill. His later collaborations, including work through initiatives like the Forecasting Research Institute, focus on building institutional systems that incorporate forecasting into decision processes.

Promising applications include:

• Evaluation of analytic methods. Comparing how different models, expert groups, or processes perform on a shared set of forecasting questions.
• Structured disagreements. Designing exercises where people with different worldviews make explicit, testable predictions about contested issues, then comparing long-run track records.
• Governance and accountability. Using prediction records to inform hiring, promotion, and budgeting in organizations that rely heavily on judgment under uncertainty.

By tying reputations and resources more closely to forecasting performance, institutions can gradually realign incentives toward accuracy and learning.

Frequently Asked Questions (FAQs)

Q: What is a forecasting tournament?

A forecasting tournament is a structured competition in which participants assign numerical probabilities to clearly defined future events, such as election results or policy decisions, over specified time horizons. Their predictions are scored using rules that reward both calibration and sharpness, allowing researchers to compare accuracy across individuals and methods over time.

Q: Are superforecasters just very intelligent people?

Superforecasters tend to be above average in cognitive ability, but research suggests their edge comes more from mindset and method than raw IQ. They are unusually open to evidence, skilled at breaking complex questions into parts, and persistent about revising beliefs in light of new information.

Q: Can ordinary people learn to forecast better?

Yes. Training in probabilistic reasoning, regular practice with feedback, and participation in forecasting platforms have all been shown to improve accuracy. Many of the core techniques—such as using base rates, decomposing problems, and recording predictions—are accessible to non-experts and can yield measurable gains over time.

Q: How reliable are forecasts for very rare events like global catastrophes?

Forecasts for extremely rare, high-impact events are inherently uncertain and difficult to validate statistically. Nonetheless, structured approaches—such as grouping related risks, checking for logical consistency in probabilities, and combining quantitative models with expert judgment—can make reasoning about these threats more coherent, even if precise numerical accuracy remains hard to establish.

Q: Why should organizations invest in forecasting rather than rely on expert intuition?

Tetlock’s work shows that unaided expert intuition often performs worse than many expect when predictions are rigorously evaluated. By investing in forecasting systems that track accuracy, organizations can identify who adds predictive value, improve strategic planning, and reduce the influence of overconfident but inaccurate voices.

Written by

Medha Deb

Medha Deb is an editor with a master's degree in Applied Linguistics from the University of Hyderabad. She believes that her qualification has helped her develop a deep understanding of language and its application in various contexts. Medha specializes in the areas of beauty, health, sports, and wellness and is committed to ensuring that the content on the website is…