Existential Risk Forecasting: Tools, Ethics, and Action

Existential risks are threats that could cause human extinction or permanently destroy our long-term potential, making all future generations impossible. Although rare and hard to quantify, these risks may be among the most important problems humanity faces, because the stakes involve not just billions of lives today, but all possible lives in the future.

This article explains how researchers try to forecast such risks, what tools and incentives they use, and how ordinary people and institutions can contribute to better long-term decision-making.

What Counts as an Existential Risk?

Researchers usually distinguish existential risks from more ordinary global crises.

• Global catastrophic risks: events that kill a large fraction of humanity or cause massive global damage, but from which recovery may be possible (for example, a severe pandemic or large regional nuclear war).
• Existential risks: events that cause human extinction, or that irreversibly lock in a permanently worse future—for example, a stable global totalitarian regime or unrecoverable civilizational collapse.

Philosopher Nick Bostrom defines an existential risk as one whose downside is “the extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential.” This framing emphasizes that what is at stake is not only current suffering and death, but the entire long-run trajectory of civilization.

Major Categories of Existential Risk

Different organizations classify existential risks in different ways, but several broad categories recur across the literature.

Why Forecasting Existential Risks Is So Difficult

Forecasting existential risks involves challenges that go beyond ordinary prediction problems.

• Extremely high stakes: Even small changes in the probability of existential catastrophe may correspond to enormous expected losses, because they affect all future generations.
• Limited data: Humanity has never experienced human extinction, and many relevant technologies (like advanced AI or synthetic biology) are novel.
• Long time horizons: Some risks unfold over decades or centuries, stretching far beyond most existing datasets and institutional planning cycles.
• Strategic uncertainty: Key actors may respond strategically to policies, treaties, and technological developments, making simple trend extrapolations unreliable.
• Ethical and value disagreements: People disagree about what counts as an unacceptable future (for example, permanent global surveillance vs. extinction), complicating how risks are evaluated.

Core Tools for Forecasting Extreme Risks

Despite these difficulties, several forecasting tools can be adapted to the x-risk context, usually in combination rather than isolation.

1. Quantitative Modeling

Where data exist, researchers build mathematical models to estimate event frequencies and possible impacts.

• Physical models: Asteroid and comet impact risk can be estimated from astronomical surveys, impact craters, and orbital dynamics, giving a rough annual probability of extinction-level impacts.
• Epidemiological models: For pandemics, compartmental models (like SEIR) can simulate how a pathogen might spread and how interventions change outcomes.
• Climate and earth-system models: These simulate how greenhouse gas emissions, feedback loops, and tipping points might lead to severe warming scenarios or cascading failures.

For many anthropogenic risks, quantitative models are combined with large subjective uncertainties, and the outputs are often used to explore scenarios rather than to give precise probabilities.

2. Structured Expert Judgment

Because historical data on existential catastrophes are scarce, expert judgment plays a central role in forecasting.

• Expert elicitation protocols encourage experts to provide probabilistic assessments in a consistent way, sometimes with calibration training.
• Delphi methods anonymize and iterate expert opinions to reduce conformity pressures and encourage convergence on better-justified estimates.
• Cross-disciplinary panels combine perspectives from technical domains (e.g., AI, biology) with social sciences, ethics, and security studies.

3. Forecasting Platforms and Tournaments

Forecasting tournaments ask many participants to make probabilistic predictions about well-defined questions, then track accuracy over time.

• Prediction markets use small financial stakes to reward accurate forecasts; prices aggregate dispersed information into a consensus probability.
• Non-market tournaments measure accuracy using scoring rules, identify aggregate forecasts (e.g., weighted averages), and analyze which reasoning styles perform best.
• Decomposition of high-level questions (such as “chance of nuclear war by 2050”) into more granular subquestions (e.g., arms control treaties, regional tensions, technological shifts) allows for more tractable predictions.

Although existential catastrophes themselves are not directly observable in historical data, forecasting tournaments can still target proximate indicators and risk factors—such as the adoption of key safety measures, arms-control treaties, or deployment of dangerous technologies.

4. Scenario Analysis and Narrative Forecasting

Scenario analysis explores different plausible futures without assigning precise probabilities to each pathway.

• Exploratory scenarios describe contrasting futures (optimistic, pessimistic, business-as-usual) to identify where key uncertainties lie.
• Stress tests examine how institutions or technologies might behave under extreme but conceptually plausible conditions.
• Red teaming adopts an adversarial mindset, asking how systems might be subverted or fail catastrophically.

These qualitative methods complement quantitative forecasts by revealing blind spots and assumptions that numbers alone might obscure.

Building Good Forecasting Questions for X-Risk

Because existential catastrophes are rare and hard to score directly, much of the craft lies in asking the right intermediate questions.

• Operationalize vague concerns: Instead of “Will AI be dangerous?”, ask about specific milestones, governance steps, or incidents (for example, “probability a major AI lab publishes a formal safety standard by 2030”).
• Prefer short- to medium-term horizons: Questions that resolve within 5–15 years allow performance feedback, even if they are only proxies for much longer-term risks.
• Link indicators to long-run outcomes: Forecasts about arms-control regimes, BSL-4 labs, international AI agreements, or carbon pathways can be connected—via models and expert reasoning—to long-run x-risk trajectories.
• Make resolution criteria precise: Clear yes/no or numeric resolution rules reduce ambiguity and dispute when the question resolves.

Calibrating and Evaluating Forecasters

Not all forecasters are equally accurate, and some reasoning styles consistently outperform others. Two central concepts are calibration and discrimination.

• Calibration: When someone says there is a 30% chance of an event across many similar cases, roughly 30% of those events should occur. Systematic overconfidence or underconfidence shows up as miscalibration.
• Discrimination: The ability to reliably assign higher probabilities to events that happen than to events that do not.

Forecasting tournaments measure these properties over hundreds of questions. Successful forecasters often:

• Break complex problems into simpler, conditional components.
• Update their views frequently as new evidence appears.
• Track their own past performance and learn where they tend to err.
• Blend inside views (detailed domain knowledge) with outside views (base rates, historical frequencies).

The Ethics of Forecasting Humanity’s Future

Existential risk forecasting is not value-neutral. It immediately raises ethical questions:

• Whose future counts? Most academic treatments assume that the lives of future people matter morally, sometimes enormously so, because there could be vast numbers of them.
• Distribution of risk: Some populations are more exposed to climate change, pandemics, or conflict than others. Forecasting can highlight these inequities and inform fairer policy responses.
• Information hazards: Publishing detailed scenarios about, for example, engineered pandemics or cyber vulnerabilities might inadvertently aid malicious actors. Forecasters must weigh openness against security.
• Governance and legitimacy: Large-scale decisions based on x-risk forecasts—such as regulation of AI or biotechnology—need transparent reasoning to be democratically accountable.

From Forecasts to Action: Using Predictions to Reduce Risk

Forecasts only matter if they inform decisions. Several pathways connect existential risk forecasting to concrete risk reduction.

• Policy design
  • Building early-warning systems for emerging infectious diseases and unusual outbreaks.
  • Setting research and safety standards in advanced AI, synthetic biology, and nuclear technology.
  • Designing robust international agreements to control weapons of mass destruction and regulate dangerous technologies.
• Institutional resilience
  • Investing in health systems, infrastructure, and emergency response that can withstand shocks.
  • Strengthening democratic institutions and rule of law to reduce the risk of catastrophic governance failures.
• Research prioritization
  • Channeling funding toward domains where small advances could greatly reduce x-risk (for example, biosurveillance, AI alignment research, or nuclear command-and-control safety).
  • Supporting methodological work on better forecasting techniques and risk metrics.

Practical Habits for Individual Forecasters

You do not need to be a professional researcher to contribute to better long-term forecasts. Many of the habits that help in ordinary judgment and decision-making also apply to existential risk.

• Quantify your uncertainty: Replace statements like “very unlikely” with explicit probability ranges and timeframes.
• Seek base rates: Ask how often analogous events have happened in the past, even if the analogy is imperfect.
• Consider disconfirming evidence: Look for reasons you might be wrong, not just reasons you might be right.
• Update in small steps: Treat new information as a nudge rather than a complete reversal, unless the evidence is overwhelming.
• Document your reasoning: Writing down your assumptions and logic makes it easier to identify mistakes later and to learn over time.

Frequently Asked Questions (FAQs)

Q: How do experts estimate probabilities for events that have never happened?

Experts combine several approaches: analogies with similar but less extreme events, physical and statistical models (where possible), and structured judgment methods. For example, the risk of asteroid impacts is inferred from crater records and astronomical surveys, while the risk of nuclear war is informed by historical crises, deterrence theory, and current geopolitical trends.

Q: Are natural existential risks more or less worrying than human-made ones?

Current research generally indicates that, over the coming centuries, anthropogenic risks—especially those related to advanced technology and governance—are more concerning than purely natural risks like asteroid impacts or supervolcanoes. Natural risks appear to be either relatively well-constrained in probability or mitigable with existing or foreseeable technologies, while human-made risks may grow as our capabilities outpace our institutions.

Q: Can ordinary people meaningfully help reduce existential risk?

Yes. Individuals can contribute by improving their own forecasting skills, participating in public forecasting platforms, supporting evidence-based policy, entering relevant careers (such as public health, AI safety, diplomacy, or climate science), and advocating for institutions that take long-term risks seriously.

Q: How do disagreements about values affect x-risk forecasting?

Different value systems can agree that human extinction would be extremely bad, but they may diverge on whether futures involving, for example, pervasive surveillance or radical technological transformation count as catastrophic. Forecasters often separate empirical predictions (what will likely happen) from value judgments (how good or bad those outcomes are) so that decision-makers with diverse ethics can still use the empirical work.

Q: Is it reasonable to focus so much attention on rare, uncertain risks?

Given the potentially astronomical moral importance of humanity’s long-term future, many scholars argue that even modest reductions in existential risk can have very large expected benefits. The key is to prioritize interventions that are both plausibly impactful and cost-effective, while remaining sensitive to present-day needs and justice concerns.

Written by

Sneha Tete

• Integrated M.A. in Applied Linguistics, University of Hyderabad
• 5+ years in editorial, instructional, SEO, and research-based writing

Sneha is a relationships and lifestyle writer with a strong foundation in applied linguistics and certified training in relationship coaching. She brings over five years of writing experience to theskillvault,  crafting thoughtful, research-driven content that empowers readers to build healthier relationships, boost emotional well-being, and embrace holistic living. Her writing and insights reflect the belief that beauty, happiness, and wellness…