What Are S-Risks? Why Extreme Future Suffering Matters

When people think about global catastrophic risks, they often focus on scenarios in which humanity goes extinct. Yet there is another class of dangers that is less widely discussed but potentially even more morally urgent: suffering risks, or S-risks. These are futures where enormous numbers of beings continue to exist, but experience vast amounts of suffering that could have been avoided.

This article introduces the idea of S-risks, explains how they might arise in the coming centuries, and outlines concrete ways individuals and institutions can help reduce them. The discussion is inspired by work in effective altruism and longtermist ethics, but presented here in fully original wording and structure.

1. What Are S-Risks?

S-risks are scenarios in which a very large number of sentient beings experience intense or prolonged suffering. The scale is so large that, in moral terms, it rivals or exceeds the importance of avoiding human extinction. While extinction eliminates the possibility of future happiness, S-risks involve futures that contain enormous suffering that continues over long stretches of time.

Importantly:

• S-risks do not require humanity to go extinct; they often assume advanced future civilizations or powerful technologies still exist.
• They are about the quality of future experiences (extreme suffering), not just quantity of life or years survived.
• The potential scale is astronomical because future technologies could support huge numbers of digital minds or biological beings.

Ethically, many moral theories agree that preventing very intense suffering for large numbers of individuals carries enormous weight. Longtermist views, which care about the far future as much as the present, treat S-risk reduction as a major priority because the future could contain immense numbers of sentient beings.

2. How S-Risks Differ from Existential and Other Global Risks

Global risk discussions often focus on existential risks, defined as events that would cause human extinction or permanently curtail humanity’s potential. S-risks can overlap with existential risks but are conceptually distinct.

An event can be both an existential risk and an S-risk (for example, a future dominated by hostile AI that endlessly exploits suffering beings), but conceptually they emphasize different moral failures: the loss of positive potential, versus the realization of enormous suffering.

3. Why Advanced AI Makes S-Risks Especially Salient

Several emerging technologies could contribute to S-risks, but advanced artificial intelligence is particularly important because it could become vastly more capable than humans and operate at massive scale.

Researchers and public bodies have outlined several failure modes for highly capable AI that are directly relevant to S-risks:

• Loss of control: If future “general” or “superintelligent” AI systems cannot be reliably controlled, they may pursue goals misaligned with human values and ignore or even exploit suffering.
• Misuse by malicious actors: Even if AI can in principle be controlled, powerful systems might be weaponized by states, corporations, or individuals to surveil, punish, or torture others at unprecedented scale.
• Creation of digital minds with moral status: AI systems might eventually host entities with conscious experiences; if they are mistreated, this could generate large-scale suffering in virtual environments.

Some researchers argue that, given the difficulty of fully aligning superintelligent AI and the risks of malevolent modification, advancing such systems without strong safety guarantees could be deeply unethical. At the same time, AI may be central to solving other global problems, which makes careful governance essential rather than simple refusal to develop the technology.

3.1 Indirect AI-Related Pathways to S-Risks

AI could also contribute to S-risks in more indirect ways:

• Authoritarian lock-in: Highly capable AI-driven surveillance and control tools might entrench extremely oppressive political regimes that persist for centuries.
• Arms races and instability: AI-accelerated military competition could trigger wars or long-term social collapse, leaving many beings in harsh conditions.
• Value misgeneralization: AI systems optimizing proxies for human welfare (e.g., economic growth) might sacrifice well-being or inflict suffering on groups that are not well represented in training data.

4. Other Technological and Social Sources of S-Risks

While AI is a central concern, S-risks can arise from broader developments in technology and society.

4.1 Large-Scale Digital Minds and Simulations

Future civilizations may create massive numbers of digital minds or high-fidelity simulations for research, entertainment, or economic purposes. If these entities are conscious and poorly treated, the resulting suffering could be enormous. For instance:

• Simulations used to study conflict or extreme environments could repeatedly subject digital individuals to painful scenarios.
• Economic incentives might encourage architects of virtual labor systems to ignore the internal experiences of workers if performance metrics appear satisfactory.
• Misaligned AI could create “tool” minds designed to experience fear or pain as part of their functionality.

Because our understanding of consciousness and moral status in artificial systems is limited, there is a risk that moral patients will be created long before adequate protection frameworks are in place.

4.2 Bio-Technologies and Engineered Suffering

Advances in biotechnology and synthetic biology can be used for substantial good, such as vaccine development and disease control. However, the same tools could be misused in ways that increase suffering:

• Creation of pathogens engineered not primarily for lethality but for chronic debilitation or psychological distress.
• Design of organisms used in intensive agriculture or industry that live short, painful lives.
• Biological mechanisms of control or punishment in future authoritarian systems.

These issues intersect with current debates over dual-use research in biology, where scientific work can be used for beneficial or harmful purposes depending on governance and norms.

4.3 Social and Political Dynamics

Technologies are embedded in social contexts. Certain political and cultural trends can make S-risks more or less likely:

• Dehumanization of out-groups can normalize cruelty, especially when combined with advanced tools of control.
• Runaway competition between states or corporations can push actors to cut ethical corners, underinvest in safety, or tolerate suffering as a by-product of growth.
• Norm erosion regarding torture or extreme punishment could allow new technologies to be used in ways that were previously unthinkable.

5. Why Take S-Risks Seriously Despite Uncertainty?

Some people are skeptical of S-risks because the most extreme scenarios involve speculative future technologies. However, several considerations support taking them seriously:

• Scale: If technological civilization continues, the number of future beings—especially digital minds—could be enormous. Even a small probability of severe suffering on that scale can have high moral importance.
• Irreversibility: Once certain architectures, institutions, or AI systems are entrenched, reversing harmful patterns could be extremely difficult or impossible.
• Early decisions matter: Governance frameworks, norms, and technical designs adopted in the near future may shape trajectories for centuries.

Many AI and policy experts emphasize that, while current systems are limited, it is prudent to build safety, alignment, and governance capabilities ahead of the most capable systems.

6. Practical Strategies for Reducing S-Risks

Although S-risks concern the far future, there are concrete interventions available today. These span technical research, governance, cultural change, and individual career choices.

6.1 Advancing AI Alignment and Safety Research

Because AI is a central pathway to large-scale S-risks, improving its safety and alignment is a priority.

• Technical alignment: Research methods that make advanced AI reliably follow human-endorsed objectives, including value learning, robustness, interpretability, and safe exploration.
• Scalable oversight: Develop systems that allow humans to oversee and evaluate complex AI behavior, including techniques like debate, recursive oversight, and anomaly detection.
• Verification and control: Explore formal methods, red-teaming, and monitoring to detect dangerous capabilities before deployment.

Interdisciplinary collaboration between computer scientists, ethicists, psychologists, and social scientists can improve our understanding of how AI systems interact with human institutions.

6.2 Strengthening AI Governance and Global Coordination

Technical solutions are insufficient without robust governance frameworks. Policymakers and institutions can reduce S-risks by:

• Establishing safety standards for training and deploying highly capable AI models, including mandatory risk assessments and incident reporting.
• Regulating dual-use capabilities, such as AI tools that could enable cyberattacks or biological threats, while preserving beneficial research.
• Enabling international cooperation to avoid unsafe AI races, share safety insights, and coordinate responses to emerging risks.
• Protecting human rights and civil liberties so powerful AI-enabled surveillance is not used to entrench large-scale repression.

6.3 Ethical Frameworks for Digital Minds and Simulations

If future systems can host conscious digital minds, early ethical guidance will be crucial. Potential measures include:

• Precautionary principles that limit creating potentially conscious systems in contexts where significant suffering is likely.
• Guidelines for experiments and simulations, including constraints on intensity and duration of negative experiences.
• Research on consciousness and moral patienthood in artificial systems, clarifying when moral protections should apply.

6.4 Preserving and Strengthening Moral Norms

Technologies are shaped by values. Advocating for and upholding norms that strongly condemn torture, cruelty, and large-scale neglect of suffering can make it politically and culturally harder to pursue futures that generate S-risks.

• Support institutions that defend human rights and humanitarian law.
• Promote empathy toward non-human animals and potential future digital minds.
• Encourage public engagement with long-term ethical questions, so citizens can hold leaders accountable.

6.5 Individual Career and Advocacy Choices

Individuals can contribute meaningfully to S-risk reduction through their careers and civic engagement. Examples include:

• Pursuing work in AI safety research, AI governance, or biosecurity in academia, industry, or government.
• Joining or supporting organizations that work on global catastrophic and AI risks.
• Contributing to public conversations about responsible AI development and the ethics of future technologies.

7. Frequently Asked Questions (FAQs)

Q1: Are S-risks more important than preventing human extinction?

Different moral views weigh these priorities differently. Longtermist perspectives often regard both as extremely important: extinction removes the possibility of a flourishing future, while S-risks represent the realization of futures filled with vast suffering. Because both involve astronomical stakes, many researchers argue for addressing them in parallel rather than ranking one as clearly more important.

Q2: If advanced AI is still limited, why worry about S-risks now?

Current AI systems have major limitations in reasoning, planning, and generalization, and there is no clear evidence that superintelligence is imminent. However, building institutions, norms, and technical expertise ahead of capability surges is much easier than retrofitting safety and governance after dangerous systems already exist. Early work can shape research directions and policy frameworks that persist into more advanced eras.

Q3: Isn’t focusing on far-future suffering speculative compared with present-day problems?

There are serious present-day harms from AI and other technologies, such as bias, privacy violations, and cyberattacks. Work to address these harms can be complementary to S-risk reduction because it builds safety culture, technical methods, and regulatory capacity. Longtermist approaches do not deny current suffering; they highlight that the total amount of suffering in the future could be vastly larger, so it is worth dedicating some portion of resources to influencing long-run trajectories.

Q4: How likely are S-risks compared with more familiar catastrophes?

There are no precise, consensus probabilities. Some surveys of AI experts indicate non-trivial chances of highly adverse outcomes from advanced AI, including existential or catastrophic failures. Even if S-risks are less probable than other risks, the potential scale of suffering is so large that they remain morally significant. Much current research treats S-risk mitigation as a form of prudent, high-stakes risk management under deep uncertainty.

Q5: What can someone without a technical background realistically do?

Non-technical contributions matter. Policy, law, communications, philosophy, and public administration skills are crucial for building effective institutions and norms around AI and other powerful technologies. Individuals can also support high-impact organizations financially, engage in informed public discourse, and help shape workplace or community norms about responsible technology use.

Written by

Sneha Tete

• Integrated M.A. in Applied Linguistics, University of Hyderabad
• 5+ years in editorial, instructional, SEO, and research-based writing

Sneha is a relationships and lifestyle writer with a strong foundation in applied linguistics and certified training in relationship coaching. She brings over five years of writing experience to theskillvault,  crafting thoughtful, research-driven content that empowers readers to build healthier relationships, boost emotional well-being, and embrace holistic living. Her writing and insights reflect the belief that beauty, happiness, and wellness…