AI Alignment Explained: Brian Christian on Machine Learning and Human Values

Artificial intelligence now shapes decisions about loans, hiring, parole, medical care, and what billions of people see online. Yet the systems we build often optimise for the wrong things, amplify historical injustices, or behave unpredictably in unfamiliar situations. This persistent gap between what we wantactuallyalignment problem.

Writer and researcher Brian Christian, known for his work at the intersection of computer science, philosophy, and human behaviour, has helped bring this problem to a wider audience. His book “The Alignment Problem: Machine Learning and Human Values” and related talks explore how technical issues in machine learning connect to ethical questions about fairness, accountability, and global risk.

This article synthesises key ideas associated with Christian’s perspective: how modern AI learns, where misalignment arises in practice, and what researchers are doing to make advanced systems safer and more reliable.

From Hand-Coded Rules to Learning Machines

For much of the history of AI, systems were built using explicit rules: programmers wrote down logical conditions for every scenario they could anticipate. This approach struggled in messy real-world domains—there are too many edge cases, and rules quickly become brittle.

Modern AI instead relies heavily on machine learning, where systems learn behaviour from data rather than explicit instruction. Two families of techniques are especially important in Christian’s work:

• Supervised learning: models are trained on labelled examples (e.g. images with the correct category) to predict outcomes as accurately as possible.
• Reinforcement learning (RL): an agent interacts with an environment, receives rewards, and learns which actions tend to maximise cumulative reward.

Both paradigms raise deep alignment questions: who chose the training data, what objective did they specify, and does that objective really capture the behaviour we want?

Why Alignment Is Harder Than It Looks

On paper, alignment might sound simple: define a goal, train a model, deploy it. In practice, every stage of the pipeline is a potential failure point. Christian and many AI researchers emphasise three recurring sources of misalignment that recur across domains:

• Data problems: biased, incomplete, or unrepresentative data.
• Objective problems: the formal metric optimised in training is a poor stand-in for what humans care about.
• Deployment problems: systems are used in contexts they were not designed for, or incentives change once the system is in the wild.

These issues show up in everything from credit scoring to content moderation and predictive policing. When models learn patterns from historical data, they may faithfully reproduce those patterns even when the underlying practices were unjust or harmful.

Reinforcement Learning: Rewards, Credit, and Unintended Behaviour

Reinforcement learning is central to Christian’s discussion because it mirrors many aspects of real-world decision-making. In RL, an agent chooses actions, observes the resulting states, and receives scalar rewards that encode success or failure. Over time, the system learns a policy that tries to maximise the expected sum of rewards.

This paradigm has powered impressive successes—superhuman performance in Atari games and Go, robotics control, and complex resource allocation tasks. But it also highlights subtle alignment challenges:

• Delayed rewards: beneficial or harmful outcomes often occur long after the decisions that caused them. Assigning “credit” to the right actions is difficult.
• Reward hacking: if the reward signal is imperfect, agents can find shortcuts that maximise the signal while undermining the underlying goal.
• Exploration vs. safety: to discover good strategies, an RL agent must try new actions. Some of those actions may be unsafe, especially in high-stakes environments like healthcare or autonomous driving.

These problems are not mere theoretical curiosities: they show how fragile our definitions of success can be when translated into code.

Temporal-Difference Learning: Learning From Changing Expectations

One influential idea in RL, emphasised in Christian’s explanations, is temporal-difference (TD) learning. Instead of waiting for the final outcome, an agent constantly updates its predictions based on how its expectations change over time.

Informally, if the agent expects a high future reward in a given state but later revises that expectation downward, it learns that something about its previous assessment was too optimistic. This allows learning to propagate backwards from outcomes to earlier decisions much more efficiently.

Interestingly, neuroscientific studies suggest that dopamine signals in the brain behave in ways that closely resemble temporal-difference errors, hinting that animals and humans may rely on similar mechanisms to evaluate actions and outcomes over time.

Alignment and Fairness: Two Sides of the Same Coin

One of Christian’s central insights is that near-term concerns about fairness and accountability are deeply connected to long-term questions about superintelligent AI safety. Both are about ensuring that learning systems optimise for what we genuinely value, not just what is easy to measure.

In areas like criminal justice or lending, this often shows up as a tension between predictive accuracy and different formal definitions of fairness. Research by ProPublica and academic groups on recidivism prediction has shown that it is mathematically impossible to satisfy certain fairness criteria simultaneously when base rates differ between groups. This forces policymakers and engineers to make normative choices: which trade-offs are acceptable, and for whom?

Christian argues that treating these as separate agendas—one for ethicists and policymakers, another for technical safety researchers—misses the fact that they share core mathematical and conceptual challenges.

Imitation, Preference Learning, and Human Feedback

One promising response to misalignment is to move beyond simple reward functions and fixed labels, and instead have models learn directly from human behaviour and feedback. Researchers use several complementary approaches:

• Imitation learning: training systems to copy expert behaviour from demonstrations (e.g., human driving).
• Inverse reinforcement learning: inferring an underlying reward function that would make a human’s observed behaviour “make sense,” then optimising that inferred reward.
• Reinforcement learning from human feedback (RLHF): humans compare or rate model outputs, and the model is trained to produce answers that people prefer.

These methods have been used to align large language models and other complex systems more closely with user intentions, safety constraints, and social norms. However, Christian and others stress that this raises challenging questions about whose preferences are learned and how to deal with the fact that human values are inconsistent, context-dependent, and evolve over time.

The Philosophy Behind the Alignment Problem

Beneath the technical work lies a web of philosophical debates. Some of the key themes Christian engages with include:

• Incomplete and conflicting values: people hold values that are often in tension (e.g., equality vs. efficiency). There may be no single coherent function that perfectly captures what we want.
• Possibilism vs. actualism: should we evaluate actions based on the best possible future they could bring about, or on what is realistically likely to happen given human limitations?
• Moral uncertainty: given disagreement among ethical theories, how should we design systems that might have enormous impacts on the future?

These questions are not just academic. They shape how we design decision-support systems for global health, climate policy, or biosecurity—areas where Christian notes that effective altruism researchers have highlighted the enormous leverage of getting decisions roughly right at scale.

Alignment at Scale: Existential Risk and Global Priorities

As AI capabilities advance, Christian and many researchers worry about systemic and existential risks—scenarios in which misaligned systems could cause large-scale or irreversible harm. These concerns have been discussed in work associated with the University of Oxford’s Future of Humanity Institute and other research centres.

Key risks include:

• Loss of control: advanced systems pursuing badly specified goals could resist human intervention or develop strategies that undermine oversight.
• Runaway optimisation: powerful optimisation for proxy objectives can reshape economic, informational, and physical environments in unintended ways.
• Strategic misuse: states or non-state actors could deploy advanced AI for cyberwarfare, autonomous weapons, or large-scale surveillance.

Christian highlights that many of the conceptual tools needed to think about long-term AI risk—uncertainty over values, trade-offs between present and future, institutional incentives—are the same tools needed to reason about today’s fairness problems.

Practical Steps Toward Better Alignment

While perfect alignment may be unattainable, Christian’s perspective underscores that there is substantial room for improvement in how we design and govern AI systems. Emerging best practices include:

• Rigorous evaluation and auditing before deployment, including testing under distribution shift and stress-testing for failure modes.
• Cross-disciplinary collaboration between computer scientists, ethicists, domain experts, and affected communities.
• Transparent documentation of datasets, model limitations, and intended use cases.
• Regulation and standards that require accountability, impact assessments, and recourse mechanisms.
• Investment in safety research, including interpretability, robust RL, scalable human oversight, and value learning.

Several governments and international bodies are beginning to develop AI policy frameworks that incorporate these ideas, though technical progress continues to outpace governance in many areas.

Frequently Asked Questions (FAQs)

Q1: What does “AI alignment” actually mean?

AI alignment refers to the effort to design and train artificial systems so that their behaviour reliably advances human goals, respects human values, and avoids harmful side effects, even in complex and novel situations.

Q2: Why is Brian Christian’s work on alignment important?

Christian’s work is important because it connects abstract technical issues—like reward design and reinforcement learning—to concrete human consequences in criminal justice, healthcare, and global risk, making alignment an accessible yet rigorous topic for a wide audience.

Q3: Is alignment only about hypothetical superintelligent AI?

No. Many alignment failures already exist in deployed systems that are far from superintelligent, such as biased risk scores or content recommendation algorithms that promote harmful content. Long-term concerns amplify the stakes but build on the same underlying problems of objectives, data, and incentives.

Q4: Can we fully capture human values in a single objective function?

Most philosophers and many AI researchers doubt that a single, fixed objective can fully represent human values, which are context-dependent, plural, and often internally conflicted. Current research instead focuses on approximations, learning from human feedback, and building systems that remain corrigible as our understanding of values evolves.

Q5: What can individuals or organisations do today to improve alignment?

Organisations can implement bias audits, invest in interpretability and monitoring tools, establish clear accountability structures, and involve diverse stakeholders in system design. Individuals can support rigorous AI safety and governance research, advocate for responsible policy, and critically evaluate AI tools they adopt or deploy.

Written by

Sneha Tete

• Integrated M.A. in Applied Linguistics, University of Hyderabad
• 5+ years in editorial, instructional, SEO, and research-based writing

Sneha is a relationships and lifestyle writer with a strong foundation in applied linguistics and certified training in relationship coaching. She brings over five years of writing experience to theskillvault,  crafting thoughtful, research-driven content that empowers readers to build healthier relationships, boost emotional well-being, and embrace holistic living. Her writing and insights reflect the belief that beauty, happiness, and wellness…