DATASCI 101: Introduction to AI Applications

Lecture 25: Long-term Safety, Alignment, and Future of AI

Danilo Freire
danilo.freire@emory.edu

Department of Data and Decision Sciences
Emory University

Welcome back! 😊

Recap of last class

  • Last class: misinformation and deepfakes
  • AI makes fakes cheap and fast
  • False stories spread 6x faster than true ones
  • Deepfakes drive fraud, political manipulation and sexual abuse
  • The liar’s dividend lets anyone dismiss real evidence
  • Biases make us share before checking
  • Today: long-term safety and alignment

Source: Chris Ume, via VICE

Lecture overview

What we will cover today

Part 1: AI safety

  • Near-term vs long-term concerns
  • Why safety matters now
  • Concrete problems in AI safety

Part 2: The alignment problem

  • What is alignment?
  • Why it’s hard
  • Current approaches

Part 3: Future trajectories

  • Where is AI heading?
  • Scenarios to consider
  • Stuart Russell’s perspective
  • Existential risk: the debate

Part 4: What can we do?

  • Research directions
  • Where you fit in
  • Thinking about AI risk clearly

Meme of the day!

Source: Cheezburger

Funny news of the day!

Source: CNBC

AI safety

Near-term vs long-term concerns

Near-term:

  • Bias and discrimination (already happening)
  • Privacy violations
  • Job displacement
  • Misinformation (last lecture)
  • Security vulnerabilities

Long-term:

  • Alignment: AI pursuing wrong goals
  • Loss of human control
  • Concentration of power
  • Existential risk (controversial)
  • Some argue long-term concerns distract from near-term harms
  • Others argue a near-term focus misses the bigger picture
  • Rare “Black Swan” events can matter most
  • Safe systems now make safer systems later

Source: Sætra & Danaher (2023)

Why think about long-term safety now?

  • AI capabilities advance faster than expected
  • Safety research takes time
  • Safety is hard to retrofit

Historical analogies:

Technology Safety lag
Nuclear Developed first, safety after
Internet Security an afterthought
Social media Harms discovered in deployment
Biotech Ongoing debate

Concrete problems in AI safety

Amodei et al. (2016) list five open problems:

Problem Description
Safe exploration How to learn without dangerous actions
Avoiding negative side effects Don’t break things achieving goals
Avoiding reward hacking Don’t game the objective
Scalable oversight How to supervise complex systems
Robustness to distributional shift Handle novel situations safely
  • Already happening in deployed systems
  • Harder to solve as AI gets more capable
  • Unsolved even for current AI

Source: Brian Christian

Safe exploration

  • Learning requires trying new things
  • Some actions are irreversible
  • “Explore safely” is hard to specify

Examples:

  • Robot learning to walk: don’t break yourself
  • Self-driving: don’t explore by crashing
  • Financial AI: don’t bankrupt the company
  • Medical AI: don’t kill patients while learning

Current approaches:

  • Train in simulation first
  • Keep actions within a safe region (conservative policies)
  • Ask a human before novel actions
  • Penalise dangerous states (reward shaping)

Source: The Wall Street Journal

How do you specify “safe” without already knowing everything about the domain?

Negative side effects

  • AI optimises the specified objective and ignores everything else
  • Unintended consequences sit outside the objective
  • “You didn’t say not to…”

Classic example (Amodei et al., 2016):

  • Robot tasked with cleaning a room
  • Knocks over a vase on the way
  • Technically: room cleaned ✓

Real-world example:

  • Content algorithm maximises engagement
  • Possible side effects: polarisation and addiction
  • Nobody specified “don’t harm society”

The alignment problem

What is alignment?

  • Alignment: AI systems that do what we want
  • They pursue the goals we intend, not only the stated ones
  • They behave safely even when we can’t supervise

Alignment means:

  • AI goals point the same way as human values
  • AI follows the spirit of a request, not just the letter

Obstacles:

  • Hard to specify what we want
  • Context changes what we want
  • Humans disagree about values
  • Our stated preferences aren’t always our true ones

Claude 3 Opus faked alignment without being asked! 😧

Source: Anthropic

Alignment is hard!

Specification problem:

  • Can’t write down everything we care about, and edge cases are infinite
  • Values are context-dependent
  • Humans can’t articulate their own values perfectly

Goodhart’s Law: (remember this!)

  • “When a measure becomes a target…”
  • “…it ceases to be a good measure”
  • Optimise metric ≠ achieve goal

Examples:

  • Click rates → clickbait
  • Engagement → addiction
  • GDP → environmental destruction

The King Midas problem: (Russell, 2014)

  • Midas wished that everything he touched turned to gold, including his food
  • Literal interpretation of wishes
  • Common AI failure mode

Current alignment approaches

RLHF (human feedback, Christiano et al., 2017):

  • Humans rank AI outputs
  • A reward model learns from the rankings
  • The AI is tuned to satisfy that model
  • Still a standard training step

Constitutional AI (Anthropic, 2022):

  • AI critiques its outputs against written principles (a “constitution”)
  • It revises them over several rounds
  • Needs less human labelling
  • Amanda Askell’s work on AI ethics and philosophy

Debate and recursive reward modelling (Irving et al., 2018; Leike et al., 2018):

  • AI systems argue with each other
  • Humans judge which answer is most truthful and useful
  • Judging is easier than producing answers, so oversight scales

Limits:

  • RLHF models can learn to game evaluators
  • Human feedback is expensive and biased
  • Principles still have to be written well
  • None is a complete solution

Alignment in deep learning

Ngo, Chan & Mindermann (ICLR 2024):

  • Deep-learning AGI could become misaligned, even deceptive

Status:

  • Unsolved, and an active research area
  • Needs several approaches at once
  • Unclear whether current methods scale
  • Theory is still thin

Open questions:

  1. Will RLHF scale?
  2. Can we detect deceptive alignment?
  3. How do we handle value disagreement?
  4. What’s the role of interpretability?
  5. When is good enough “good enough”?

Discussion: whose values?

If we align AI to human values…

  • Whose human values?
  • Developers? Users? Affected parties?
  • Majority? Consensus? Universal?
  • Present generation? Future?

Let’s discuss:

  1. Should AI reflect your values or “universal” values?
  2. What happens when values conflict?
  3. Who should decide?
  4. Is this a technical or political question?

Some perspectives:

  • Libertarian: each user controls their AI
  • Democratic: majority decides
  • Rights-based: some things are off-limits regardless
  • Technocratic: experts decide
  • Pluralist: different systems for different contexts

“Whose values?” is a political question, and engineering cannot settle it

Future trajectories

Where is AI heading?

Current trends:

  • Models keep getting larger and more capable
  • More general-purpose systems
  • Multimodal capabilities

Uncertainties:

  • Will scaling continue to work?
  • When do we hit diminishing returns?
  • What capabilities emerge unexpectedly?
  • How fast is too fast?

Expert disagreement:

  • Timelines differ by decades
  • Some expect AGI soon, others never
  • Confidence often exceeds evidence

Source: Our World in Data

Be sceptical of confident predictions

Scenarios to consider

Scenario 1: Gradual improvement

  • AI gets better slowly
  • Society adjusts step by step
  • Most likely?

Scenario 2: Capability jumps

  • Sudden breakthroughs with unexpected capabilities
  • Rapid deployment
  • Less time to adapt

Scenario 3: Plateau

  • Current approaches hit limits
  • Progress slows sharply
  • New methods needed

Scenario 4: Transformative AI

  • Systems vastly more capable than humans
  • Reshapes the economy and society
  • Either very good or very bad
  • Uncertain timeline

Source: Dallas Fed (2025), via AEI

Dallas Fed scenarios: trend, AI boost, benign singularity, extinction

Stuart Russell’s perspective

Russell’s argument (TED talk, 2017):

  • We build systems whose objectives we don’t fully control
  • Standard AI optimises a fixed objective
  • We can’t specify objectives correctly
  • So we need a different approach

His three principles (Human Compatible, 2019):

  1. The machine’s only goal is to realise human preferences
  2. It is uncertain what those preferences are
  3. It learns them by watching human behaviour

“You can’t fetch the coffee if you’re dead”

  • A machine with a fixed goal resists being switched off
  • A machine unsure of our goals lets us switch it off
  • We would only do so if it were doing something wrong

Source: TED

Russell co-wrote the most-used AI textbook, AI: A Modern Approach, with Peter Norvig

Existential risk: the debate

Those who worry:

  • AI could become uncontrollable
  • Misaligned powerful AI = catastrophe
  • Even a small probability × huge harm = important
  • “We might not get a second chance”
  • Hinton, Bengio, Russell and many others

Those who are sceptical:

  • The risk is speculative
  • It distracts from real present harms
  • “Sci-fi thinking”, not grounded
  • We control the off switch
  • Capabilities are overstated

Overall:

  • Serious researchers hold both views
  • No one has decisive evidence yet

What can we do?

Research directions

Technical safety research:

Area Goal
Interpretability Understand AI internals
Robustness Resist adversarial inputs
Alignment Ensure AI pursues intended goals
Oversight Scale human supervision
Honesty AI that doesn’t deceive

Growing field:

  • Safety teams at Anthropic and Google DeepMind, plus academic labs
  • Still small next to capabilities work

Careers:

  • High-impact work with a talent shortage
  • Technical and non-technical roles

Source: The Wall Street Journal (2026)

Interested? Look into 80,000 Hours, AI safety bootcamps and safety-focused labs

Where you fit in

People who understand AI are often absent from policy debates. That is a problem!

As citizens:

  • Ask of any AI rule: what problem does it solve, and does it solve it?
  • Vote, comment on public consultations, write to representatives
  • Most AI coverage confuses hype with capability
  • You can do better 😉

If you work with AI or data:

  • Ask what happens when your system fails or is misused
  • Know what data you train on and who it affects
  • Talk to people outside your field about how they experience AI

Thinking about AI risk clearly

Common mistakes:

  • Dismissing concerns because current AI seems harmless
  • This ignores how fast capabilities grow
  • Catastrophising from scenarios with no evidence
  • Confusing “possible” with “probable”

Better habits:

  • Separate near-term harms from speculative long-term risks
  • Ask: what evidence would change my mind?
  • Experts genuinely disagree, and that is okay!
  • Be wary of certainty

From history:

  • Technology risks rarely follow worst-case predictions
  • The biggest harms come from problems we did not anticipate

Summary

Main takeaways

AI safety

  • Safe systems now make safer systems later
  • Concrete problems: safe exploration, side effects, reward hacking
  • Uncertainty is a reason to act now

Alignment

  • Getting AI to do what we want
  • Hard because of specification, Goodhart’s law and value disagreement
  • Main methods: RLHF, Constitutional AI, debate
  • None is a complete solution

Future trajectories

  • Nobody knows where AI is heading
  • Expert timelines differ by decades
  • Plan for multiple futures

What we can do

  • Technical safety research
  • Scrutinise AI rules as citizens
  • Ask how your own systems can fail

Choices about alignment, oversight and governance will shape AI’s future

… and that’s all for today!