DATASCI 101: Introduction to AI Applications

Lecture 23: Misinformation, Deepfakes and Trust Online

Danilo Freire
danilo.freire@emory.edu

Department of Data and Decision Sciences
Emory University

Welcome back! 🕵️

Recap of last class

  • Last class: AI and wellbeing
  • Algorithms optimise engagement, not wellbeing
  • Filter bubbles look overstated, while mental health effects are debated
  • AI therapy helps short term, little data beyond
  • Training is a one-off cost, but inference at scale keeps growing
  • Platforms follow incentives, not stated intentions
  • Today: what happens when AI is used to deceive?

Pope Francis in Balenciaga! 😂

Lecture overview

What we will cover today

Part 1: Misinformation and AI

  • What is misinformation?
  • AI and synthetic media
  • Why it matters now

Part 2: Deepfakes

  • Technical foundations
  • Real-world examples

Part 3: Why we fall for it

  • Evolutionary wiring and fake news
  • System 1 vs System 2
  • Confirmation bias and motivated reasoning
  • Bandwagon effect and the “in the know” feeling

Part 4: Responses

  • Technical detection
  • Platform policies
  • Legal frameworks
  • Media literacy

Misinformation: definitions and scale

Three kinds of false content

Term Definition
Misinformation False information spread without intent to deceive
Disinformation False information spread deliberately to deceive
Malinformation True information shared to cause harm

Source: Wardle & Derakhshan (2017), Council of Europe

Synthetic media: content created or manipulated by AI

Responses differ by type:

  • Actors and motives differ
  • The law treats each differently

Source: CUNY Library

A grandmother sharing a bad health tip and a state disinformation campaign need different responses

Why AI changes things

Before generative AI:

  • Fakes needed Photoshop skills, studios or voice actors
  • Production was expensive and slow, and detection often possible

After generative AI:

  • Anyone can make convincing fakes at near-zero cost
  • Better quality makes detection harder
  • Thousands of fake articles in minutes, targeted at specific audiences
  • Human moderators can’t keep up

Source: Encyclopedia Britannica

The “Protocols of the Elders of Zion” (1903) was fabricated and fuelled hatred for a century. A chatbot can draft something similar in minutes

The attention economy revisited

How platforms amplify misinformation:

  • Engagement = revenue, and outrage drives engagement
  • False information is often more engaging

Corrections:

  • Lies spread faster than corrections
  • Corrections reach far fewer people than the original
  • Even debunked claims persist in memory

Vosoughi et al. (2018):

  • Analysed 126,000 stories on Twitter
  • False stories were 70% more likely to be retweeted
  • They reached 1,500 people 6x faster

Source: NYU School of Engineering (a Facebook study)

Platform incentives work against truth

Deepfakes

What are deepfakes?

Technical definition:

How they work:

  1. Collect images or video of the target
  2. Train a neural network to generate the target’s face
  3. Apply it to a source video or image
  4. Refine to reduce artefacts

Main techniques:

Technical detail: Tolosana et al. (2020)

Real-world examples

Political deepfakes:

Example Impact
Obama PSA (2018) BuzzFeed/Jordan Peele demonstration
Zelensky surrender video Attempted wartime deception
Indian election audio AI voice clones of politicians

Financial fraud:

Non-consensual intimate imagery:

Source: Mikael Thalen on X (story: NPR)

Nearly all deepfakes in Sensity’s 2019 audit were porn of women

RAND’s primer on deepfakes

  • Deepfakes are one tool among many
  • “Cheap fakes” often work just as well (Paris & Donovan, 2019 coined the term)
  • Context matters more than technical quality
  • Attacks exploit social divisions

The liar’s dividend:

  • Real evidence can be dismissed as fake
  • “That video of me is a deepfake!”

RAND’s four risks:

  • Manipulated elections
  • Deeper social divisions
  • Lower trust in institutions
  • Undermined journalism (the liar’s dividend)

Source: RAND Corporation

People now doubt real video too

Do deepfakes erode trust?

Chesney & Citron (2019):

  • Deepfakes threaten public knowledge, beyond privacy and fraud
  • Politicians can deny anything caught on camera
  • People retreat into views they already hold

Later studies:

Discussion: what would convince you?

A video shows a candidate you support saying something terrible. They say it’s a deepfake

  1. How would you decide if it’s real?
  2. What evidence would convince you either way?
  3. Does your political alignment affect your judgement?
  4. What if experts disagree?

Why we fall for it

Are we hard-wired for fake news?

Evolution:

  • Our brains evolved for small groups and immediate threats
  • Novelty once meant survival-relevant information
  • We still react to surprising claims that way
  • False news is usually more novel than truth (Vosoughi et al., 2018)

Why outrage spreads:

  • Threat detection is fast and automatic
  • Emotion bypasses careful evaluation
  • Fear and disgust trigger sharing
  • We evolved to warn the group before checking facts

System 1 vs System 2

Kahneman’s (2011) dual-process theory:

System 1 System 2
Speed Fast, automatic Slow, deliberate
Effort Effortless Requires concentration
Mode Intuitive, emotional Analytical, logical
Default? Yes Only when triggered

Social media is a System 1 environment:

  • Infinite scroll rewards quick reactions
  • System 2 kicks in only when something jolts you
  • Deepfakes are designed to feel real at System 1 speed

Catching fakes:

  • Spotting a fake needs System 2
  • Outrage and fear suppress it
  • Pennycook & Rand (2019): people who reflect more spot more fake news, whatever their party

Confirmation bias

  • We seek evidence that confirms what we already believe
  • We dismiss what contradicts it
  • It’s how brains manage information overload
  • Nickerson (1998) called it “ubiquitous”, a top candidate for reasoning’s worst flaw

With deepfakes:

  • We believe a trusted source even when it’s fake
  • We doubt a distrusted source even when it’s real
  • Taber & Lodge (2006): people argue against opposing evidence
  • Strong partisans polarise further
  • Corrections feel like an attack on identity

Educated people are sometimes better at rationalising bad evidence

The bandwagon effect

Asch (1956):

  • People deny what their own eyes tell them if enough others disagree
  • About 75% conformed at least once on an obvious task
  • Bond & Smith (1996): 133 studies in 17 countries
  • Conformity is weaker in individualistic cultures

On social media, numbers are social proof:

  • A video with 2 million views “must be real”
  • We outsource our judgement to the crowd

With deepfakes:

  • Each early share adds perceived legitimacy
  • By the time fact-checkers respond, millions have seen it
  • Taking it down can feed “they’re hiding something” stories

The “in the know” feeling

Sharing as social currency:

  • Breaking news makes you feel like an insider
  • Being first buys status and attention

Sharing before checking:

  • People race to share before verifying
  • Gabielkov et al. (2016): 59% of news links shared on Twitter were never clicked

How the biases combine:

  • Confirmation bias picks what we share (things we agree with)
  • The bandwagon effect picks when (once it’s already popular)
  • The “in the know” feeling picks how fast (immediately)

Source: Adam Grant

Responses

Technical detection

Detection approaches:

Method How it works
Artefact detection Unnatural blinking, lighting
Biological signals Pulse, micro-expressions
Source forensics Compression artefacts, metadata
AI vs AI Train detectors on known fakes

Limits:

  • Detectors can be fooled
  • Adversarial training improves generators
  • Real-time detection is difficult
  • Too many images for too few moderators
  • Groh et al. (2022): 15,016 people and a detector made different mistakes, so together they did best

Content provenance

A new idea:

  • Verify real content at the source instead of hunting fakes
  • The C2PA standard signs a photo at capture and logs every edit
  • Backed by Adobe, Microsoft, Google, OpenAI, Meta, Amazon and the BBC
  • Viewers can trace content back to its source
  • Unsigned content looks suspect

Limits:

  • Needs near-universal adoption
  • Old content has no provenance
  • Signatures can be stripped
  • Screenshots remove them
  • Tracks creation but doesn’t prevent it
  • Can tie images to a device or person, a privacy risk

Platform policies

Platform rules:

After removal:

  • Content moves to less moderated platforms
  • Removal can deepen distrust
  • Bak-Coleman et al. (2021): platforms are a planetary-scale system, to study like the climate
  • Case-by-case rules can’t fix system-level effects

Source: Vice - YouTube

Media literacy

What works:

Summary

Main takeaways

Misinformation and AI

  • AI makes fakes cheap and fast to produce
  • Platforms reward engagement over truth

Deepfakes

  • Quality improves fast
  • Used for fraud, political manipulation and sexual images
  • The liar’s dividend lets anyone dismiss real evidence

Why we fall for it

  • Novelty and outrage trigger System 1 before System 2
  • Confirmation bias shapes what we believe and share
  • The bandwagon effect and the “in the know” feeling speed up sharing

Responses

  • Detectors can be fooled
  • Provenance needs near-universal adoption
  • Laws are slow and stop at borders
  • Literacy effects fade fast
  • We need several defences at once

… and that’s all for today!