DATASCI 101: Introduction to AI Applications

Lecture 23: Misinformation, Deepfakes and Trust Online

Danilo Freire
danilo.freire@emory.edu

Department of Data and Decision Sciences
Emory University

Welcome back! 🕵️

Recap of last class

  • AI and wellbeing: attention, mental health, environment
  • Algorithms optimise engagement, not wellbeing
  • Filter bubbles look overstated; mental health effects are debated
  • AI mental-health tools: short-term promise, thin long-term evidence
  • Training is a one-off cost; inference at scale keeps growing
  • Incentive structures beat intentions
  • Today: what happens when AI is used to deceive?

Pope Francis in Balenciaga! 😂

Lecture overview

What we will cover today

Part 1: Misinformation and AI

  • What is misinformation?
  • AI and synthetic media
  • Why it matters now

Part 2: Deepfakes

  • Technical foundations
  • Real-world examples

Part 3: Why we fall for it

  • Evolutionary wiring and fake news
  • System 1 vs System 2
  • Confirmation bias and motivated reasoning
  • Bandwagon effect and the “in the know” feeling

Part 4: Responses

  • Technical detection
  • Platform policies
  • Legal frameworks
  • Media literacy

Misinformation: definitions and scale

Three kinds of false content

The trichotomy (Wardle & Derakhshan, 2017, Council of Europe):

Term Definition
Misinformation False information spread without intent to deceive
Disinformation False information spread deliberately to deceive
Malinformation True information shared to cause harm

A broader concept: synthetic media (content created or manipulated by AI)

Why the distinction matters:

  • Intent shapes the response
  • Different actors, different motivations
  • Legal treatment varies by category

The old problem:

  • Misinformation is ancient. Technology changes scale and speed

Source: CUNY Library

A grandmother sharing a bad health tip and a state disinformation campaign need different responses. Intent matters

Why AI changes things

Before generative AI:

  • Fake images required Photoshop skills
  • Fake videos needed studios, fake audio needed voice actors
  • Production was expensive and slow, and detection often possible

After generative AI:

  • Anyone can make convincing fakes at near-zero cost
  • Quality improves rapidly, so detection gets harder

The scale problem:

  • Thousands of fake articles in minutes, targeted at specific audiences
  • Human moderators can’t keep up (Lazer et al., 2018)

Source: Encyclopedia Britannica

The “Protocols of the Elders of Zion” (1903) was fabricated and fuelled hatred for a century. Now anyone can produce the same in minutes

The attention economy revisited

How platforms amplify misinformation:

  • Engagement = revenue, and outrage drives engagement
  • False information is often more engaging
  • Algorithms optimise for clicks, not truth

The correction problem:

  • Lies spread faster than corrections
  • False stories reach millions, corrections reach thousands
  • Even debunked claims persist in memory

Vosoughi et al. (2018):

  • Analysed 126,000 stories on Twitter
  • False stories were 70% more likely to be retweeted and reached 1,500 people 6x faster

Source: NYU School of Engineering (a Facebook study)

Platform incentives work against truth. Not a bug, a feature of ad-supported media

Deepfakes

What are deepfakes?

Technical definition:

How they work:

  1. Collect training data (images/video of the target)
  2. Train a neural network to generate the target’s face
  3. Apply it to a source video or image
  4. Refine to reduce artefacts

Main techniques:

Technical detail: Tolosana et al. (2020)

Real-world examples

Political deepfakes:

Example Impact
Obama PSA (2018) BuzzFeed/Jordan Peele demonstration
Zelensky surrender video Attempted wartime deception
Indian election audio AI voice clones of politicians

Financial fraud:

Non-consensual intimate imagery:

Source: Mikael Thalen on X; story: NPR

The first big audit (Sensity, 2019) found almost all deepfakes were pornographic, and almost all targeted women

RAND’s primer on deepfakes

  • Deepfakes are one tool among many
  • “Cheap fakes” often work just as well (Paris & Donovan, 2019 coined the term)
  • Context matters more than technical quality
  • Social vulnerability enables technical attacks

The liar’s dividend:

  • When anything could be fake, real evidence can be dismissed as fake
  • “That video of me is a deepfake!”
  • Truth becomes contested terrain

RAND’s four risks:

  • Manipulated elections and deeper social divisions
  • Lower trust in institutions
  • Undermined journalism (the liar’s dividend)

Source: RAND Corporation

Label every deepfake tomorrow and the damage stands: people now doubt real video too

The authenticity crisis

Chesney & Citron (2019):

  • Deepfakes are an epistemic threat: a danger to public knowledge, not just privacy or fraud
  • News loses the ability to verify its own sources
  • Politicians get plausible deniability for anything caught on camera
  • People retreat further into the views they already hold

What the empirical work has found since:

  • Vaccari & Chadwick (2020): UK survey (N=2,005): deepfakes leave people uncertain, not deceived, and trust news less
  • Schiff, Schiff & Bueno (2024): five experiments, 15,000 Americans
  • False “deepfake!” claims help politicians dodge scandals, but mostly over text, not video

Discussion: what would convince you?

Scenario:

A video emerges of a political candidate you support saying something terrible. The candidate claims it’s a deepfake.

  1. How would you decide if it’s real?
  2. What evidence would convince you either way?
  3. Does your political alignment affect your judgment?
  4. What if experts disagree?

Why we fall for it

Are we hard-wired for fake news?

The evolutionary angle:

  • Our brains evolved for small groups and immediate threats
  • Novelty once meant survival-relevant information. We still react to surprising claims that way
  • False news is almost by definition more novel than truth

Why outrage spreads:

  • Threat detection is fast and automatic
  • Emotionally charged content bypasses careful evaluation
  • Fear and disgust trigger sharing
  • We evolved to warn the group, not to fact-check first

System 1 vs System 2

Kahneman’s (2011) dual-process theory:

System 1 System 2
Speed Fast, automatic Slow, deliberate
Effort Effortless Requires concentration
Mode Intuitive, emotional Analytical, logical
Default? Yes Only when triggered

Social media is a System 1 environment:

  • Infinite scroll rewards quick reactions: see a video, react, share, move on
  • System 2 never kicks in unless something jolts you
  • Deepfakes are designed to feel real at System 1 speed

The problem:

  • Catching a fake requires System 2, but outrage and fear suppress it
  • Pennycook & Rand (2019): higher cognitive reflection scores predict spotting fake news, regardless of party

Confirmation bias

What it is:

  • We seek evidence that confirms what we already believe and dismiss what contradicts it
  • This isn’t laziness; it’s how brains manage information overload
  • Nickerson (1998) called it “ubiquitous”, a top candidate for reasoning’s worst flaw

How it works with deepfakes:

  • Who said it? We believe a trusted source even when fake, and doubt a distrusted one even when real
  • Taber & Lodge (2006): people argue against opposing evidence, and strong partisans polarise
  • Corrections feel like an attack on identity, even when they aren’t

Confirmation bias is not about intelligence. Educated people are sometimes better at rationalising bad evidence

The bandwagon effect

What Asch (1956) showed in the 1950s:

  • People will deny what their own eyes tell them if enough others disagree
  • About 75% of participants conformed at least once on an obvious task
  • Bond & Smith (1996) pooled 133 Asch studies in 17 countries: weaker in individualistic cultures

On social media, numbers are social proof:

  • A video with 2 million views “must be real”
  • We outsource our judgment to the crowd

How this helps deepfakes spread:

  • Early shares snowball, each adding perceived legitimacy
  • By the time fact-checkers respond, millions have seen it
  • Taking it down can increase belief (“they’re hiding something”)

Asch used lines on a card. The same psychology now runs on deepfakes shared millions of times

The “in the know” feeling

Sharing as social currency:

  • Breaking news makes you feel like an insider
  • “Did you see this?!” buys status, and being first gets the attention

Why this matters:

Combined with the other biases:

  • Confirmation bias picks what we share (things we agree with)
  • The bandwagon effect picks when (once it’s already popular)
  • The “in the know” feeling picks how fast (immediately, no checking)

Source: Adam Grant

We share what confirms our views and makes us look informed. The truth can wait, the dopamine hit can’t

Responses

Technical detection

Detection approaches:

Method How it works
Artefact detection Unnatural blinking, lighting
Biological signals Pulse, micro-expressions
Source forensics Compression artefacts, metadata
AI vs AI Train detectors on known fakes

Why it’s hard:

  • Detectors can be fooled, and adversarial training improves generators
  • Real-time detection is difficult
  • Scale: millions of images, few moderators
  • Groh et al. (2022): 15,016 people and a detector made different mistakes; together they did best

Content provenance

A new idea:

  • Instead of catching fakes afterwards, verify the real thing at the source
  • The C2PA standard signs a photo at capture, then logs every edit
  • Backed by Adobe, Microsoft, Google, OpenAI, Meta, Amazon and the BBC
  • Viewers check the chain back to source, and unsigned content looks suspect

Why it isn’t a fix on its own:

  • Needs near-universal adoption to be useful at all
  • Legacy content has no provenance to attach
  • Signatures can be stripped, and screenshots wash them out
  • Tracks creation, but doesn’t prevent it
  • Can tie images to a device or identity, raising privacy concerns

Platform policies

What platforms actually do:

What happens when platforms remove content:

  • Content moves to less moderated platforms, and removal can deepen distrust
  • Bak-Coleman et al. (2021): platforms are a planetary-scale system, to study like the climate
  • Piecewise content rules cannot fix system-level effects

Source: Vice - YouTube

Media literacy

What the evidence says works:

  • Pennycook et al. (2021): a small accuracy nudge before sharing improves the quality of news people pass along
  • Roozenbeek et al. (2022): inoculation videos teaching manipulation tactics, with measurable effects across 22,632 YouTube users
  • Guess et al. (2020): media literacy tips improved discernment of false news by 26.5% in the US and 17.5% in an online Indian sample
  • Effects are real but often short-lived (Capewell et al., 2024). None scales to billions alone

Summary

Main takeaways

Misinformation and AI

  • Misinformation is old. AI changes scale and speed
  • Platform incentives reward engagement, not truth

Deepfakes

  • Quality is improving fast: fraud, political manipulation, and NCII above all
  • The liar’s dividend lets anyone dismiss real evidence

Why we fall for it

  • Novelty and outrage travel on System 1, before System 2 wakes
  • Confirmation bias is the vulnerability no patch can fix
  • Bandwagon and “in the know” decide when and how fast we share

Responses

  • Detection is necessary but insufficient; provenance is promising but years away
  • Law is slow and jurisdictional; literacy helps but doesn’t scale
  • No single fix. Layering defences is the best we have

… and that’s all for today!