DATASCI 101: Introduction to AI Applications

Lecture 19: Privacy and Data Protection

Danilo Freire
danilo.freire@emory.edu

Department of Data and Decision Sciences
Emory University

Welcome back!

Recap of last class

  • EU AI Act: first comprehensive AI law
  • Risk tiers: banned, high, limited, minimal
  • Fines up to 7% of global turnover
  • High-risk rules delayed to December 2027
  • US: sectoral rules plus state laws
  • A 2025 executive order targets state AI laws
  • China: censorship, innovation, algorithm registry
  • No global consensus, but principles may converge
  • Today: how does AI intersect with privacy?

Oh, well 😅

Lecture overview

What we will cover today

Part 1: Privacy in the AI age

  • Why AI makes privacy harder
  • Data collection at unprecedented scale
  • Inferring sensitive traits

Part 2: Legal frameworks

  • GDPR and data protection principles
  • Rights of individuals
  • US privacy landscape

Part 3: Technical approaches

  • Differential privacy
  • Federated learning
  • Synthetic data

Part 4: Challenges and tensions

  • Case study: Clearview AI
  • Discussion: where’s your line on privacy?
  • What can you do?

Meme of the day

Source: CanIPhish

Good news of the day

Source: Financial Times

Privacy in the AI age

Why AI makes privacy harder

Older privacy laws did not anticipate these problems:

Scale of collection:

  • Unlike staffed surveillance, AI runs 24/7, almost for free
  • Every click, scroll and location ping becomes data

Inference capabilities:

  • AI predicts sensitive traits from harmless data
  • Shopping → health, typing → mood, friends → politics
  • It infers what you never disclosed

Persistence:

  • Data doesn’t decay: today’s models affect you forever
  • Decisions follow you across contexts
  • “Right to be forgotten” is technically hard

Source: AI Multiple

“If you’re not paying for the product, you are the product.” Even paying customers are often the product

What can be inferred from your data?

Research shows AI can predict:

Data source Inferred
Facebook likes Political views, sexuality, personality
Smartphone sensors Depression, anxiety, Parkinson’s
Typing patterns Age, gender, emotional state
Purchase history Pregnancy, health conditions
Location data Home address, workplace, religion
Voice recordings Emotional state, health, age

Example: Target and pregnancy

  • Target mailed baby coupons to a pregnant teen
  • Her father complained: she was “still in high school”
  • Days later, he apologised: she was pregnant
  • The algorithm knew before her family did (NYT)

Source: Time

You can control what you share, but you cannot control what can be inferred from it

Training data and privacy

The training data problem:

  • Web-scale data includes personal information
  • Models can memorise and regurgitate training data
  • Your name, address or phone number may be in there

Demonstrated attacks:

  • Carlini et al. (2021) pulled verbatim data from GPT-2
  • Leaked: names, phone numbers, emails
  • Extraction attacks keep improving

The consent problem:

  • Most people don’t know what’s in training sets
  • “Publicly available” ≠ “consented to AI training”

Source: The Hacker News

An LLM may know things about you that you never told it

Discussion: would you share?

Quick poll (raise your hand):

Would you share your data if…

  1. A health app predicts disease but sells data to insurers?
  2. A smart speaker helps but records all conversations?
  3. A job site tailors results but shares with employers?
  4. A social app connects friends but profiles you for ads?
  5. An AI tutor helps you learn but reports to your school?

The privacy paradox:

  • People value privacy but share anyway

Why?

  • Benefits come now, but harms come later
  • Default settings favour sharing
  • Terms of service are unreadable
  • “Everyone does it”
  • We’re bad at judging risk

GDPR fundamentals

The General Data Protection Regulation (2016/679, in force since 2018) is the EU’s comprehensive privacy law

Main principles:

  1. Lawfulness, fairness, transparency
  2. Purpose limitation: use data only for the stated purpose
  3. Data minimisation: collect only what’s necessary
  4. Accuracy: keep data correct
  5. Storage limitation: delete it when no longer needed
  6. Integrity and confidentiality: protect data properly
  7. Accountability: demonstrate compliance

Six lawful bases: consent, contract, legal obligation, vital interests, public task and legitimate interests

  • Legitimate interests is the most contested basis
  • Firms use it to skip consent, even for AI training

Like the EU AI Act, GDPR covers anyone processing EU residents’ data, wherever they are. A US startup with EU users must comply

Individual rights under GDPR

You have the right to:

Right What it means
Access Get a copy of your data
Rectification Correct inaccurate data
Erasure “Right to be forgotten”
Portability Move data to another service
Restriction Limit processing
Object Stop certain processing
Automated decisions Ask for a human in significant ones

Article 22: automated decisions

  • Right not to be subject to purely automated decisions
  • Consequential decisions need human involvement
  • Exceptions: contracts and explicit consent

Source: GDPR.eu (Proton)

In practice, these rights are hard to use: firms hide forms, reply slowly or claim exemptions

GDPR and AI tensions

  1. Purpose limitation vs model training
    • Can data given for X train a model for Y?
    • AI companies claim legitimate interest
  2. Data minimisation vs big data
    • AI wants more data than GDPR allows
    • What’s “necessary” for an LLM? Everything?
  3. Right to explanation vs black boxes
    • You can ask why an AI decided about you
    • Many AI systems can’t explain themselves
  4. Right to erasure vs model training
    • Can you demand removal from a trained model?
    • “Unlearning” is technically hard

Source: CertPro

GDPR predates LLMs, so applying 2018 law to 2026 technology raises open questions

US privacy landscape

No comprehensive federal law, just a patchwork of sector rules:

Law Scope Protects
HIPAA Healthcare data Patient medical records
FERPA Education records Student academic data
COPPA Children’s data Under-13 online activity
GLBA Financial data Bank and loan records
FCRA Credit reporting Credit scores and history
ECPA Electronic comms Emails, calls, stored data

States fill the gap:

Source: IAPP

GDPR enforcement

Selected major fines:

Company Fine Violation
Meta (2023) €1.2B Sent EU data to the US without adequate safeguards
Amazon (2021) €746M Targeted ads without valid consent (fine annulled in 2026, breach upheld)
Meta (2022) €405M Exposed children’s contact details on Instagram
Google (2022) €150M Made rejecting cookies harder than accepting them
TikTok (2023) €345M Failed to protect children’s data and settings

Patterns:

  • Big tech is the main target
  • Regulators focus on data transfers, consent and children
  • Fines are getting larger
  • Enforcement varies, and critics call Ireland’s regulator slow

Source: Secureframe

Meta made ~$135 billion in 2023. Is €1.2B a fine or a cost of doing business?

Technical approaches to privacy

Differential privacy

Differential privacy adds random noise to results, so no one can tell if you are in the data (Dwork & Roth, 2014)

  • “How many patients have diabetes?” True answer: 137
  • Released: 134 or 141, close enough for research
  • Patterns stay visible, but individuals stay hidden

Epsilon (ε) is set by the data owner before release:

  • ε limits how much one person can change a result
  • Small ε (below 1): more noise, strong privacy
  • Large ε (around 10): less noise, weak privacy
  • Small ε: no one can tell if the count is 136 or 137

Source: Flower AI

Used by: Apple (emoji suggestions), Google (Chrome stats), US Census (2020, ε = 19.61)

Federated learning

Federated learning trains models without pooling the data (McMahan et al., 2017)

How it works:

  1. A server sends the same model to many devices
  2. Each device trains it on its local data
  3. Each returns only what it learned (weights)
  4. The server averages updates and repeats

Privacy gains and limits:

  • Raw data never leaves your device
  • Updates can still leak information
  • Coordinating thousands of devices is hard

Source: Wikipedia

Example: Google’s phone keyboard (Gboard) learns from your typing, but your texts stay on your phone

Synthetic data for privacy

Synthetic data looks real but isn’t (Jordon et al., 2022)

How it works:

  • Train a generative model on real data
  • Generate new, fake records
  • Train your AI on the fakes

Benefits and limits:

  • Fewer direct links to real people
  • But generators can still leak training records
  • Easier to share for research
  • May miss rare cases

Source: GOV.UK

Used by: health researchers (fake patient records), banks (fraud detection tests)

Do these techniques actually help?

What they can do:

  • Allow research on sensitive health data
  • Give engineers concrete tools
  • Differential privacy is provable
  • Federated learning keeps data on the device

What they cannot do:

  • Stop data collection in the first place
  • Fix the power imbalance with platforms
  • Make users understand data use

However:

  • The best protection is not collecting data
  • AI firms’ logic: more data = more profit
  • Technical fixes work within the system, not on it

Beware “privacy washing”: differential privacy with large ε (weak privacy), or federated learning that still collects metadata

Challenges and tensions

Case study: Clearview AI

  • Jan 2020: the NYT exposed 3 billion scraped photos
  • Clearview now has over 70 billion social media photos
  • It built a face search engine without consent or notice
  • Sold to thousands of US police agencies
  • Police ran ~1M searches by 2023

Legal fallout:

  • The ACLU sued under Illinois BIPA (2020)
  • 2022 settlement bans sales to most private firms
  • France (CNIL) and Italy: €20M fine each
  • Netherlands: €30.5M fine (2024)
  • Australia, Canada: ordered to delete data
  • UK: still in court

Source: Library of Congress

Sold as police-only, but accounts went to retailers, banks and investors’ friends. Critics call it a “perpetual police line-up”

Quick quiz: what’s wrong and how would you fix it?

  1. A social media company uses your messages to train an AI without telling you

  2. A shopping app logs your location every 30 seconds, even when closed

  3. A company keeps customer data “just in case” with no deletion policy

  4. An AI hiring tool rejects candidates without any human review

Where’s your line?

Scenario:

A new app offers free health monitoring. It tracks:

  • Your heart rate and sleep
  • Your location and activity
  • What you eat and drink
  • Your social interactions

In exchange, it provides:

  • Personalised health advice
  • Early warning of health issues
  • Discounts from health insurers
  • Connection with others like you

Discuss:

  1. Would you use this app?
  2. What would make you change your mind?
  3. What data would be “too much”?
  4. Does it matter if a firm, hospital or state runs it?

No right answer: identify your own values and the trade-offs you’ll accept

What can you do?

Individual actions:

  • Review your privacy settings
  • Use privacy tools (Signal, DuckDuckGo, Firefox)
  • Limit location sharing
  • Use a different email for each service
  • Use your GDPR/CCPA rights (access, deletion)
  • Be sceptical of “personalisation”

Limits of individual action:

  • Opting out often means losing the service
  • Privacy is collective: others’ data reveals yours
  • The power imbalance is structural

Collective action:

  • Back privacy laws
  • Demand transparency from companies
  • Support EFF, EPIC or noyb
  • Choose privacy-respecting services
  • Vote for pro-privacy candidates

Signal over WhatsApp helps, but only legislation changes how insurers use your health data

Summary

Main takeaways

AI and privacy

  • Collection at unprecedented scale
  • Inference makes “non-sensitive” data sensitive
  • Models can memorise and leak personal data

Legal frameworks

  • GDPR: comprehensive and rights-based
  • US: a sectoral patchwork
  • AI development and data protection pull against each other

Technical approaches

  • Noise, decentralised training, artificial data
  • All have trade-offs

Key insights

  • Collection is the core problem
  • The same data enables benefits and surveillance
  • Privacy is collective, not just individual
  • Technical fixes don’t address power imbalances
  • Rules and oversight decide whether data helps or harms

… and that’s all for today! 🎉

Appendix

Quick quiz: possible answers

1. Messages used to train AI

  • Breaks transparency and purpose limitation
  • Fix: tell users and ask for opt-in consent

2. Location every 30 seconds

  • Breaks data minimisation
  • Fix: track only while the app is open
  • Use approximate location where possible

3. Data kept “just in case”

  • Breaks storage limitation
  • Old data adds breach risk for no benefit
  • Fix: set retention periods, then delete

4. Hiring tool with no human review

  • Breaks GDPR Article 22 on automated decisions
  • Fix: a human reviews every rejection
  • Candidates can ask why and contest it

Where’s your line: points to raise

1. Would you use it?

  • The app is free, so your data likely pays for it
  • Insurer discounts can turn into higher premiums

2. What would change your mind?

  • Data stays on your phone
  • No sale to third parties
  • Easy deletion when you quit

3. What is “too much”?

  • Location, diet and sleep reveal health conditions
  • Social data exposes your friends too

4. Who runs it?

  • Tech firm: ads and data sales
  • Hospital: HIPAA applies
  • Most health apps fall outside HIPAA
  • Government: hard to opt out