DATASCI 101: Introduction to AI Applications

Lecture 26: Course Revision

Danilo Freire

danilo.freire@emory.edu

Department of Data and Decision Sciences
Emory University

Welcome back! 😊

Course review

• Last class: safety and alignment
• Safety problems include side effects and reward hacking
• Alignment: getting AI to do what we want
• Goals are hard to specify, and people disagree on values
• No method is complete: RLHF, Constitutional AI, debate
• Expert timelines differ by decades
• Today: one slide per lecture
• Link boxes point to lectures where an idea returns

A quick favour 🙏

Course evaluations are open!

• I would love to hear your thoughts on the course!
• Log on to Canvas and go to Account → Profile → Course Evaluations
• It takes a few minutes on any device
• Thank you so much! 😊

Module 0: Orientation
(Lectures 1-2)

What AI is and how we got here (Lectures 1-2)

• Today’s AI is narrow: good at specific tasks, weak outside them (McKinsey, 2024)
• LLMs predict the next token but don’t understand language as we do (Radford et al., 2018)
• Rules (1950s-80s), AI winters, backpropagation (1986), transformers (2017)
• ChatGPT added instruction tuning and RLHF to a large model

Next-token prediction explains hallucinations (L11). RLHF returns in L4 and L25.

The transformer architecture (2017)

Module 1: How AI systems are designed
(Lectures 3-5)

Data, labels, and the proxy problem (Lecture 3)

• A better algorithm cannot fix bad data (Ng, 2021)
• Labels often measure a proxy: healthcare spending for health needs (Obermeyer et al., 2019)
• Selection bias: training data differ from the cases the model meets (Gender Shades)
• Cohen’s kappa: agreement beyond chance (0.61-0.80 is substantial)

The proxy problem returns as Goodhart’s Law (L5) and measurement bias (L17).

Data-centric vs model-centric AI (Karpathy, 2018)

Learning paradigms and RLHF (Lecture 4)

Paradigm	Learns from	Example
Supervised	Labelled examples	Spam detection
Unsupervised	Unlabelled data	Customer segments
Reinforcement	Rewards for actions	AlphaGo, RLHF

• Bias-variance: simple models underfit, complex ones overfit
• RLHF: a reward model learns human rankings, then RL tunes the LLM to it
• Reward hacking: the model games the reward instead of doing the task

The bias-variance trade-off

Metrics and Goodhart’s Law (Lecture 5)

• Accuracy paradox: with 1 case in 1,000, “always healthy” scores 99.9%
• Precision: share of flags that were right
• Recall: share of real cases caught
• Cross-validation catches overfitting
• Subgroup analysis catches gaps that averages hide
• Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure”

Goodhart returns in alignment (L25). Subgroup analysis becomes disaggregated evaluation (L14).

The confusion matrix: TP, TN, FP, FN

Module 2: Language and perception
(Lectures 6-7, 10)

Tokens, embeddings, and context (Lecture 6)

• About 100 tokens make 75 English words (more for other languages)
• In embeddings, words with similar meanings have nearby vectors (Mikolov et al., 2013)
• If a chat exceeds the context window, you get an error or a cut-off answer
• A higher temperature makes answers more random and creative

Embeddings return for images (L7) and for RAG search (L12).

king - man + woman ≈ queen

How machines see and hear (Lecture 7)

• Images are grids of numbers. CNNs learn edges first, then whole objects
• A spectrogram turns sound into a picture, so image methods work on audio
• Text, images and audio all become embeddings of the same kind
• After that, the LLM cannot tell whether the input was text, an image or audio
• That is why ChatGPT, Claude and Gemini can read images
• Should AI images and voices need watermarks, disclosure or consent?

CLIP uses the same embedding idea as L6. It was the basis for the image generators behind deepfakes (L23).

CLIP: images and text in one shared space

Prompting techniques (Lecture 10)

• No prompt works everywhere, so test on your data
• PTCF: persona, task, context, format
• Specific prompts get better answers than vague ones
• Few-shot prompts show the model 3-5 examples
• Chain-of-thought helps on multi-step problems
• Hidden system prompts set each product’s rules
• Tool-using ReAct agents risk prompt injection
• Models fail unpredictably, so verify (jagged AI)

Agents return in L15. Prompt injection comes up in L14, L15 and L16.

Chain-of-thought prompting

Module 3: Retrieval, generation, pipelines and agents
(Lectures 11-12, 14-16)

Hallucinations and creativity (Lecture 11)

• Hallucinations are false answers that sound plausible
• LLMs predict likely words, not true ones
• Sycophantic models cave when you push back
• Making things up helps in fiction but hurts with facts
• Fake citations got two lawyers sanctioned
• Check numbers, names and every citation

Root cause: next-token prediction (L1 and L2). One fix: RAG (L12).

Same mechanism, different outcomes

RAG and semantic search (Lecture 12)

• RAG feeds your documents to the model
• Semantic search finds text by meaning
• RAG chunks, embeds and retrieves your files
• Retrieval cut hallucinated replies by over 60%
• Models miss facts lost in the middle of long inputs
• Gemini Notebook and ChatGPT do this without code
• RAG reduces errors, so still check the citations

Uses embeddings (L6) against hallucinations (L11). L16 connectors work like RAG with live data.

The RAG pipeline

Pipelines and what can go wrong (Lecture 14)

• Every step can fail: input, model, filters, output
• Four failure types: hallucination, policy, drift, pipeline
• In drift, inputs or their meaning change
• Models also degrade so slowly that nobody notices
• Companies monitor errors and hallucination rates
• Validation checks both inputs and outputs
• Test properties like safety, not exact outputs

Drift is why models need watching after launch: the world changes, the model doesn’t.

The AI application stack

Documentation as accountability (Lecture 14)

• Datasheets record where data came from and why
• Model cards list intended and out-of-scope uses
• System cards document whole AI products
• A 90% average can hide 70% for one group
• So results should be reported by group
• Much training data was taken without consent

The EU AI Act will require this documentation for high-risk systems from December 2027 (L18).

Datasheets: the “nutrition label” for datasets

AI agents (Lecture 15)

• An agent uses tools in a loop to reach a goal
• Chatbots answer, but agents act
• In ReAct, the model plans and software acts
• Autonomy is a dial, from workflows to free agents
• Errors compound, so delegate what you can undo
• Use permissions and sandboxes, not prompts
• Untrusted text can hijack an agent

Agents extend ReAct (L10). Irreversible actions return as safe exploration (L25).

The agent loop. Source: ByteByteGo

Setting up AI (Lecture 16)

• Four layers: instructions, knowledge, memory, tools
• A setup is an instruction that stays between chats
• Constraints work better than personas
• Keep rules few and first
• Memory is on by default, so read it or go incognito
• A text file is not a permission system
• Give apps least access and confirm writes

Instructions are prompting (L10) made permanent. Connectors turn a chatbot into an agent (L15).

Anthropic’s augmented LLM

Module 4: Data ethics and bias
(Lecture 17)

Where bias comes from (Lecture 17)

• AI learns from human data, so it inherits our biases
• Historical bias bakes past discrimination into models
• Groups missing from the data get worse results
• Neutral proxies like ZIP codes can encode inequality
• Predictions shape new data in feedback loops
• No model meets every fairness test at once
• Choosing a fairness definition is a values question

The proxy problem comes from L3. Checking by group returns as disaggregated evaluation (L14).

Bias at every stage

Module 5: Policy, governance and social impact
(Lectures 18-19, 21)

AI regulation around the world (Lecture 18)

• AI is regulated for market failures and rights
• The EU AI Act sorts AI into four risk tiers

Tier	Rule	Examples
Unacceptable	Banned	Social scoring
High	Strict checks	Hiring, credit, policing
Limited	Transparency	Chatbots, deepfakes
Minimal	None	Spam filters, video games

• High-risk rules start in December 2027
• The US has no federal AI law, so states fill the gaps
• China’s AI rules bind companies, not the state
• EU rules may become global standards

High-risk systems are the hiring and credit cases from L17. GDPR (L19) is the Brussels Effect in practice.

The EU AI Act: the first comprehensive AI law

Privacy and data protection (Lecture 19)

• AI collects personal data at unprecedented scale
• Inference makes harmless data sensitive
• Models can memorise and leak personal data
• GDPR limits automated decisions
• The US has a patchwork of state laws
• Privacy tech does not stop collection
• We say we care, then accept all cookies

Inference is the proxy problem (L3) applied to people.

Clearview AI: 3 billion scraped photos in 2020, 70+ billion today

Labour markets (Lecture 21)

• Past automation was net positive over time
• AI also exposes cognitive tasks
• Think tasks, not jobs
• Productivity gains are real but task-specific
• Entry-level hiring takes the first hit
• No effect on earnings or hours yet
• Whether AI augments or replaces is still open

Augmentation or replacement is a policy choice, like the rules in L18.

AI is the sixth wave of innovation

Module 6: Applications, limits and projects
(Lectures 22-23, 25)

AI and wellbeing (Lecture 22)

• Feeds optimise engagement, not wellbeing
• Filter bubbles look overstated
• Effects on teen mental health are debated
• Most people who need mental health care never get it
• AI therapy helps short-term, but evidence is thin
• Inference, not training, drives AI’s energy use
• Efficiency gains can raise total use

Recommendations learn from your clicks, like the feedback loops in L17.

Your attention is the product

Misinformation and deepfakes (Lecture 23)

• Mis-, dis- and malinformation differ in truth and intent
• AI makes fakes cheap and fast
• False news spreads faster than true news
• Deepfakes drive fraud, manipulation and sexual abuse
• The liar’s dividend lets anyone dismiss real evidence
• Fast thinking and confirmation bias make us share
• No single fix works, so we layer defences

Deepfakes reuse image generators built on CLIP (L7). The EU AI Act requires labels (L18).

The misinformation spectrum

Safety, alignment, and the future (Lecture 25)

• Safety problems include reward hacking
• Alignment means AI does what we actually want
• Like King Midas, we may get exactly what we asked for
• Goals are hard to specify, and values differ
• RLHF, Constitutional AI and debate are incomplete
• AI should learn our goals, not assume them

Builds on RLHF (L4), Goodhart (L5), bias (L17) and agents (L15).

Summary

What we covered this semester

How AI works

• LLMs predict the next token
• Everything becomes embeddings
• Metrics can mislead (Goodhart’s Law)

Using AI well

• Specific prompts with examples work better
• RAG grounds answers in your documents
• Agent errors compound

Ethics and bias

• Bias enters at every stage
• Fairness is a values question

Policy and society

• The EU regulates AI by risk
• AI infers more than you share
• Think tasks, not jobs

Safety and the future

• Deepfakes create a liar’s dividend
• Alignment: what we actually want

Good luck on Quiz 05 (Lectures 23 and 25) and your projects! 🍀

…and that’s all for today! 🎉