DATASCI 350 - Data Science Computing

Lecture 18 - Web APIs and JSON

Danilo Freire

danilo.freire@emory.edu

Department of Data and Decision Sciences
Emory University

Hey, there! 😊
I hope you’re all doing well!

Brief recap 📚

Lecture 17: a machine in the cloud

What we did on AWS

Launched an EC2 t3.micro running Ubuntu 26.04
Connected with a key pair and ssh -i
Installed software with apt
Moved files with scp and wget
Ran Jupyter on the instance through a forwarded port
Met AWS CloudShell, the browser terminal
Stopping pauses compute, but the disk still charges. Terminating deletes the machine
A t3.micro left running costs about $12 a month

Source: K21

Check your AWS account!

Confirm that the bill reads zero

Check that no instance is still running
Terminate your EC2 instances and delete your S3 buckets
Open the Billing and Cost Management Dashboard. The total should be zero
Charged by mistake? Ask for a refund:
- Open the AWS Support Center and click Create case
- Choose Account and billing support and explain what happened
- Replies are usually quick. Email me if you get stuck

Billing and Cost Management Home

Final project: what you will build

A data pipeline, from API to report

Pull data from a web API with requests
Process it with pandas or DuckDB
Save a snapshot in data/raw/. The report reads only that snapshot
Quarto report: 1,500 to 2,500 words, two or more visualisations, visible code
Ship it in a Docker container, with the code on GitHub
Track A (default): the World Bank Indicators API, no key needed
Track B: a free, well-documented public API without OAuth, cleared with me first
Groups of three to four. Email me the names by Thursday 5 November, or I assign you at random

The starter repository: https://github.com/danilofreire/datasci350-project-starter

Due Tuesday 8 December. Send the repository link on Canvas
Full instructions: project-instructions.pdf

Final project: how it is graded

Where the 20% goes, and what I do not grade

The project is worth 20% of your course grade

Component	Weight
Reproducibility	30%
Analysis quality	30%
Communication	20%
Code quality	10%
Git workflow	10%

Sophistication is not graded. Careful descriptive work beats a regression you cannot explain

Every member commits to the shared repository, and I read the commit history
No meaningful contribution means a zero for the project
Clearly unequal shares change individual grades
The report ends with a contribution statement, which I check against the repository

If docker run on my machine does not reproduce your report, your project does not exist 😅

Today’s lecture 📚

You have already used an API

Lecture 14, when you talked to a language model

Remember Lecture 14?
You saved an API key in a .env file
Python sent your prompt to a model on OpenRouter
The model’s reply came back
That was an API call
The openai library hid the details
Today we see what happens underneath
The same idea works for weather, economic data, census records, and more!

# Lecture 14
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)

response = client.chat.completions.create(
    model="google/gemma-4-31b-it:free",
    messages=[{"role": "user", "content": "Hi!"}],
)
print(response.choices[0].message.content)

base_url is a web address. By the end of today you will know what the library sent there, and what came back

Lecture overview

What we will cover today

1. What an API is

A contract between two programs
The client and the server
Why data scientists care

2. URLs and status codes

The parts of a URL, and building one step by step
The codes a server sends back

3. JSON

The format almost every API answers in
How to find one value inside a nested reply

4. APIs from the terminal

curl, a browser for your terminal
Saving a reply to a file

5. A first taste of Python

Four lines that do what curl did. Lecture 19 takes them apart

What is an API?

An API is a contract

Ask in this exact way, get an answer in that exact shape

API stands for Application Programming Interface
It is a contract between two programs
For example, the World Bank API turns a URL ending in /country/BRA/indicator/SP.POP.TOTL into Brazil’s population, year by year
Neither side needs to know how the other works inside
Their database, language, and servers are none of your business 😂

The restaurant

A tired metaphor, but it works

An API works like a restaurant 😅
The menu tells you what you may ask for
You order in terms the kitchen recognises
The dish arrives in a predictable form
You never enter the kitchen
Order something off the menu and you get an error, never a surprise dish

Source: Manutan

Restaurant	API
Menu	Documentation
Order	HTTP request
Kitchen	Server
Dish	JSON response
“We’re out of that”	`404 Not Found`
“One order per customer”	Rate limit

Client and server

You have played both roles this semester
The client asks: your laptop, script, or browser
The server answers: a machine in a data centre, waiting for requests
On Tuesday you launched a server on EC2
The World Bank API is the same idea, with a bigger budget and (hopefully!) better uptime 😂
There is nothing magical about it. It is someone else’s computer 🤓

sequenceDiagram
    participant C as Your laptop
    participant S as api.worldbank.org
    C->>S: GET /v2/country/BRA/...
    Note right of S: look up data
    S-->>C: 200 OK + JSON
    C->>C: parse and analyse

Web APIs specifically

What makes an API a web API

“API” is a broad word. pandas has an API, and so does your operating system
This module is about web APIs:
- The request is an HTTP request to a URL, like the ones your browser sends
- The response is usually JSON, a text format we meet shortly
So any language that can fetch a URL and read text can use one: Python, R, JavaScript, or curl
Terms you will see in documentation:
- REST (Representational State Transfer): each URL names a resource (/country/BRA). Most public data APIs follow it
- Endpoint: one URL pattern the API offers, such as /v2/country/{code}/indicator/{code}
- Rate limit: how many requests you may send per unit of time

Why this matters for data science

Public data, and the script that goes and gets it

The data are already out there

Many organisations share data through APIs:
- The World Bank, the IMF, the US Census Bureau, the ECB, Eurostat, NOAA
- GitHub, Wikipedia, and Emory’s course catalogue (Atlas)
- Most national statistics offices
The World Bank API alone has 29,544 indicators for 217 countries
Most of these data are free
A script can collect exactly the data you need

Scripts are always better than clicking

A script records every step of how you got the data
Run it again next year to get the new numbers
A colleague who runs it gets the same dataset
A reviewer can check every step on GitHub
Lecture 10 asked for “the raw inputs, or a script that fetches them”

Three kinds of API you will meet

Keyless, key-required, and the ones that send a bill

Kind	What it costs you	Examples
Open and keyless	Nothing. Just fetch the URL	Open-Meteo, World Bank, National Weather Service, USGS earthquakes, GitHub
Free but key-required	A signup form, then a key in every request	NASA, OpenWeatherMap, OpenRouter’s free models, GitHub for higher limits
Paid	A key and a bill	OpenRouter’s paid models, most commercial data vendors

An API key is a password for programs. It tells the server who is asking
GitHub answers keyless requests, up to 60 per hour per IP address
Today we stay in the first row, so nobody needs an account

api.github.com/users/danilofreire, fetched with no key at all (click to enlarge)

URLs and status codes 🔗

Anatomy of a URL

Four parts you will see in every request

Every API call starts with a URL, and every URL has the same parts
Here is a real one: Brazil’s GDP per person, 2014 to 2025

https://api.worldbank.org/v2/country/BRA/indicator/NY.GDP.PCAP.KD?format=json&date=2014:2025

Part	Name	What it does
https	scheme	Which protocol. Almost always `https`
api.worldbank.org	host	Which machine to ask
/v2/country/BRA/indicator/…	path	Which data on that machine
?	separator	Ends the path. Everything after it is the query string
format=json&date=2014:2025	query string	Options, as `key=value` pairs joined by `&`

Building a URL

Step	Add this	What it means
1	`https://api.worldbank.org/v2`	The API’s base URL, version 2
2	`/country/BRA`	Brazil, by its three-letter code
3	`/indicator/NY.GDP.PCAP.KD`	GDP per person, in constant US dollars
4	`?format=json`	Start the options. Send JSON, not XML
5	`&date=2014:2025`	Only these years

The path (steps 1 to 3) says what you want
The query string (steps 4 and 5) says how you want it

Swap BRA for URY and you get Uruguay
Swap the indicator for SP.POP.TOTL and you get population
Indicator codes are listed at data.worldbank.org/indicator
In NY.GDP.PCAP.KD:
- NY = national accounts (income)
- GDP = gross domestic product
- PCAP = per capita
- KD = constant US dollars

Reading the query string

Everything after the question mark

The query string starts at the ?
Each option is key=value, and & joins them

?format=json&date=2014:2025&per_page=100
 ^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^^
 option 1    option 2       option 3

Order does not matter: ?a=1&b=2 and ?b=2&a=1 are the same request
Leave out an option and the API uses its default
Misspell a key and the API usually ignores it without telling you

Some characters already have a job in a URL: ?, &, =, #, and the space
To put one inside a value, you percent-encode it
A space becomes %20, and a / becomes %2F
So America/New_York travels as America%2FNew_York
Python’s requests library does this for you, so you rarely type it
The full table is in Appendix 04

Status codes

How to tell if it worked

Every response carries a three-digit number saying how it went
The first digit sums it up: 2xx worked, 4xx is your mistake, 5xx is theirs

Code	Name	What it really means
`200`	OK	It worked. Go ahead and read the data
`301` / `302`	Moved	The data lives elsewhere now. Browsers and `requests` follow these for you
`400`	Bad Request	Your URL is malformed. Check the query string
`401`	Unauthorized	You need a key, or yours is wrong
`403`	Forbidden	Your key works, but it does not give you access to this
`404`	Not Found	Nothing at that path. Usually a typo
`429`	Too Many Requests	You are asking too fast. Slow down
`500`	Internal Server Error	Their problem. Wait and retry

Some APIs, like the World Bank, answer errors with 200 and hide the failure in the reply. A status check alone is not enough. We will see this in the terminal soon

GET and POST

Two main verbs

`GET`: “please send me this”

Asks for data and changes nothing
The whole request fits in the URL
Your browser sends one for every page you visit
Safe to repeat: fetching twice changes nothing on the server
About 95% of what you will do in this course

`POST`: “here is some data”

Sends data to the server
The data travels in the request body, not the URL
Used for logins, uploads, and long inputs
In Lecture 14, the openai library sent your prompt as a POST
A paragraph of text does not belong in a URL

Other methods exist (PUT, DELETE, PATCH), but you rarely need them to read public data

Try it in the browser first

The address bar is an HTTP client you already know

Before writing any code, paste the URL into your browser (try this one)
The browser sends a GET and shows the raw reply
JSON on screen: the URL is right. If your code fails later, the problem is in the code
An error on screen: the URL is wrong, and no code will fix it
This habit will save you hours of debugging
Firefox shows JSON as a tree you can open and close, which helps with long replies

Open-Meteo answering a plain GET from the address bar, no code involved

Try it yourself! 🤓

Five minutes, in the browser. Do everything manually!

Open the Open-Meteo documentation. The API is free
Start from the base URL https://api.open-meteo.com/v1/forecast
Find the parameters for the current temperature. Two are required, and a third asks for current conditions
Use Atlanta’s coordinates: latitude 33.75, longitude -84.39
Write the query string yourself, without copying the full URL from the docs
Paste your URL into the browser address bar and press Enter
Change the coordinates to another city and reload the page

What to look for

A page of JSON, with a temperature somewhere inside it
The units the API reports back alongside the number
A different temperature once the coordinates change
Now delete the longitude option. What does the page say?

Stuck, or want to compare your URL with mine?

Appendix 01

JSON 📦

What JSON is

Plain text that every language can read

JSON means JavaScript Object Notation. The name records its origin, not where it is used today
It is plain text, so any editor opens it
Almost every web API uses it, because every language can read it
It holds data only: no code, functions, or classes
Two building blocks, nested as deep as needed:
- An object { } holds "key": value pairs
- An array [ ] holds values in order

{
  "city": "Atlanta",
  "temperature": 29.6,
  "raining": false,
  "sensors": ["a1", "b2"],
  "last_checked": null
}

Each JSON type has a Python twin

JSON	Python
`{...}` object	`dict` (dictionary)
`[...]` array	`list`
`"text"`	`str` (string)
`42`, `3.14`	`int`, `float`
`true` / `false`	`True` / `False`
`null`	`None`

Keys are always text in double quotes
No comma after the last item
In the example, "sensors" is an array inside an object

A real reply

What Open-Meteo sent back for Atlanta

{
  "latitude": 33.759865,
  "longitude": -84.39586,
  "timezone": "America/New_York",
  "elevation": 316.0,
  "current_units": {
    "time": "iso8601",
    "temperature_2m": "°C",
    "wind_speed_10m": "km/h"
  },
  "current": {
    "time": "2026-08-21T16:00",
    "temperature_2m": 33.2,
    "wind_speed_10m": 4.4
  }
}

Shortened from the saved reply, openmeteo_atlanta.json

The outer { } is an object. It has nine keys, and six are shown here
Two of them, current and current_units, hold objects of their own
That is what “nested” means: a box inside a box
The temperature is inside current. Its unit is inside current_units
time is text, because JSON has no date type
The API moved our coordinates slightly, to the nearest point on its own grid

Finding a value: follow the path

One step per level, from the outside in

To reach a value, write down the steps from the outside in
In an object, a step is a key name
In an array, a step is a position, counting from 0
Temperature in the Open-Meteo reply:

current → temperature_2m → 33.2

Its unit:

current_units → temperature_2m → °C

Arrays need positions. In this reply:

{"sensors": ["a1", "b2"]}

sensors → 0 → “a1”

sensors → 1 → “b2”

Positions start at 0, as in Python
In Python, each step becomes a pair of square brackets: data["current"]["temperature_2m"]
Get the path right first. The Python is then easy

The World Bank’s little surprise

Metadata first, data second, in a two-element array

[
  {"page": 1, "pages": 1, "per_page": 100,
   "total": 12, "lastupdated": "2026-07-13"},
  [
    {"country": {"id": "BR", "value": "Brazil"},
     "date": "2025",
     "value": 9747.99557762692},
    {"country": {"id": "BR", "value": "Brazil"},
     "date": "2024",
     "value": 9566.74518687619},
    ...
  ]
]

Shortened from wb_gdp_bra.json, the reply to our Brazil URL

The outer brackets are an array, not an object
Position 0 is metadata: how many records, and when the data was last updated
Position 1 is the data: one object per year, newest first
The 2025 value: 1 → 0 → value
The country name: 1 → 0 → country → value
Look at the shape before you write any code

Try it yourself! 🤓

Five minutes, on paper. Write the paths

A reply from a fictional course API:

{
  "department": "Data and Decision Sciences",
  "term": "Fall 2026",
  "courses": [
    {"code": "DATASCI 350",
     "title": "Data Science Computing",
     "enrolled": 40,
     "instructor": {"name": "Danilo Freire",
                    "office": "PAIS 480"}},
    {"code": "DATASCI 101",
     "title": "Introduction to AI Applications",
     "enrolled": 65,
     "instructor": {"name": "Danilo Freire",
                    "office": "PAIS 480"}}
  ],
  "updated": null
}

Write the path to each value, like term → “Fall 2026”:

The title of the second course
The office of the first course’s instructor
The number enrolled in each course. What is the total?
The value of updated. What Python type will it become?
Which keys hold an array, and which hold an object?

Stuck, or want to compare your paths with mine?

Appendix 02

APIs from the terminal 💻

`curl`: a browser for your terminal

You met it in Module 02. Now it talks to APIs

curl sends a GET to a URL and prints the reply
It comes with macOS, Linux, and WSL. Check with curl --version
- Missing in WSL? Run sudo apt install curl
- On macOS, brew install curl gets a newer version, but the built-in one works fine
Put the URL in double quotes. Without them, the shell treats & and ? as its own symbols

curl "https://api.open-meteo.com/v1/forecast?latitude=33.75&longitude=-84.39&current=temperature_2m"

{"latitude":33.759865,"longitude":-84.39586,
"generationtime_ms":0.0399,"utc_offset_seconds":0,
"timezone":"GMT","timezone_abbreviation":"GMT",
"elevation":316.0,"current_units":{"time":"iso8601",
"interval":"seconds","temperature_2m":"°C"},
"current":{"time":"2026-09-27T06:30","interval":900,
"temperature_2m":15.8}}

The reply is correct, but it arrives as one long line
Pipe it (|) into Python’s JSON tool to indent it:

curl -s "https://api.open-meteo.com/v1/forecast?latitude=33.75&longitude=-84.39&current=temperature_2m" | python3 -m json.tool

{
    "latitude": 33.759865,
    ...
    "current": {
        "time": "2026-09-27T06:30",
        "interval": 900,
        "temperature_2m": 15.8
    }
}

-s means “silent”: no progress bar
python3 -m json.tool reads JSON and prints it with indentation
- -m means “module”: it runs a tool that ships with Python, so there is nothing to install

Status codes from the terminal

`-i` shows the headers, and the first line is the status

-i asks curl to print the response headers before the data
Headers are notes about the reply. Appendix 05 has more
| head -1 keeps only the first line, which holds the status

# Everything right
curl -si "https://api.open-meteo.com/v1/forecast?latitude=33.75&longitude=-84.39&current=temperature_2m" | head -1

HTTP/1.1 200 OK

# longitude missing
curl -si "https://api.open-meteo.com/v1/forecast?latitude=33.75&current=temperature_2m" | head -1

HTTP/1.1 400 Bad Request

Now a World Bank request for a country that does not exist, XYZ:

curl -s "https://api.worldbank.org/v2/country/XYZ/indicator/NY.GDP.PCAP.KD?format=json"

[{"message":[{"id":"120","key":"Invalid value",
"value":"The provided parameter value is not valid"}]}]

The status of that reply is 200
The World Bank says “OK” and puts the error inside the JSON
So check the status, and then look at the data too

Saving a reply to a file

Fetch once, then work from the copy

-o means “output”: save the reply to a file instead of printing it

curl -s -o wb_gdp_bra.json "https://api.worldbank.org/v2/country/BRA/indicator/NY.GDP.PCAP.KD?format=json&date=2014:2025"

ls -lh wb_gdp_bra.json
python3 -m json.tool wb_gdp_bra.json | head -12

-rw-r--r--  1 danilo  staff  2.7K Sep 27 02:34 wb_gdp_bra.json
[
    {
        "page": 1,
        "pages": 1,
        "per_page": 50,
        "total": 12,
        "sourceid": "2",
        "lastupdated": "2026-07-13"
    },
    [
        {
            "indicator": {

This is how the files in this lecture’s data/ folder were made
A saved file does not change. The live API does
Your analysis then runs without the network, and gives the same numbers every time
That is exactly what your project does: the pull script saves a snapshot in data/raw/, and the report reads only that file
In Lecture 19 we do the same in Python

Try it yourself! 🤓

Ten minutes, in the terminal

Build a World Bank URL for Uruguay’s population, 2014 to 2025
- Country code URY, indicator SP.POP.TOTL
Test it in your browser first
Save the reply with curl to wb_pop_ury.json
Print it with indentation using python3 -m json.tool
Find the 2020 value by reading the output
Write the path to that value
Bonus: misspell the indicator code. What status and what reply do you get?

What to look for

Twelve records, newest first
A population close to 3.4 million
The same two-element shape as the GDP reply
A series that stops growing near the end

Stuck, or want to compare your commands with mine?

Appendix 03

A first taste of Python 🐍

Four lines of Python

The same request, from a script

requests is the Python library for talking to web APIs
Install it once: pip install requests

import requests

url = "https://api.open-meteo.com/v1/forecast?latitude=33.75&longitude=-84.39&current=temperature_2m"

r = requests.get(url)
print(r.status_code)

data = r.json()
print(data["current"]["temperature_2m"])

200
15.8

requests.get(url) does what curl did: it sends a GET and keeps the reply in r
r.status_code is the status code, here 200
r.json() turns the JSON text into a Python dictionary
data["current"]["temperature_2m"] is the path from earlier, one pair of brackets per step
The reading is live, so your number will differ

Where this goes next

Lecture 19 takes these four lines apart

How Python reads JSON, from a reply or a saved file
Letting requests build the query string for you
Checking for errors before you trust the data
Turning the World Bank reply into a pandas DataFrame
Saving a snapshot for your project
Reading the project’s pull_data.py line by line

Everything you did today still applies:

Test the URL in the browser first
Check the status code
Read the shape of the JSON
Write the path before the code

Conclusion 📚

What we learned today

How to read a web API, and how to get its data

Concepts

An API is a contract: ask this way, get that back
A URL has a host, a path (what you want), and a query string (how you want it)
A status code says how the request went: 2xx, 4xx, or 5xx
JSON is made of objects and arrays, nested inside each other
A value is reached by a path: key names for objects, positions for arrays

Habits worth keeping

Paste the URL into your browser before you write code
Quote URLs in the terminal
Check the status, then look at the data anyway
Read the shape of the reply before you look for a value
Save a copy of the reply, then work from the file
Read the documentation’s example request first, then adapt it

Next class

Lecture 19, and what to do before it

Next class we do all of this in Python
Reading JSON from a reply and from a file
Building requests with params, and checking them for errors
From the World Bank reply to a pandas DataFrame
Your project’s pull script, line by line

Before then

Install the packages you will need on Tuesday
Check that your AWS bill reads zero
Read the project instructions and form a group of three to four
Finish Exercise 03 if you did not have time in class

pip install requests pandas pyarrow

A list of keyless APIs to explore is in Appendix 06

Group names are due by Thursday 5 November, and I assign the rest at random

And that’s all for today! 😊

Appendix 01

Exercise 01 solution

Open-Meteo needs latitude and longitude, and current asks for present conditions

https://api.open-meteo.com/v1/forecast?latitude=33.75
&longitude=-84.39&current=temperature_2m

Adding a timezone makes the time readable:

https://api.open-meteo.com/v1/forecast?latitude=33.75
&longitude=-84.39&current=temperature_2m
&timezone=America%2FNew_York

Without longitude, the page shows:

{"error":true,"reason":"Parameter 'latitude' and
'longitude' must have the same number of elements"}

Both coordinates are required. The request fails without either one
The / in the timezone is written %2F, because / already has a job in a URL
Without timezone, the time comes back in GMT, which reads oddly for Atlanta
The reading is live, so your number will differ from mine

Back to exercise

Appendix 02

Exercise 02 solution

courses → 1 → title → “Introduction to AI Applications”
courses → 0 → instructor → office → “PAIS 480”
courses → 0 → enrolled is 40, and courses → 1 → enrolled is 65. The total is 105
updated → null, which becomes None in Python
courses holds an array. Each course and each instructor is an object

The same paths in Python, which we write in Lecture 19:

data["courses"][1]["title"]
data["courses"][0]["instructor"]["office"]

courses is an array, so the second course is position 1, not 2
The office is four steps down: array, then course, then instructor, then office
null is JSON’s word for “no value”. It is not the text "null"

Back to exercise

Appendix 03

Exercise 03 solution

curl -s -o wb_pop_ury.json "https://api.worldbank.org/v2/country/URY/indicator/SP.POP.TOTL?format=json&date=2014:2025"

python3 -m json.tool wb_pop_ury.json

Part of the output, around 2020:

        {
            ...
            "countryiso3code": "URY",
            "date": "2020",
            "value": 3398968,
            ...
        },

The path: 1 → 5 → value → 3,398,968

Only the country and the indicator change. The query string is the same as for Brazil
Records run newest first, 2025 at position 0, so 2020 is at position 5
Uruguay’s population peaks in 2020 and falls a little every year after
A misspelt indicator still returns status 200, with an error message inside the JSON
The saved file is also in data/wb_pop_ury.json

Back to exercise

Appendix 04: percent-encoding

Characters that need a code inside a value

You write	It travels as	You write	It travels as
space	`%20` or `+`	`=`	`%3D`
`/`	`%2F`	`:`	`%3A`
`?`	`%3F`	`#`	`%23`
`&`	`%26`	`%`	`%25`

The pattern is always the same: % and the character’s byte in hexadecimal (remember them from Lecture 02? 😉)
America/New_York becomes America%2FNew_York
requests encodes values for you, so you rarely write these by hand
Full tables and rules: MDN, W3Schools, and RFC 3986 §2.1

Appendix 05: headers

The envelope, not the letter

Every request and every reply carries headers: notes about the message itself

Header	Purpose
`User-Agent`	Who is asking. Browsers set this, and polite scripts should too
`Accept`	What format you want back (`application/json`)
`Authorization`	Your API key, for APIs that need one
`Content-Type`	The format of the data being sent

curl -sI "https://api.worldbank.org/v2/country/BRA/indicator/NY.GDP.PCAP.KD?format=json&date=2014:2025"

HTTP/2 200
content-type: application/json;charset=utf-8
cache-control: public, max-age=86400
server: cloudflare
...

-I asks for the headers only, without the data
content-type confirms the reply is JSON
max-age=86400 says the reply can be reused for a day (86,400 seconds)

Appendix 06: free APIs worth exploring

Six keyless places to practise this week

All of these answered a keyless request on 21 August 2026

API	What it gives you	Documentation
Open-Meteo	Weather forecasts and history, anywhere	https://open-meteo.com/en/docs
World Bank	29,544 development indicators, all countries	https://datahelpdesk.worldbank.org/knowledgebase/topics/125589
National Weather Service	US forecasts and alerts, from `api.weather.gov/points/{lat},{lon}`	https://www.weather.gov/documentation/services-web-api
GitHub	Repositories, users, commits (keyless with low limits)	https://docs.github.com/en/rest
Open Library	Books, authors, covers, by ISBN (occasionally flaky)	https://openlibrary.org/developers/api
USGS Earthquakes	Every recorded earthquake, live	https://earthquake.usgs.gov/fdsnws/event/1/

Pick one this week and fetch something from it, in the browser or with curl

DATASCI 350 - Data Science Computing

Hey, there! 😊 I hope you’re all doing well!

Brief recap 📚

Lecture 17: a machine in the cloud

What we did on AWS

Check your AWS account!

Confirm that the bill reads zero

Final project: what you will build

A data pipeline, from API to report

Final project: how it is graded

Where the 20% goes, and what I do not grade

Today’s lecture 📚

You have already used an API

Lecture 14, when you talked to a language model

Lecture overview

What we will cover today

What is an API?

An API is a contract

Ask in this exact way, get an answer in that exact shape

The restaurant

A tired metaphor, but it works

Client and server

Web APIs specifically

What makes an API a web API

Why this matters for data science

Public data, and the script that goes and gets it

The data are already out there

Scripts are always better than clicking

Three kinds of API you will meet

Keyless, key-required, and the ones that send a bill

URLs and status codes 🔗

Anatomy of a URL

Four parts you will see in every request

Building a URL

Reading the query string

Everything after the question mark

Status codes

How to tell if it worked

GET and POST

Two main verbs

GET: “please send me this”

POST: “here is some data”

Try it in the browser first

The address bar is an HTTP client you already know

Try it yourself! 🤓

Five minutes, in the browser. Do everything manually!

JSON 📦

What JSON is

Plain text that every language can read

A real reply

What Open-Meteo sent back for Atlanta

Finding a value: follow the path

One step per level, from the outside in

The World Bank’s little surprise

Metadata first, data second, in a two-element array

Try it yourself! 🤓

Five minutes, on paper. Write the paths

APIs from the terminal 💻

curl: a browser for your terminal

You met it in Module 02. Now it talks to APIs

Status codes from the terminal

-i shows the headers, and the first line is the status

Saving a reply to a file

Fetch once, then work from the copy

Try it yourself! 🤓

Ten minutes, in the terminal

A first taste of Python 🐍

Four lines of Python

The same request, from a script

Where this goes next

Lecture 19 takes these four lines apart

Conclusion 📚

What we learned today

How to read a web API, and how to get its data

Concepts

Habits worth keeping

Next class

Lecture 19, and what to do before it

And that’s all for today! 😊

Appendix 01

Hey, there! 😊
I hope you’re all doing well!

`GET`: “please send me this”

`POST`: “here is some data”

`curl`: a browser for your terminal

`-i` shows the headers, and the first line is the status