https://ollama.com/

Everything so far has been typed at a prompt. Today it becomes something your code calls, which is the difference between answering one question and ten thousand

Source: Cloud Now

Today you use an API. In lecture 18 we open it up properly: how a URL is built, what a status code means, and the requests library, so you can collect data for your final project

• curl is short for client for URL. It fetches an address and prints whatever the server sends back. More at https://curl.se/
• A browser would render that response into a page. curl shows you the raw text, which is what your Python will receive
• -s hides the download progress meter, and | is the pipe from lecture 04, feeding what curl printed into the next command
• python3 -m runs a module that ships with Python instead of a file of your own, and json.tool is the module that indents JSON

If this prints Connection refused, the server is not running. Open the Ollama application, or run ollama serve in another terminal

These are the numbers ollama show printed for you in lecture 12, now arriving in a form your code can read.

• size is in bytes. 1,321,098,329 is the 1.3 GB you downloaded
• digest is a sha256 fingerprint of the exact weights. Two people whose digests match are running the identical model, which is how you pin one in a paper
• quantization_level is Q8_0, the compression from lecture 12
• context_length is 131,072 tokens, the most the model can hold at once
• embedding_length is 2,048, the size of the vectors you drew as arrows
• capabilities lists what the model can do. Keep tools in mind for part 4

That last address is the one your Python will use

Run it, and this appears in your terminal:

The sky appears blue to us because of a
phenomenon called Rayleigh scattering,
named after the British physicist Lord
Rayleigh. He discovered that when sunlight
enters Earth's atmosphere, it encounters
tiny molecules of gases such as nitrogen
and oxygen. [...]

CompletionUsage(completion_tokens=271,
                prompt_tokens=31,
                total_tokens=302)

• The package is called openai, but nothing here touches OpenAI. It is a client for a protocol that many servers now speak
• base_url is the address, and /v1 is the OpenAI-shaped door from the last slide
• api_key is required by the library and ignored by Ollama. Write "ollama" and move on
• The file is demo/ask_local.py in the repository

• The text you want is four levels down, which is why every example writes response.choices[0].message.content
• choices is a list because you can ask for several answers to the same question by passing n=3. You almost always want [0]
• role is assistant, the third of the three roles from lecture 12
• finish_reason tells you why the model stopped, and there are two you will meet

stop means it finished. length means it ran out of room and the text is cut off mid-sentence. Add max_tokens=40 to the call above and you get:

"content": "...it encounters tiny molecules of gases"
"finish_reason": "length"

Nothing raises an error. A truncated answer looks exactly like a complete one until you check

The conversation is resent every time

The API has no memory. To continue a conversation, you send the whole messages list again, including everything already said

So turn 20 pays for turn 1 for the twentieth time. A long chat costs much more per reply than a short one, and this is why agents get expensive

• Prices are quoted per million tokens, listed separately for input and output, and you look them up on the model’s page

Print usage on ten rows before you run ten thousand. It is the cheapest mistake you will ever avoid

https://openrouter.ai/

• load_dotenv() reads .env and puts what it finds into the environment
• os.environ is the same idea as echo $SHELL from lecture 03, seen from Python
• The script can be committed, shared, and published. The key stays on your machine
• If a collaborator needs one, they make their own
• Ship a .env.example with the names and no values, so people know what to fill in

pip install python-dotenv for that second import

Design for the quota, do not fight it

• Run the loop on three rows first, and print what you are about to send
• Write each result to disk as it arrives. A crash at row 40 should not cost you rows 1 to 39
• Never send the same text twice. Keep what you already have and skip it
• Add time.sleep(3) between calls if you are near the per-minute limit
• Develop against your local model, where the only limit is patience

Today’s exercise uses 15 rows, so you can run it locally and again on a hosted model and still be inside the daily 50

• Fifteen short financial headlines, each already labelled bullish, bearish or neutral
• I wrote those labels by hand. That column is the answer key, and it is the only reason we will be able to say anything about accuracy at the end
• Some are easy. Rows 3, 7, 10 and 13 are deliberately neutral: a rate held, an appointment, a flat close, a report due on Thursday
• Nothing has happened in those four. Watch what the model does with them
• The task is small enough to check by eye and dull enough that you would not want to

• The answer is in there. Getting it out of there means writing a parser for English
• You would reach for a regular expression, and it would work until the model writes “I would lean bearish”, or hedges, or answers in a different order
• The word neutral also appears in the question, so even a careful search finds the wrong one
• Two runs on two headlines can come back in two different shapes, so there is nothing stable to match against
• Notice that it is wrong as well as verbose. A regulatory inquiry is bad news for a bank

Free text is fine for a person to read. It is a poor interface for a program, and the model is happy to give you either

• The system prompt says which keys you want and which values are allowed. It is the same one you baked into a Modelfile in lecture 12
• response_format is the enforcement. The server restricts what the model is allowed to produce next, so the output always parses as JSON
• The prompt alone is a request. response_format is a guarantee, and you want both
• One line of Python then turns it into data you can use:

json.loads(response.choices[0].message.content)
# {'sentiment': 'bearish', 'confidence': 0.2}

Valid JSON is not a correct answer. That number is text the model generated, not a probability it computed. Do not filter on it and do not report it

• One row of the file becomes one request, and 15 rows become 15 calls to the server
• The list comprehension is an ordinary loop written on one line, with a network call inside it
• json.loads turns each reply into a dictionary, so ["sentiment"] reaches the label
• The result lands as a column beside the data, which is where it belongs and where pandas can work on it
• Fifteen headlines took 5.8 seconds on my laptop, with nothing leaving it

This is the whole idea of the lecture. A language model has become a function you can map over a column

demo/classify.py and demo/headlines.csv are in the repository

• Every bullish and every bearish headline was right. Half the neutral ones were wrong
• Asked to pick a direction, the model picks one, even when the honest answer is that nothing has happened yet
• The errors are not random. They have a shape, they sit in one class, and a single accuracy number hides that
• 87% sounds fine. “Cannot recognise the absence of news” is a different sentence, and it is the true one

This is the step that turns a demonstration into a measurement. Without labels you have output. With them you have an error rate and somewhere to look

• shasum boils a whole file down to 40 hexadecimal characters. Identical fingerprints mean byte for byte identical files
• Change one comma and the fingerprint changes completely
• digest in the Ollama JSON was the same kind of fingerprint, taken of the model file instead
• That is how you say which model produced a result and let someone else prove they have it

Three habits worth keeping

• Set temperature=0 for anything you will report
• Pin the exact model id, never a moving alias like :latest
• Save the raw responses, the model id and the date, next to your results

• Nothing here is new. It is chat.completions.create from part 1, inside a while loop
• The model never runs anything. It returns a function name and some arguments, and your code decides whether to call it
• TOOLS_I_ALLOW is the entire security boundary. If deleting files is not in that dictionary, the agent cannot delete a file
• json.loads on the arguments is structured output again, from part 3, now doing real work
• Remember "capabilities": ["completion", "tools"] in your own model’s JSON? That is what it was telling you
• messages grows on every turn, and the whole list is resent each time. This is why a long agent session costs so much more than a short one

Commit before you let one loose

If your work is committed, git diff shows you everything that changed and git checkout undoes all of it. If it is not, you are relying on the agent’s judgement about your own files

The general shape of the problem:

Your agent cannot tell the difference between

• instructions from you, and
• text it happened to read

Both arrive in the same context window as the same tokens

Everything the agent reads is data, but a language model has no reliable way to stop data from behaving like a command

This is not a bug that will be patched. It follows from how the models work

Matt Shumer, July 2026. A cleanup command expanded $HOME wrongly and ran rm -rf on his home directory

The full policy is in the syllabus. If a situation is not covered there, ask me before you submit, not after

Why the emphasis on explaining:

• The classifier scored 13 out of 15, which sounds fine
• Which two it missed, and that both were the same mistake, only showed up by reading the output against the labels
• That reader is the job, and a better model does not remove it
• Building that reader in you is what this course is for

Worth noticing: api_key="ollama" is not a secret and not a password. The library refuses to start without one, so Ollama accepts any string and throws it away.

The first time you see a required argument that does nothing, it looks like a bug. It is a compatibility shim, and it is the reason one client library can talk to both backends

Back to the exercise

My local run scored 13 of 15, missing two neutral headlines.

A larger hosted model usually gets those two, because “annual report to be published on Thursday” needs a little more world knowledge to read as routine.

The question the exercise is really asking: is that gap worth a key, a quota, and sending your data to someone else?

Sometimes yes. For 15 headlines on your own laptop, almost never

If you got a 401, your key is wrong or not being read. If you got a 429, you have hit the rate limit and should wait a minute

Back to the exercise

Useful beyond the joke: a Modelfile pins a system prompt, a temperature, and a base model into something reproducible that you can share with a collaborator