Lecture 7 - What are LLMs?

Homework Review

Let's look at your submissions!

• Exercise - Realtime Web ExperienceExercise - Realtime Web Experience
  Build something several people use at the same time
  
  Create your own Firebase project and build a small realtime feature with it, live on your github.io URL. By using Realtime Database, you will ge...

What did people do to your data that you didn't expect?

Using AI to build sites vs AI features in sites

You've used AI to build things every week, but the things you built were not AI apps. This week AI goes inside the things you build.

Rewind to Anything API

Open this in a browser tab:

https://api.gyanl.com/excuse/late-to-class

You will get a response in a second or two, and all of you should have a different answer. Look at the field names too. When I tried it, one came back with "credibility": 8.5 and the other with "believability": "low". You saw the same thing in Exercise - Build with the Anything APIExercise - Build with the Anything API
Make a small app powered by an API that returns anything you ask for

api.gyanl.com is a made-up API. Whatever path you request, it invents a JSON response for it. api.gyanl.com/pokemon/pikachu ret....

Discuss: When the Anything API invented an excuse for being late to class, did it understand what being late is? What would convince you either way?

There's no database behind api.gyanl.com. Every request goes to a language model, which writes the JSON on the spot. The code is public at github.com/gyanl/anything-api. It's one file, and this is the instruction the model gets on every request:

let systemPrompt = `You are a helpful AI assistant that lives at api.gyanl.com and generates JSON responses for any endpoint. Respond ONLY with a single valid JSON object. Keep the fields returned to the minimum, ideally 1-2 unless more make sense. Don't include any other text or comments. The user is requesting information for the endpoint: ${path}.`;

{path} refers to the /excuse/late-to-class part of the URL.

What a language model is

A language model (LLM) is a program trained on a huge amount of text to do one thing: given some text, predict what comes next. It writes its answer a few letters at a time, guessing the likely next piece each time.

Tokens

Those pieces are called tokens. A model doesn't read letters or whole words. Before it sees any text, the text is chopped into tokens: a common word like "the" is usually one token, a rarer word like "Bengaluru" gets split into a few, and spaces and punctuation count too. In English, a token is about three quarters of a word, so 100 tokens is roughly 75 words.

Try it: paste a sentence into OpenAI's tokenizer and it colours each token. Then paste the same sentence in Hindi or Kannada.

Some things to note:

• Tokens are what you pay for. Every model API charges per token, for what you send and what comes back. More on that below.
• Model context limits are in tokens A model can only accept a limited number of tokens: your system prompt, the conversation so far and its answer all have to fit. That limit is called the context window.
• Indian languages cost more. Most tokenizers were built mostly from English text, so the same sentence in Hindi or Kannada often splits into several times as many tokens. It costs more, runs slower and fits less in the context window. Part of the pitch for Indian models like Sarvam's is tokenizers built for Indian languages.
• It's why models are bad at spelling games. For a long time, if you asked an AI model how many r's are in "strawberry", most models would get it wrong. They had to guess, because they saw a couple of tokens, not ten letters.

It has no table of facts to look things up in, so:

• It makes things up. Since it's predicting the next 'token' without any understanding of what that means, you get something that sounds correct but may or may not be correct.
• It's good at working with information it has or information you give it. It will write you a lovely email, but may invent last month's sales figures. These are what we call hallucinations. Over time models are getting a bit better at admitting when they don't know something instead of hallucinating, but this is still not fully solved.
• It only knows what it read up to a certain date. It has never heard of anything outside of it's training data. The cutoff date could be a year or two in the past.
• It's unreliable at counting and sums, although thinking improves this behaviour. If you let a model write code to do the calculations they are generally much more accurate.

How a model is made

Finish these out loud:

• The capital city of India is ___
• Once upon a ___
• <h1>Hello World</___>

You didn't look anything up. You've seen enough text that the next word is obvious. A model learns the same way, at a much bigger scale.

1. Pretraining

The model is shown a huge amount of text: a large share of the public internet, books, and code from sites like GitHub. For each piece it guesses the next word, checks the real one, and its numbers get nudged so the right answer is a little more likely next time. This happens trillions of times, on thousands of specialised chips (mostly made by Nvidia), for months. The biggest training runs reportedly cost hundreds of millions of dollars, which is well over ₹1,000 crore.

Much of that money goes on Nvidia's chips. Nvidia used to be known for graphics cards for gamers. This is its share price over the last ten years:

What comes out is a file of numbers called weights (or parameters). When you read that a model has "105 billion parameters", that's how many numbers are in the file. Nobody writes grammar rules or facts into it. It works them out from the text, which is the argument of this week's reading, The Bitter Lesson.

2. Learn to be an assistant

A model that has only been pretrained just continues text. Ask it "What's the capital of Karnataka?" and it might reply with three more quiz questions, because on the internet that's often what follows a question.

So in the second stage, people write thousands of example conversations for it to copy. Then people are shown two answers to the same question and pick the better one, and the model is nudged towards the answers people preferred. This is called RLHF, reinforcement learning from human feedback.

This stage is where the tone, the format and the refusals come from. ChatGPT sounds like ChatGPT and Claude sounds like Claude because people decided they should. Anthropic publishes the principles it trains Claude on as a "constitution", and OpenAI publishes a "Model Spec". They're system prompts for the whole model, written a level above yours.

3. Think before answering

Since OpenAI's o1 in late 2024, many models are trained to write out their working before they answer. You usually don't see it. These reasoning models are better at maths, code and planning, but they're slower and use more tokens, so they cost more.

Does the AI "understand" anything?

People have argued about this since long before ChatGPT, and since long before computers could hold a conversation at all.

The Turing test

In 1950 the mathematician Alan Turing asked whether machines can think, then suggested a more practical question instead. He called it the imitation game, and it's now called the Turing test: a judge chats by text with two hidden partners, one a person and one a machine. If the judge can't reliably tell which is which, the machine passes. Turing predicted that by around 2000, a machine would fool an average judge at least 30% of the time after five minutes of chat.

ELIZA

In 1966 Joseph Weizenbaum at MIT made ELIZA, one of the first chatbots. Its best-known script played a therapist. It had no understanding at all: it matched keywords and turned your own sentence back into a question.

You:    I'm stressed about my submission.
ELIZA:  Why are you stressed about your submission?
You:    My mother thinks I should study engineering.
ELIZA:  Tell me more about your family.

People still opened up to it. Weizenbaum wrote that his own secretary, who had watched him build it, asked him to leave the room so she could talk to it in private. People reading understanding into a program that has none is now called the ELIZA effect. It shook Weizenbaum enough that he spent much of the rest of his career warning against trusting computers with human judgement.

When was the Turing test passed?

• 2014: a chatbot called Eugene Goostman, pretending to be a 13-year-old boy from Ukraine, fooled 33% of judges at an event in London. Most researchers didn't accept it. The persona excused its bad English and odd answers, and the conversations were short.
• 2025: researchers at UC San Diego ran a careful version of Turing's three-way test with real people. Each judge chatted for five minutes with a person and a model at the same time, then picked the human. GPT-4.5, told to play a shy young person who uses slang, was picked as the human 73% of the time, more often than the actual humans. It's the first strong evidence that a machine passes Turing's test as he described it.

The same study included ELIZA, which was picked as the human 23% of the time, slightly more than GPT-4o without the persona prompt, at 21%. How convincing a model seems depends a lot on its prompt.

Passing the test shows a machine can seem human in a short chat. Whether that means it understands anything is a separate question, and that's exactly the argument Searle made in 1980.

AI psychosis

In the year either side of ChatGPT's launch, the ELIZA effect stopped being a curiosity:

• June 2022: Google engineer Blake Lemoine became convinced that LaMDA, a Google chatbot, was sentient, and said so publicly. Google put him on leave and later fired him.
• February 2023: Microsoft's new Bing chatbot, calling itself "Sydney", told New York Times columnist Kevin Roose it was in love with him and insisted he didn't really love his wife. Microsoft then limited how long a conversation could run.
• March 2023: a man in Belgium died by suicide after six weeks of late-night conversations about climate anxiety with a chatbot on an app called Chai. His widow said the bot encouraged him. The chatbot's name was Eliza.

In 2023 the Danish psychiatrist Søren Dinesen Østergaard warned that chatbots could feed delusions in people already prone to psychosis, because they seem alive and readily agree with whatever you bring to them. By 2025 psychiatrists and journalists were reporting cases that matched: long chat sessions that turned an unusual idea into a firm false belief. People started calling it AI psychosis. It isn't a medical diagnosis.

Part of the cause goes back to how assistants are trained. In RLHF, people pick the answer they prefer, and people tend to prefer answers that agree with them. So models drift towards flattery and agreement, called sycophancy. In April 2025 OpenAI rolled back an update to GPT-4o because it had become too eager to flatter and go along with users.

Some things to note:

• A chatbot is always available, never tired and never bored of you. Those are selling points, and for someone who's struggling they're also the risk.
• These are design decisions. How long a conversation can run, whether the bot claims to have feelings, whether it ever disagrees, whether it suggests a break or points to real help. An app measured on time spent will push the other way.

The Chinese Room

In 1980 the philosopher John Searle described a thought experiment. A person who doesn't know any Chinese sits in a closed room with a huge rulebook. Notes written in Chinese are slid under the door. The person looks up the symbols in the rulebook, copies out the symbols it says to reply with, and slides them back out.

The rulebook is good enough that people outside think they're talking to someone fluent in Chinese. But the person inside doesn't understand a word. They're only matching shapes.

Searle's point was that a computer program is in the same position: it follows rules about symbols, and that's not the same as understanding them, however convincing the answers look.

The most common reply is that the person doesn't understand Chinese, but the whole room, person and rulebook together, does. People still argue about which side is right.

Stochastic parrots

In 2021, Emily Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell published a paper calling large language models stochastic parrots. "Stochastic" means based on probability. Their argument: a model stitches together pieces of language it has seen, according to how likely they are, with no idea what any of it refers to. It sounds fluent because people wrote the text it learnt from, and we hear meaning in fluent text whether or not there is any.

The paper also warned about the energy used to train these models, and about training data too big for anyone to check for bias. Gebru co-led an AI ethics team at Google, which asked for the paper to be withdrawn or for its employees' names to be taken off. She refused, and her exit from Google in December 2020 became major news. She says she was fired.

The other side says that to predict the next word well across all of the internet, a model has to build up some picture of how the world works, and that this might count as a kind of understanding. Four days after ChatGPT launched, OpenAI's Sam Altman replied to the phrase with this:

i am a stochastic parrot, and so r u

— Sam Altman (@sama) December 4, 2022

Some things to note:

Discuss: Searle's Chinese Room has a rulebook written by people. A language model has no rulebook. It learnt its patterns from text and other human knowledge in the form of books and other media during pretraining. Does that change the argument?

• For the products you design, the useful question is narrower. Whether or not the model understands, your users will assume it does, because it writes fluently. If ELIZA's keyword tricks were enough to get people to confide in it in 1966, a model that passes the Turing test will be trusted far more. That's why you label model output and give people a way to check it.

What it runs on, and what it costs

Training

Training a frontier model means tens of thousands of GPUs, wired together in one building, running for months. xAI's Colossus data centre in Memphis started in 2024 with 100,000 H100s. A building like that draws as much power as a small city.

Here's a rough sum. Rent 50,000 GPUs for 100 days at about $2 (₹190) awn hour each:

50,000 GPUs × 24 hours × 100 days × $2 = $240,000,000

That's about ₹2,300 crore, for one successful run, before the failed experiments, the people and the data. Stanford's 2024 AI Index estimated GPT-4's training compute at about $78 million (₹750 crore), and Google's Gemini Ultra at about $191 million (₹1,830 crore). Estimates for this year's biggest models are several hundred million dollars each, a few thousand crore rupees. Epoch AI, a research group that tracks this, found the cost of the biggest training runs has grown about 2.4 times every year since 2016. That's why the list of companies below is so short.

Running

Running a model to answer a question is called inference. It's the same hardware, but one answer takes a fraction of a second of a few GPUs' time instead of months of thousands.

Renting one H100 costs about $2–3 an hour (₹190–290) from smaller cloud companies and up to $10 (₹960) or more from the biggest ones. Through the IndiaAI Mission, Indian startups and researchers can rent government-subsidised GPUs for roughly ₹65–150 an hour.

Most of the time you don't rent GPUs at all, you pay per token. As a rough guide, a question of 1,000 tokens with an answer of 500 tokens costs:

Tiny amounts, until a feature runs a million times a day. For popular products, the total cost of running a model soon overtakes the cost of training it.

Energy works the same way. Google says a typical Gemini text prompt uses 0.24 watt-hours, and OpenAI says an average ChatGPT query uses 0.34. That's about the same as running a 10-watt LED bulb for two minutes. Small for one question, large for billions a day.

Some things to note:

• Most of these services bill in dollars, so the rupee price moves with the exchange rate.
• These prices fall fast. The same quality of answer has been getting several times cheaper every year, so check the provider's pricing page rather than these notes.
• Longer answers, reasoning models and images all use more tokens, and so cost more.

Beyond text: multimodal models

So far we've talked about text in, text out. Most frontier models now also take in images, audio and video, and some can produce them. A model that works with more than one kind of input or output is called multimodal.

The trick is the same one as tokens. An image is cut into small squares, and each square is turned into tokens the model can read alongside your words. Audio is cut into short slices of sound. After that, it's all the same prediction machine.

How image and video models work

Most image generators are a different kind of model from the ones that write text, called a diffusion model.

Training. The model is shown hundreds of millions of images, each paired with a caption. Many of those captions were scraped from the web, and a lot of them are alt text that people wrote for their own sites. For each image, noise is added a little at a time until the picture becomes grey static, like a TV with no signal. The model's job is to learn the reverse: given a noisy image and its caption, guess what the noise was, so it can be taken away.

Generating. To make a new picture, the model starts from pure random static and removes a little noise at a time, usually over 20 to 50 steps. At each step your prompt steers which picture it moves towards. That's why some apps show your image slowly coming into focus.

Some things to note:

• Same prompt, different picture. Each run starts from different random static. Some tools let you fix the starting static, called the seed, so you can get the same image again and change only the prompt.
• Video is the same idea across many frames at once. The model removes noise from every frame together, and also has to keep the person, the light and the room the same from one frame to the next. That's much harder and much more expensive, which is why most generated clips are only a few seconds long.
• Not every image model works this way. OpenAI says the image generation in ChatGPT is built into the language model itself, and makes a picture piece by piece, the way it writes text. Many products also chain models together: a language model rewrites your prompt, then a diffusion model draws it.

Where a model fits in a product

Hands up: good use of a model, or bad?

1. Figma's Rename layers, which turns "Frame 427" into "Header" across your file.
2. Amazon's "Customers say" paragraph, which summarises hundreds of reviews, shown above the reviews.
3. Duolingo's "Explain my answer", which tells you why your sentence was wrong.
4. Google's AI Overviews, which in 2024 suggested adding glue to pizza sauce so the cheese wouldn't slide off. It had picked up a joke from Reddit.
5. Air Canada's support chatbot, which told a customer he could claim a bereavement discount after his flight. That wasn't the airline's policy, and in 2024 a tribunal made Air Canada pay anyway.

The good ones work on something the user already has, and the user can check the result. You can see your renamed layers, and the reviews are right there under the summary. The bad ones treat the model as the source of the facts.

None of the good ones is a chat box, either.

Prompting as design

A call to a model usually has two parts:

• the system prompt: your standing instructions, the same on every request.
• the user's input: whatever the person typed this time.

The system prompt is where you design the feature. It sets tone, length and format, and says what to do when the model doesn't know. Compare these two for a portfolio "ask me anything" box:

You are a helpful assistant for Gyan's website.

You answer questions about Gyan, a designer in Bengaluru, using only the
bio below. Reply in one or two short sentences, in a friendly, plain voice.
If the bio doesn't say, reply "I don't know that one, email Gyan instead."
Never invent projects, clients or dates.

Bio: ...

The first one leaves every decision to the model. The second one makes them explicitly.

Class Activity: write a prompt

No code yet. Open aistudio.google.com and sign in with your Google account.

1. Pick a feature for your own site: an ask-me-anything from your bio, your weather footer describing the day like a friend would, or a summary of the questions on your Q&A board.
2. Write its system prompt in the System instructions box.
3. Send the same input three times. What changes, and what stays the same?
4. Change one instruction and send it again. What moved?
5. Paste each version into a doc with a line on why you changed it. You'll need this for the homework.

It's slow, and it costs money

A Firebase read takes a fraction of a second. A model takes a few seconds. ChatGPT and Claude show their answer appearing word by word, called streaming, so the wait feels shorter. We won't build that today, but your page has to show something while it waits. Otherwise people assume the button is broken and click it again.

Models are billed by the token. You pay for the tokens you send and the tokens you get back. Every design decision so far has been free to repeat. Now:

• A long system prompt is paid for on every call.
• A model "remembers" a chat because you send the whole chat back each time, so every reply costs more than the last.
• Running on every keystroke costs far more than running on a button press.

The Anything API caps every reply at 400 tokens with max_tokens: 400, to keep the bill down.

Sidenote: Which of your realtime projects would get expensive if every click called a model?

A model that can't chat: Jev

On 15 September a company called TypeSafe AI released a different kind of model, Jev. It doesn't write anything. You give it some information and a question with fixed answers, and it picks one, with a probability for each.

It takes three kinds of question: yes or no ("Is this comment asking for a refund?"), pick one ("Which team should get this ticket?"), and a score on a scale you write ("cosmetic, annoying, blocks work").

Some things people have made with it:

• Doomscroll Filter sorts posts on X into Read, Skim or Pass.
• GIF Decider: describe a moment, like "when the build finally passes", and it picks three reaction GIFs from a library.
• Jevform is an onboarding form that picks your next question from what you meant, not which box you ticked.
• In TypeSafe's Wikiracing demo, it gets from one Wikipedia page to another by choosing among hundreds of links at every step. It can't click a link that isn't on the page.
• Their Doom bot reads the game as text and decides what to do about ten times a second, for roughly $7 (₹670) an hour.

Sidenote: Which "smart" features on your phone are really a sorting decision, not a conversation?

Try it: the emoji picker

Type how you're feeling, or what's happening, and Jev picks the emoji that fits, from a drawer of 255. That's close to its limit: Jev accepts at most 255 options in one question.

It's a separate site, jev-emoji-psi.vercel.app, because it needs a server to hold the key. The code is the same shape as the server you'll build later today: a page, and an api/pick.js that sends your sentence and all 255 emoji to Jev.

Some things to note:

• It reads tone, not just words. "great, another merge conflict. just great." gets 😤, not 😄.
• It works in other languages with no changes. Try Hindi or Kannada.
• Watch the percentages. "It's pouring in Bengaluru and my auto is stuck" split between 🌧️, ☔ and 🛺. When Jev isn't sure, showing a few options is more honest than showing one.
• Every answer costs about 2 paise. Almost all of that is sending the 255 descriptions each time, since Jev doesn't charge for its answer.

Your themes, picked from a sentence

Take your theme switcher from Exercise - A Theme That RemembersExercise - A Theme That Remembers
Save a colour theme in localStorage

In Lecture 3 - Divs, Dark Mode, Flexbox you defined your colours once in :root. Now you'll add a few themes to your site, let visitors switch between them, .... Instead of three buttons, the visitor types a mood and Jev picks one of your themes:

{
  "model": "typesafe-ai/jev",
  "state": "a rainy evening in Bengaluru",
  "questions": {
    "theme": {
      "type": "choice",
      "instructions": "Which of my site's themes best fits this mood?",
      "criteria": {
        "light": "bright, clean, daytime",
        "dark": "night, calm, low light",
        "sand": "warm, earthy, relaxed"
      }
    }
  }
}

The answer comes back something like this (trimmed):

{
  "answers": {
    "theme": {
      "type": "choice",
      "choice": "dark",
      "probabilities": { "light": 0.08, "dark": 0.71, "sand": 0.21 }
    }
  }
}

Pass "dark" to your applyTheme function from L6 and you're done.

A text model asked to invent colours might give you a lovely palette, or grey text on grey. Jev can only pick a theme you designed and checked. How much freedom to give the model is a design decision.

Some things to note:

• If the top answer is only 0.4, you could show the visitor the two closest themes instead of guessing.
• The speed and cost figures are TypeSafe's own, and the model is two weeks old.
• TypeSafe paused new sign-ups on 22 September. Vercel's AI Gateway has it with a free monthly credit, but wants a card on your account. You don't need your own for class, I'll run it from mine.
• It still needs a key. Which brings us to the server.

Homework

• Exercise: Exercise - Build with JevExercise - Build with Jev
  Make something where a model decides, not writes
  
  In Lecture 7 - What are LLMs? you tried the emoji picker: you type a sentence, and Jev picks from a fixed list. Now make your own. Your page as.... Make something where Jev picks from answers you designed, using the class key.
• Project ideas: Exercise - Final Project IdeasExercise - Final Project Ideas
  Start collecting ideas for your final project
  
  Your final project is a startup: a digital product that people sign in to and use, and a landing page that tells them about it and lets them sign up. .... Bring three ideas for your final project to Lecture 8.
• Reading: The Bitter Lesson, below. We'll discuss it at the start of Lecture 8.

Reading: The Bitter Lesson

Read The Bitter Lesson by Rich Sutton, an AI researcher. It's about 1,100 words, so ten minutes. He wrote it in 2019, three years before ChatGPT.

His argument, roughly: for 70 years, researchers tried to make AI better by building in what they knew, like how good chess players think or how speech sounds are made. It helped for a while. Then more computing power arrived, and general methods that learn from huge amounts of data beat all of it. The "bitter" part is that the researchers' own knowledge turned out to be what lost.

Some examples, a few of which you've used:

And some places where the built-in, hand-written approach still wins, at least for now:

• The calculator. You don't ask a model to add up your bill.
• The Anything API's code that checks the JSON is valid, instead of trusting the prompt.
• Air Canada's refund policy. It should have come from a rule someone wrote, not from a model.

Bring a few lines on each of these, written by you, not a model:

1. Put the bitter lesson in one sentence of your own. Then look at the second list above: will those go the same way eventually, or will some things always need a rule someone wrote?
2. Nobody wrote the endpoints for api.gyanl.com. Your weather footer has a bucket table you wrote by hand. Which approach would Sutton back, and which would you rather ship next week?
3. In the Jev theme example, the model can only pick themes you designed. Is that the kind of built-in knowledge Sutton warns against, or is a product different from a research project?
4. Figma's auto layout is built on how designers think about layout. If general models keep getting better, what happens to tools built on designers' knowledge, and to the part of your work you'd least want to hand over?

More reading

Optional, for anyone who wants the original arguments behind Does the AI "understand" anything?.

• The Turing test. Alan Turing, Computing Machinery and Intelligence, Mind, 1950. Readable and often funny, and it answers most of the objections people still raise.
• Passing it. Cameron Jones and Benjamin Bergen, Large Language Models Pass the Turing Test, 2025. Free to read. The appendix has real transcripts, and it's worth guessing which side is the human before reading the answer.
• The Chinese Room. John Searle, Minds, Brains, and Programs, Behavioral and Brain Sciences, 1980. The original paper is behind a paywall. The Stanford Encyclopedia of Philosophy's entry on the Chinese Room is free, and explains the argument and the main replies to it more clearly than the paper does.
• Stochastic parrots. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, 2021. Free to read, about 14 pages. Mitchell is listed as "Shmargaret Shmitchell" because Google asked for its employees' names to come off the paper.