Under the hood
How does an AI work?
Ask a real AI chatbot a question and watch it write its reply. Then step through how the reply was made, using the real data behind it, with every value labeled.
The options for the next token
What you’ll see
Tokens
Text is split into : often a whole word with its leading space, sometimes a piece of one, like building blocks of text. AI services count tokens, not words: a model can take in , each reply is capped at a set number, and every request is priced by the token.
See it in the sample- "Why"13903
- space, "is"382
- space, "the"290
- space, "sky"17307
- space, "blue"9861
- "?"30
Calculated tokens and IDs, split by this app’s tokenizer
A weighted random pick
At every step the model the options for the next token. One is , like a raffle where likelier options hold more tickets, so the top option doesn’t always win.
See it in the sample- space, "Dust"
- space, "and"
- space, "tiny"
- space, "particles"
- space, "can"
- ␣intens39.4%
- ␣scatter26.2%
- ␣enhance16.2%picked
- ␣deepen5.57%
Chance this piece comes next, not chance it is true.Recorded tokens, reported by OpenAICalculated percentages, from its
Context, not memory
With each message, this app sends the again, with its own , like handing over the whole transcript each time. Chatting doesn’t change the model.
See it in the sample- InstructionsYou are the assistant on Token by Token, an educational website that shows how language models generate text one token at a time. Answer helpfully, accurately, and concisely, usually in under 150 words and in plain language. Use Markdown only when it helps. If you aren't sure about something, say so. You have no tools, no internet access, and no memory of other conversations.80 tokens
- YouWhy is the sky blue?6 tokens
- AssistantSunlight contains many colors, but air molecules scatter shorter wavelengths more than longer ones. Blue light has a shorter wavelength than red light, so it gets scattered in all directions across the sky and reaches your eyes from everywhere. That’s why the daytime sky looks blue.53 tokens
- YouWhy are sunsets red, then?7 tokens
Recorded text, sent by this appCalculated token counts
What’s real here
Every value carries one of four labels, always as a word, never just a color:
- Recorded
- Sent, done, or measured by this app, or reported by OpenAI, for your conversation.
- Calculated
- Computed here from recorded values, with the method named.
- Reference
- Documented facts, such as the model’s context limit, with a source and a date.
- Example
- A teaching drawing, not measured from the model. Views of the network’s insides are always examples: OpenAI hasn’t published the design of its hosted models.
A What-if tag marks anything simulated on the page. The model wasn’t asked again.