Tokens and the Context Window in Plain English

What tokens and the context window are - in plain terms

It seems like the model reads text the way you do: word by word, in order. Nope. First it chops your text into tiny chunks - and those chunks are called tokens A token is a chunk of text that the model operates on. It's not always a whole word: often it's part of a word, sometimes a single character or punctuation mark. The model sees text as a stream of tokens, not as letters. .

Think of it like eating. You don’t swallow a whole piece - you take small bites. The model does the same. A long word turns into two or three bites, a short one into a single bite. Those chunks are what it chews on instead of letters.

Now the second part. There’s a context window The context window is the maximum number of tokens a model holds at once: your request, the conversation history, and its reply. Anything that doesn't fit, it can't see. - its working memory. Picture it as a desk holding everything the model can see right now: your question, the entire past conversation, and room for its answer. That desk has an edge. And nothing fits past that edge.

Why a beginner should care about tokens at all

This isn’t theory for theory’s sake. Tokens and the window drive three things you’ll run into on your very first working day.

• Why the model gets dumb in a long chat. The conversation grew, the beginning slid off the edge of the window - and it really did forget where you two started.
• Why the bill goes up. You’re not paying per message, you’re paying per token. Pour huge walls of text into the model and you pay more. Simple as that.
• Why other languages cost more than English. The exact same meaning often takes more tokens in other languages than in English. So it’s literally more expensive for the same text.

Nine out of ten complaints along the lines of “the model isn’t listening to me, it repeats itself, it forgot what I asked” in a long chat aren’t a bug or a personality flaw. It’s an overflowing window: the old stuff has already fallen off the edge.

How tokens and the window actually work

A token is a chunk of text, not a word

The model can’t read letters directly. It comes with a ready-made dictionary of chunks, and it breaks any text down into them. A common short word often goes as a single token. A long or rare word gets split into several pieces.

Here are some rough guides for eyeballing it. These are approximate, not an exact formula:

• in English, one token is roughly 4 characters or about 3/4 of a word;
• punctuation and spaces count too;
• in most other languages, the same meaning takes noticeably more tokens: models are trained mostly on English, and they slice everything else into smaller pieces.

The context window is a desk with an edge

The window is how many tokens fit on the model’s desk in a single pass. What’s sitting on the desk at the same time:

• your current question;
• the entire conversation history - or however much of it fits;
• the system instruction, if there is one;
• and room for the model’s upcoming reply.

The desk is full, you add something new - and the oldest stuff slides off the edge. The model doesn’t decide to forget. It physically just can’t see it anymore. That’s why in very long chats the beginning dissolves on its own.

What tokens cost: input and output

The main thing about money: you pay by the token, and they’re counted in two baskets.

• Input - everything you sent the model: your request plus the history it had to reread.
• Output - everything the model wrote back.

And here’s the surprise for beginners: output tokens are usually more expensive than input tokens - sometimes by several times. That’s why a long answer hits your wallet harder than a long question.

A real-life example: why the chat gets dumber by evening

Say you spend the whole day with the model in one huge chat: you dumped a spec into it, a chunk of code, some discussion, edits, more code. By evening you get two annoyances. First, the model seems to get dumber and forgets what you two decided in the morning. Second, every answer gets pricier, even though the questions feel the same. The reason is the same: the window is packed to the brim and gets reread in full at every step.

So what do people do about it in practice?

Don’t drag your entire life story into one chat. Done with a topic - open a fresh, clean chat: the window is empty and cheap again.
Don’t paste giant walls of text “just in case.” Give the model exactly the piece it actually needs for the task.
The conversation got bloated - ask the model to briefly summarize what you agreed on, copy the summary, open a new chat, and paste only that summary.
Remember: a long answer costs more than a long question. Ask for “shorter” and you’re saving on the expensive output tokens specifically.
If you write in a language other than English and saving matters - the same request in English usually takes fewer tokens.

Common beginner mistakes with context

• Thinking the model remembers everything. It only remembers what fit in the window right now. The old stuff from a long chat has already dissolved for it.
• Assuming one token equals one word. Nope. It’s a chunk, often part of a word. And in most languages other than English it takes noticeably more of them.
• Pasting in everything “to be safe.” Extra text means extra tokens, which means extra money and extra drag. Zero benefit.
• Forgetting that output costs more than input. A huge answer hits the bill harder than a huge question. Ask for what you actually need.
• Living in one chat for months. The window overflows, the model gets dumber, and you overpay for rereading the history. Start a fresh chat for a new topic.