AI tokens: what they are and why they cost money
A player asked me why an AI bill counts “tokens” when nobody had purchased a treasure chest. Fair question. In this layer, a token is not currency. It is a small piece of text—and every prompt you send and answer the model writes is assembled from them.
What is an AI token?
An AI token is a chunk of text that a language model processes. It may be a whole short word, part of a longer word, punctuation, or even a space attached to nearby letters. The model turns your text into token IDs, processes those IDs, and predicts the tokens that become its reply.
That is why a token is not the same thing as a word. “Cat” might fit into one token while an unusual name splits into several. Capitalisation, punctuation, code, and language can all change the count. OpenAI’s token guide gives a rough English rule of thumb: one token is about four characters, or roughly three quarters of a word. It is an estimate, not an exchange rate.
Tokens are the meter, not the meaning: you pay for how much text the model must read and write, not how important the request feels.
Why AI APIs charge by token
A token is a useful unit because the work changes with the amount of material involved. A two-line question does not ask a model to process the same volume as a 200-page contract. Counting requests alone would price those two jobs as if they were equal.
Token pricing makes the meter follow usage more closely. The provider counts the tokens the model receives and the tokens it generates, then applies the selected model’s rates. Providers usually publish rates per million tokens, but your actual charge is proportional to the tokens used; you do not need to consume a million at once.
There are several counters hiding under the tidy word “tokens”:
- Input tokens are what the model reads: your prompt, relevant chat history, system instructions, and any text supplied from files or tools.
- Output tokens are what the model produces, including the visible answer.
- Cached input tokens are repeated prompt material a provider can reuse at a discounted rate when its caching rules are met.
- Reasoning tokens may be used internally by some models before the visible answer. Whether and how they appear in billing depends on the provider and model.
Input and output often have different prices. Current rates also vary by model, batch mode, cache use, and context length, so treat any price copied into a blog post or spreadsheet as temporary. Check the provider’s live pricing page before making a budget.
The simple token-cost formula
For a basic request, estimate each part separately:
cost = input tokens × input rate + output tokens × output rate
Use rates per token in that equation. If a provider quotes per million, divide each published rate by 1,000,000 first—or, more conveniently, divide your token count by 1,000,000 and multiply by the quoted rate. Add cached or reasoning categories only if the model reports them and the pricing page treats them separately.
For example, imagine a request uses 10,000 input tokens and produces 2,000 output tokens. At invented rates of 1 credit per million input tokens and 4 credits per million output tokens, the calculation is 0.01 × 1 plus 0.002 × 4: 0.018 credits. Those rates are deliberately fictional; the method is the useful part.
Why the same document can have different counts
Tokenisation depends on the model’s vocabulary and encoding. Two models can split the same sentence differently. English estimates also travel badly: text in another language, dense code, tables, IDs, or unusual formatting may use a different number of tokens per visible character.
There can also be material you do not see in the chat box. An application may add instructions, retrieved passages, tool results, or conversation history before calling the model. At Wistkey, this is why we inspect the actual request rather than estimating from the player’s last message. The small prompt on screen may be travelling with a large inventory.
Use the tokenizer or token-counting method recommended for the exact model when precision matters. Anthropic, for example, provides a token-counting endpoint; OpenAI provides a tokenizer and reports usage in API response metadata. A word-count shortcut is fine for a sketch, not for a production budget.
Tokens also define how much the model can hold
The model’s context window is measured in tokens too. Input and output must fit inside that allowance, which is why attaching more material can leave less room for the answer. The detailed version is in my field note on what a context window is.
A large advertised context window is a capacity limit, not a recommendation to fill it. More context costs more to send, can take longer to process, and may bury the useful passage among irrelevant ones. The winning move is not “include everything.” It is “include the smallest complete set of evidence.”
How to reduce token costs without making the AI worse
Start by measuring. Most APIs return usage counts with each response. Log input, cached input, output, model, and task type. Without that, cost cutting becomes ceremonial trimming of adjectives while a 40-page policy is attached to every turn.
- Send only relevant context. Retrieve the passages needed for this question instead of pasting the entire knowledge base. This is the practical point of retrieval-augmented generation.
- Stop resending stale history. Summarise an old conversation into decisions, constraints, and open tasks, then continue from that smaller state.
- Cache stable prefixes. If the provider supports prompt caching, keep reusable instructions and reference material consistent so they can qualify.
- Set an output limit. Ask for the format and depth you need, and cap the maximum output where the API allows it.
- Route easy work to a smaller model. Classification, extraction, and simple rewriting may not need the most expensive model in the catalogue.
- Test the whole task. The cheapest request is not the cheapest workflow if weak output causes three retries and a human repair quest.
What to remember
Tokens are fragments of text, not words and not coins. They measure the material a language model reads and generates. They determine both the request’s size and how much of the model’s context window you consume, while the provider’s model-specific rates turn those counts into cost.
My quest log now records tokens in two columns: “useful evidence” and “inventory carried for no reason.” The second column is usually larger. Fortunately, unlike most encumbrance systems, this one lets you delete the spare armour before entering the dungeon.
