The short answer: a token is a short chunk of text, typically three to four characters of English, and it is the unit a language model actually processes. Model APIs bill per token because token count measures the real computation performed, on the way in and on the way out.

What a token is

Before a model reads anything, a tokeniser splits the text into pieces. Common short words become one token each, longer or rarer words split into several, and punctuation and whitespace count too. A useful rule of thumb: 1,000 tokens is roughly 750 words of English.

The same rule does not hold across languages. Tokenisers are trained mostly on English, so other languages often need more tokens to say the same thing. Swedish compounds, German technical vocabulary, and non-Latin scripts can all run 20 to 50 per cent higher for equivalent content. If your product serves users in several languages, the same feature has a different unit cost per market, which is worth knowing before you price it.

Why output costs more than input

Every provider prices input tokens and output tokens separately, and output is typically three to five times dearer. The reason is mechanical: a model can process your input in parallel, but it generates its answer one token at a time, each step depending on the last. More computation per token, higher price per token.

The practical consequence: response length drives cost harder than prompt length. A feature that produces long answers when short ones would do is overpaying on the expensive side of the metre. Asking the model for concise output is a genuine cost control, and often a product improvement too.

Where the tokens in a real request come from

A production request is rarely just the user’s message. The input usually stacks up from several sources: the system prompt, the conversation so far, any retrieved documents, and finally what the user typed. In a chat interface the history is resent with every turn, so the input grows as the conversation does, and a long session costs several times what its first exchange did.

Retrieval is the other multiplier. A search step that places three documents into the context can add thousands of input tokens per request, which is how one query ends up costing fifty times another against the same model.

Cached tokens, the discount worth designing for

Most major APIs now discount tokens they have processed recently, which in practice means repeated prefixes: the system prompt, shared reference documents, standard instructions. The discount is substantial, often the majority of the input price.

Qualifying for it is a design choice. Keep the stable parts of your prompt at the front and identical between requests, and put the parts that change (the user’s message, fresh context) at the end. The reshuffle costs nothing and the saving recurs on every request.

Estimating tokens for a feature

You can price a feature before building it in half an hour:

1. Write three or four realistic example requests, including the system prompt and any documents you would retrieve.

2. Run them through your provider’s token counter (every major provider publishes one, and most return counts in every API response).

3. Note the input and output counts, apply the published per-token prices, and multiply by the requests you expect per month.

A worked example: 400 tokens of input and 150 of output per request, at €3 per million input tokens and €12 per million output, is about €0.003 per request. At 100,000 requests a month, roughly €300. The arithmetic is small; having done it before launch is the difference between choosing your pricing and discovering it.

Why the same feature costs differently on different models

Three things vary at once between models: the tokeniser (the same text becomes a different number of tokens), the per-token price, and the verbosity of the answers. Price sheets only show the middle one. The reliable comparison is to run your own example prompts through each candidate and compare cost per request end to end, which the half-hour estimate above already sets you up to do.