What a token is, and why input and output have different prices
A token is a chunk of text the model treats as one unit — usually a common word, a word fragment, or a piece of punctuation. English averages roughly four characters per token, which works out at about 1.33 tokens per word. Code, non-Latin scripts and unusual proper nouns tokenise less efficiently, so a page of JSON costs more tokens than a page of prose of the same length. Those figures are rules of thumb for estimating; the only exact count comes from running your text through the provider's own tokeniser.
Billing separates the two directions because the compute differs. Reading a prompt is a single forward pass over the whole sequence and parallelises well. Generating a reply produces one token at a time, each pass attending to everything written so far, and cannot be parallelised across the sequence. That is why output typically costs three to five times what input costs on the same model.
Two consequences follow, and they are the reason this calculator separates the two lines. First, a long prompt is cheaper than it feels: 10,000 input tokens at $3 per million is 3 cents. Second, a long answer is more expensive than it feels: 2,000 output tokens at $15 per million is 3 cents as well, from a fifth of the text. If you want to reduce spend, find out which side dominates before you start editing anything.
Prompt caching adds a third price. Where a request begins with a long, unchanging prefix — a system prompt, a tool schema, a document being asked about repeatedly — providers can retain the computed state and charge a heavily reduced rate for re-reading it, commonly around a tenth of the standard input price. The saving applies only to the unchanged prefix and only while the cache entry lives, which is typically minutes rather than hours.
Working the arithmetic in units that do not trip you up
Prices are quoted per million tokens, and token counts are quoted in hundreds or thousands, so the safest habit is to divide the token count by 106 first and then multiply by the price. A 1,500-token prompt at $3 per million is 1,500 ÷ 1,000,000 × 3 = $0.0045. A 500-token reply at $15 per million is 500 ÷ 1,000,000 × 15 = $0.0075. Total $0.012 a request.
Multiplying up is where the number becomes a budget. At 5,000 requests a day for 30 days that is 150,000 requests, so $0.012 × 150,000 = $1,800 a month and $21,600 a year. The per-request figure looked negligible; the annual figure is a hiring decision. That gap is why per-request cost should never be the only number you look at.
With caching, split the input before pricing it. If 60% of a 1,500-token prompt is a cached prefix, that is 900 cached tokens at the cache rate and 600 fresh tokens at the standard rate. At $3 and $0.30 per million the input cost falls from $0.0045 to 600 ÷ 106 × 3 + 900 ÷ 106 × 0.30 = $0.0018 + $0.00027 = $0.00207, a 54% reduction on the input line. Note the sign convention this calculator uses: the caching figure it reports is the difference against pricing all input at the standard rate, so it is positive when the cache rate is below the input rate and negative in the unusual case where you have entered a cache write price by mistake.
Reasoning tokens deserve a mention because they break naive estimates. Models that produce an internal chain of thought bill those tokens as output even when they are not shown to the user, so measured output can be several times the visible answer length. If your provider exposes a reasoning-token count in its usage response, add it to the output figure here rather than counting the characters you received.
Worked example: a support assistant at 5,000 requests a day
A customer-support assistant sends a 1,200-token system prompt plus roughly 300 tokens of user message and retrieved context, and produces a 500-token reply. Prices are $3 per million input, $15 per million output and $0.30 per million cached. Volume is 5,000 requests a day, 30 days a month.
- Input tokens. 1,200 + 300 = 1,500 per request.
- Input cost with no caching. 1,500 ÷ 106 × $3 = $0.0045.
- Output cost. 500 ÷ 106 × $15 = $0.0075.
- Cost per request. 0.0045 + 0.0075 = $0.012.
- Output share. 0.0075 ÷ 0.012 = 62.5%. Output is the larger line despite being a third of the tokens.
- Requests per month. 5,000 × 30 = 150,000.
- Monthly spend. $0.012 × 150,000 = $1,800, and $21,600 a year.
- Now cache the system prompt. 1,200 of the 1,500 input tokens are a fixed prefix, so the hit rate is 80%. Input cost becomes 300 ÷ 106 × $3 + 1,200 ÷ 106 × $0.30 = $0.0009 + $0.00036 = $0.00126.
- New cost per request. 0.00126 + 0.0075 = $0.00876, a 27% reduction, taking monthly spend to $1,314 and saving $486 a month.
- Compare against shortening the reply. Capping the answer at 300 tokens instead of 500 changes output cost to 300 ÷ 106 × $15 = $0.0045, which with the cached input gives $0.00576 per request — $864 a month, saving $936 against the original.
Steps 9 and 10 are the point of the exercise. Caching the entire system prompt saves $486 a month; asking for shorter answers saves $936, because output is priced five times higher. Both are worth doing, but if you only have time for one change, the arithmetic tells you which.
How to read the result
Start with the output share. Above about 70% the bill is a generation-length problem: cap max_tokens, ask for structured or terse output, and stop the model summarising what it just did. Below about 30% it is a prompt-length problem: trim the system prompt, retrieve fewer chunks, and enable caching for whatever prefix is stable.
Cost per request is the wrong number for a decision on its own. Multiply it by realistic volume before you conclude anything, and remember that volume for a successful product grows faster than token counts fall.
The monthly figure is a floor. Retries, failed parses that trigger a second call, evaluation runs, and development traffic all add requests that no usage estimate contains. A 20–30% allowance over the calculated figure is a reasonable planning margin, and your provider's usage dashboard will tell you the real ratio within a week of launch.
Compare models on cost per completed task, not cost per token. A model at four times the token price that succeeds first time beats a cheaper one that needs two attempts and a repair prompt. Build a small evaluation set, measure success rate and mean tokens per task for each candidate, and multiply.
Watch for tiering. Some providers charge a higher rate above a context-length threshold, offer batch endpoints at a discount for latency-tolerant work, and price cache writes above the standard input rate. All three change the arithmetic and none of them appear in a single headline price.
Token counts for familiar amounts of text
| Content | Approximate words | Approximate tokens | Input cost at $3/1M |
|---|---|---|---|
| One tweet-length message | 40 | 53 | $0.00016 |
| A paragraph | 100 | 133 | $0.0004 |
| One page of prose | 500 | 665 | $0.002 |
| A detailed system prompt | 900 | 1,200 | $0.0036 |
| A ten-page document | 5,000 | 6,650 | $0.020 |
| A 50-page report | 25,000 | 33,250 | $0.100 |
| A 150-page book section | 75,000 | 100,000 | $0.300 |
| A 300-page book | 150,000 | 200,000 | $0.600 |
Tokens = words × 1.33, rounded. Cost = tokens ÷ 10^6 × price. Output at $15 per million costs five times these figures for the same token count.
Where prompt caching actually pays
Caching rewards a long, byte-identical prefix that is re-sent often. A fixed system prompt with tool definitions, a document being interrogated repeatedly in one session, or a few-shot example block all qualify. It does not help when the prefix changes between calls — putting a timestamp or a user name at the top of the system prompt invalidates everything after it, because the cache matches on an exact prefix. Cache entries also expire after a short idle period, so a low-traffic endpoint may pay the write cost repeatedly and read the cache rarely. Order your prompt with the most stable content first and the variable content last, and measure the realised hit rate from your provider's usage response rather than assuming it.
What this calculator does not price
- Cache write costs. Some providers charge a premium above the standard input rate to create a cache entry. On a low-hit-rate endpoint that premium can exceed the read saving.
- Reasoning tokens. Where a model produces hidden intermediate reasoning, those tokens are billed as output. Read them from the usage field and add them to the output count here.
- Embeddings, image and audio tokens. These are priced on separate schedules, often per image, per second of audio, or per million embedding tokens at a much lower rate.
- Batch and off-peak discounts. Latency-tolerant workloads submitted through a batch endpoint typically attract a substantial discount that this calculation ignores.
- Retries and evaluation traffic. Failed calls, schema-repair retries and regression suites all consume tokens that a usage forecast built from user actions will miss.
- Rate limits. Cost is not the only constraint. Tokens-per-minute and requests-per-minute limits shape architecture at least as much, and hitting them produces retries that cost money.
Fitting model spend into the rest of the bill
Token cost is rarely the whole cost of an AI feature. Retrieval-augmented generation adds an embedding call per document chunk at ingestion and per query at runtime, plus a vector database that is billed by stored dimension count and query volume. Those lines are usually small next to generation, but they scale with corpus size rather than with traffic, so they behave differently in a forecast.
The infrastructure around the model is priced like any other cloud workload: application servers, queues, storage for conversation history and egress for the responses. The cloud VM monthly cost calculator handles that side, and it is worth running both before you present a total, because a feature whose model spend is $1,800 a month can easily carry $600 of surrounding infrastructure.
Two structural choices move token spend more than any per-token negotiation. The first is routing: send the easy majority of requests to a small cheap model and escalate only the hard ones, which on a typical support workload cuts spend by more than half without a measurable quality change — provided you measure. The second is context discipline: retrieving three relevant chunks instead of twenty is both cheaper and usually more accurate, because irrelevant context degrades answers as well as costing money.
Finally, budget for growth in requests rather than in prices. Per-token prices have generally fallen as models improve, but successful features grow their call volume faster than prices fall. Model the volume curve you expect, run it through the table above, and check the annual figure at the volume you are hoping for rather than the volume you have — the same discipline the bandwidth requirement calculator applies to circuits, and for the same reason.