📋 Table of Contents
- What Is a Token?
- Why Not Just Charge Per Word?
- Input Tokens vs Output Tokens
- Why Pricing Is Quoted Per Million Tokens
- What Actually Drives Your Bill
- Context Windows and Caching
- Estimating Your Own Usage
- Practical Ways to Control Token Costs
- Token Costs in Retrieval-Augmented Applications
- Comparing Providers Fairly
- Budgeting an AI-Powered Feature or Project
- FAQ
What Is a Token?
A token is the basic unit of text an AI language model actually processes — not a word, not a character, but a chunk somewhere in between, determined by the model's tokenizer. In everyday English text, a common rule of thumb is that 1 token is roughly 4 characters, or about 0.75 words — meaning 100 words of English text is typically around 130-150 tokens. Short, common words are often a single token; longer or less common words, and most punctuation, may be split into multiple tokens.
Tokenization also varies notably by language and content type: languages that use non-Latin scripts, and content like source code, dense math notation, or heavily repeated symbols, can tokenize less efficiently than plain English prose — meaning the same amount of "meaning" can cost more tokens depending on what you're actually sending.
Why Not Just Charge Per Word?
Tokens are a more accurate proxy for the actual computational work a model does than word count, because the model's internal processing operates directly on tokens, not on human-defined word boundaries. Two sentences with the same word count can require meaningfully different amounts of computation if one uses common short words and the other uses rare technical terms, code syntax, or non-English text — token-based pricing captures that difference; a flat per-word price would not.
Input Tokens vs Output Tokens
Nearly every major AI provider prices two separate token streams differently:
- Input tokens — the text you send to the model: your prompt, any system instructions, and any conversation history or reference documents included in the request.
- Output tokens — the text the model generates back to you: the actual response.
Output tokens are typically priced higher per token than input tokens — often several times higher — because generating each output token requires a full forward pass through the model, whereas input tokens can be processed more efficiently as a batch. This is why a request asking for a long, detailed response costs meaningfully more than one asking for a short, direct answer, even if the input prompt is identical in both cases.
Why Pricing Is Quoted Per Million Tokens
Individual tokens are worth fractions of a cent, so providers quote pricing as dollars per million tokens (sometimes abbreviated "per 1M tokens" or, on older pricing pages, "per 1K tokens") purely for readability — the underlying unit price is the same regardless of how it's displayed. As of recent years, publicly listed rates across the industry have spanned roughly from well under $1 per million tokens for smaller or open-source models up to $15 or more per million output tokens for the most capable frontier models — a wide range that changes frequently as providers release new models and adjust pricing, so always check current pricing pages rather than relying on a remembered figure.
What Actually Drives Your Bill
| Factor | Effect on Cost |
|---|---|
| Length of your prompt/instructions | More input tokens = more input cost, generally the smaller side of the bill |
| Length of the model's response | More output tokens = more output cost, often the larger side of the bill |
| Conversation history included each turn | Multi-turn chat re-sends prior messages as input tokens each time, so costs compound as a conversation grows longer |
| Attached documents/context | Large reference documents pasted into a prompt count as input tokens, sometimes the dominant cost driver |
| Model choice | More capable ("frontier") models are typically priced higher per token than smaller/faster models |
Context Windows and Caching
A model's "context window" is the maximum number of tokens (input + output combined) it can consider in a single request — modern models commonly support context windows ranging from tens of thousands to over a million tokens, though larger context windows don't reduce per-token price on their own. Some providers offer "prompt caching," which charges a reduced rate for input tokens that are repeated across multiple requests (like a long system prompt or reference document reused every call) — this can meaningfully cut costs for applications that repeatedly send the same large context with only a small varying portion.
Estimating Your Own Usage
To roughly estimate what a workload will cost:
- Estimate your typical input length in words, and convert to tokens using the ~0.75 words-per-token rule of thumb (words ÷ 0.75, or words × 1.33).
- Estimate your typical output length the same way.
- Multiply each by the provider's current per-million-token rate for that direction (input vs output), divided by 1,000,000.
- Multiply by your expected number of requests per day/month to get a total.
Example: A 500-word prompt (≈ 667 tokens) generating a 300-word response (≈ 400 tokens), run 1,000 times a month, at illustrative rates of $3 per million input tokens and $15 per million output tokens:
| Component | Calculation | Monthly Cost |
|---|---|---|
| Input tokens | 667 tokens × 1,000 requests × ($3 ÷ 1,000,000) | ~$2.00 |
| Output tokens | 400 tokens × 1,000 requests × ($15 ÷ 1,000,000) | ~$6.00 |
| Total illustrative monthly cost | ~$8.00 |
These are illustrative rates for demonstrating the math, not a quote for any specific provider or model — actual published rates vary widely and change over time, so substitute the current rate for whichever model you're actually using.
Practical Ways to Control Token Costs
- Trim unnecessary context — only include the conversation history or reference material the model actually needs for the current request.
- Set explicit output length limits where the API allows it, if your use case doesn't need long-form responses.
- Use a smaller/cheaper model for simpler tasks and reserve the most capable (and most expensive) model for genuinely complex requests.
- Take advantage of prompt caching if your provider offers it and you repeatedly send the same large context.
- Batch and monitor usage — most providers offer usage dashboards; check them regularly rather than discovering cost surprises at the end of a billing cycle.
Token Costs in Retrieval-Augmented Applications
Retrieval-augmented generation (RAG) — where an application searches a document store and inserts relevant passages into the prompt before asking the model a question — can quietly become one of the largest sources of input token cost, since every retrieved passage counts as input tokens on every single query, even though the model didn't generate that text itself. A RAG system that pulls in 3-4 lengthy reference documents per query can easily use far more input tokens than the user's actual question, meaning the real cost driver is the retrieval and context-assembly step, not the visible conversation. Reviewing exactly how much context your retrieval step injects — and trimming it to only the most relevant passages rather than entire documents — is often the single highest-leverage cost optimization available in a RAG-based application.
Comparing Providers Fairly
Because pricing structures, token-counting methods, and model capability all vary between providers, comparing "price per million tokens" alone across providers can be misleading. A cheaper per-token rate doesn't help if that model requires a much longer, more detailed prompt to get a usable result, or produces a lower-quality answer that requires additional follow-up requests to fix. A more complete comparison accounts for: the per-token rate for both input and output, the typical prompt length needed to get a good result from that specific model, the typical output length for your use case, and — for models with genuinely different capability levels — whether the cheaper model actually succeeds at your task reliably enough to avoid wasted retry requests, which themselves cost tokens.
Budgeting an AI-Powered Feature or Project
For teams building a product feature around an AI API rather than just experimenting casually, a rough monthly cost estimate before launch avoids unpleasant billing surprises. A reasonable approach: estimate your expected number of requests per day based on projected user activity, estimate typical input and output token counts per request using a representative sample of real prompts (not a best-case short example), apply the current per-million-token rates for your chosen model in both directions, and add a buffer (commonly 20-30%) for usage variability, retries, and any system-prompt or context overhead you might be underestimating. Revisit the estimate against actual billing data after the first few weeks of real usage — token consumption per request is often higher in production than in initial testing, once real user inputs and longer conversations are included.