LLM API costs are easier to control when they are measured as a repeatable unit rather than treated as a monthly surprise. This guide shows how to track tokens, estimate per-request and monthly spend, choose models by task, reduce unnecessary output, use caching and batching carefully, and set a review process that remains useful as traffic, prompts, and provider pricing change.
Overview
The basic cost of a language-model request is usually related to the amount of input and output text processed. Providers may price these token categories differently, and some offer separate treatment for cached input, batch processing, tool calls, images, audio, or other modalities. Because pricing structures and model names change, a durable cost-control system should store the pricing inputs separately from application code.
LLM API cost optimization is not simply a matter of selecting the cheapest model. A smaller model may require retries, longer prompts, or additional validation if it performs poorly on a task. A more capable model may reduce failure handling or produce a useful answer in one pass. The practical goal is to find the lowest total cost that still meets your quality, latency, reliability, and safety requirements.
A useful operating loop has five parts:
- Measure input and output tokens by feature, model, and environment.
- Estimate unit cost using the provider's current pricing inputs.
- Reduce avoidable tokens through prompt optimization, output limits, and better data preparation.
- Route each task to an appropriate model and execution mode.
- Review spend and quality together before making a change permanent.
For model-selection decisions, pair this workflow with a structured evaluation such as the LLM model selection and testing framework. Cost alone is not a sufficient production metric.
How to estimate
Start with a per-request calculation. Use the following variables:
- I: input tokens per request
- O: output tokens per request
- Pi: input price per million tokens
- Po: output price per million tokens
- R: number of requests
- S: number of successful or billable requests, depending on the provider's billing model
The estimated request cost is:
((I / 1,000,000) × Pi) + ((O / 1,000,000) × Po)
The estimated period cost is:
request cost × S
If your application uses several features, calculate each feature separately and add the results. A support assistant, document summarizer, classification job, and retrieval-augmented generation workflow can have very different token profiles. A single average may hide the feature responsible for most of the spend.
Track at least these fields in logs or an analytics table:
- timestamp and environment
- feature or route name
- model identifier
- input tokens and output tokens
- cached input tokens, when reported separately
- retry count and failure reason
- latency and quality outcome
- estimated cost using the pricing version active at the time
Do not estimate from character count alone when a tokenizer is available. Tokenization varies by language, formatting, code, identifiers, and markup. For early planning, a rough estimate can be useful, but production budgeting should use token counts returned by the API or measured with a compatible tokenizer.
Provider comparison framework
Instead of copying a price table into application logic, compare providers and plans using the same questions. Confirm the current values in each provider's official billing documentation before committing budget.
| Option | Cost inputs to compare | Best fit to investigate | Important check |
|---|---|---|---|
| Standard hosted API | Input tokens, output tokens, minimums, and included features | Interactive production requests | Whether retries, tools, and multimodal inputs add separate charges |
| Cached-prompt option | Uncached input, cached input, cache writes, and cache duration | Requests that reuse a large stable prefix | Eligibility rules, minimum prefix size, and cache invalidation behavior |
| Batch or asynchronous processing | Input and output rates, batch limits, and completion windows | Summaries, enrichment, and offline classification | Whether the feature supports your latency and retry requirements |
| Self-hosted or local inference | Hardware, hosting, electricity, maintenance, and engineering time | Predictable workloads or privacy-sensitive processing | Utilization, model quality, scaling, and operational overhead |
This framework belongs alongside an application architecture review. The AI developer tools comparison guide can help identify where observability, evaluation, and deployment tooling fit into the decision.
Inputs and assumptions
A cost estimate is only as useful as its assumptions. Record them in a small worksheet that can be copied for every model or feature:
| Input | Example assumption | How to improve it |
|---|---|---|
| Requests per day | 2,000 | Use production logs or a traffic forecast with peak and average cases |
| Average input tokens | 3,000 | Measure system prompt, user text, retrieved context, and tool results separately |
| Average output tokens | 600 | Measure actual responses and enforce a suitable maximum |
| Retry rate | 5% | Count automatic retries, fallbacks, and client-side timeouts |
| Cached-input share | Assume none until verified | Use provider-reported cache metrics rather than an optimistic forecast |
| Billable days | 30 per month | Separate normal traffic from launches, imports, and one-time backfills |
Include the hidden token sources that often make an estimate too low:
- Repeated system instructions sent on every request
- Conversation history that grows without a retention policy
- Retrieved documents that are longer than the task requires
- Tool schemas and function definitions
- Structured-output instructions and examples
- Retries caused by timeouts, malformed JSON, or failed validation
- Evaluation traffic, staging tests, and background jobs
Prompt optimization can reduce cost without changing the model. Remove duplicated instructions, summarize old conversation turns, retrieve only relevant passages, and ask for the smallest output that satisfies the downstream task. For structured workflows, validate concise JSON or labels instead of requesting prose that another service must parse. The prompt optimization framework provides a useful test-and-compare approach.
Security changes can also affect token use. Do not remove safety checks merely to shorten a prompt. Instead, separate stable policy instructions from variable user data, keep untrusted retrieved text clearly delimited, and evaluate the result against injection risks. See the prompt injection defense guide before optimizing a security-sensitive workflow.
Worked examples
Example 1: interactive requests
Assume, for illustration only, that a model's input price is set in your worksheet to 0.50 currency units per million input tokens and its output price is 1.50 currency units per million output tokens. These are example inputs, not current market prices.
Suppose each request uses 3,000 input tokens and produces 600 output tokens:
Input cost = (3,000 / 1,000,000) × 0.50 = 0.0015Output cost = (600 / 1,000,000) × 1.50 = 0.0009Estimated request cost = 0.0024
At 2,000 requests per day for 30 days, the unadjusted estimate is:
0.0024 × 2,000 × 30 = 144 currency units
If the application adds a 5% retry allowance, multiply the request volume by 1.05, provided that this reflects the actual billing behavior of the integration. The revised estimate is approximately 151.20 currency units under these illustrative assumptions.
Example 2: reducing output rather than changing models
Using the same assumptions, reducing the average output from 600 to 300 tokens changes the output portion from 0.0009 to 0.00045 per request. The saving per request is 0.00045, before considering any quality or retry impact. At scale, this can matter, but only if the shorter response still contains the fields or explanation the user needs.
Test this change on a representative evaluation set. Measure task completion, validation failures, follow-up requests, and human corrections. A shorter answer that causes users to ask a second question may not reduce total cost.
Example 3: deciding whether caching helps
Consider a request with a stable 2,500-token system prompt and policy prefix plus 500 tokens of changing user content. Caching is worth investigating because much of the input repeats. Compare the provider's uncached-input treatment, cached-input treatment, cache-write behavior, and eligibility rules. Then calculate savings using observed cache-hit rates rather than assuming every request will qualify.
Also account for cache misses after prompt changes, expiration, tenant separation, and deployments. A cache design that exposes one customer's context to another would be unacceptable regardless of its cost benefit. For retrieval-heavy applications, review both caching and context reduction in the RAG workflow guide.
When to recalculate
Make cost review a normal part of AI application development, not an emergency response. Recalculate when:
- the provider changes model pricing, billing units, cache rules, limits, or plan terminology;
- you migrate models or add a fallback model;
- traffic increases, especially during a launch or seasonal peak;
- a prompt, retrieval setting, tool schema, or response format changes;
- retry, timeout, or validation-failure rates move materially;
- you add images, audio, embeddings, batch jobs, or other billable operations;
- quality requirements change and a more capable model becomes necessary;
- you begin a monthly finance, capacity, or infrastructure review.
Use this monthly checklist:
- Export usage by feature, model, environment, and tenant where appropriate.
- Compare actual input and output tokens with the worksheet assumptions.
- Separate normal usage from tests, retries, backfills, and one-time migrations.
- Review the top cost drivers and identify whether they are volume, prompt size, output size, model choice, or failure related.
- Check quality metrics before routing work to a cheaper model or shortening outputs.
- Verify provider pricing, context limits, caching terms, and billing definitions.
- Update the worksheet, dashboards, and budget alerts with a dated pricing version.
Set alerts at both daily and monthly levels, with a separate threshold for unusual request volume or token growth. A budget alert should lead to an operational action: rate limiting, queueing non-urgent jobs, switching an approved fallback, or pausing a backfill. Define those actions before an incident occurs.
Finally, keep cost data close to evaluation data. The best LLM API cost optimization decision is not the one with the smallest token bill; it is the one that delivers the required result with measured quality, predictable reliability, and a cost the application can sustain.
