Token Cost Optimization
Master the economics of LLM applications. Understand pricing models, calculate costs accurately, and implement strategies to reduce spend by 50-80% without sacrificing quality.
💰 Why Token Costs Matter
📥 Input Tokens
System prompt + context + user message. Cheaper but adds up with long contexts.
📤 Output Tokens
Generated response. 3-5x more expensive and harder to predict accurately.
📊 Model Pricing Comparison (2024)
| Model | Provider | Input $/M | Output $/M | Context | Best For |
|---|---|---|---|---|---|
| GPT-4o | OpenAI | $2.50 | $10.00 | 128K | Multimodal, vision |
| Claude 3.5 Sonnet | Anthropic | $3.00 | $15.00 | 200K | Coding, analysis |
| Gemini 1.5 Pro | $1.25 | $5.00 | 2M | Long context | |
| GPT-4o-mini | OpenAI | $0.15 | $0.60 | 128K | Cost-effective general |
| Claude 3.5 Haiku | Anthropic | $0.80 | $4.00 | 200K | Fast responses |
| Gemini 2.0 Flash | $0.075 | $0.30 | 1M | Best value | |
| Llama 3.1 70B | Self-host | ~$0.20* | ~$0.20* | 128K | Open, customizable |
| Mixtral 8x22B | Self-host | ~$0.15* | ~$0.15* | 64K | MoE efficiency |
* Self-hosted costs vary based on GPU and utilization. Estimates based on cloud GPU pricing.
🧮 Quick Cost Estimation
Formula
Example Calculation
🎯 Cost Optimization Strategies
Model Routing
Route requests to the cheapest capable model:
- • Simple queries: GPT-4o-mini, Gemini Flash
- • Complex reasoning: GPT-4o, Claude Sonnet
- • Classification: Fine-tuned small model
Prompt Optimization
Reduce tokens without losing quality:
- • Remove redundant instructions
- • Use shorter variable names in few-shot
- • Compress context with summaries
Caching
Reuse previous responses:
- • Exact match: Cache identical queries
- • Semantic: Embeddings similarity (>0.95)
- • Prompt caching: Anthropic/OpenAI native
Output Control
Limit expensive output tokens:
- • Set appropriate
max_tokens - • Request concise responses in prompt
- • Use structured output (JSON)
⚡ Prompt Caching (Native)
Both Anthropic and OpenAI offer native prompt caching for repeated system prompts—up to 90% cheaper!
Anthropic Prompt Caching
- • Mark cacheable content with
cache_control - • 90% discount on cached input tokens
- • 5-minute TTL (refreshes on use)
- • Min 1024 tokens to cache
OpenAI Prompt Caching
- • Automatic for repeated prompts
- • 50% discount on cached tokens
- • Works on GPT-4o and mini
- • Min 1024 tokens prefix match
📈 Cost Monitoring Best Practices
Cost per Request
Track by endpoint/feature
Daily/Monthly Spend
Set budget thresholds
Cost per User
Identify heavy users
Anomaly Alerts
Spike detection