AI costs have a habit of creeping up. What starts as a £200/month prototype can become a £5,000/month production bill surprisingly fast. The good news is that most AI systems are dramatically over-spending because they have not been optimised. We routinely help clients cut their AI costs by 50-70% without any meaningful loss in quality.
Here are the strategies that work.
Model Selection: Stop Using a Sledgehammer for Every Nail
The single biggest cost mistake we see is using the most powerful (and expensive) model for every task. GPT-4o and Claude Opus are remarkable, but they cost 10-30x more per token than their smaller siblings. For many tasks, that premium buys you nothing.
Consider what your task actually requires:
- Simple classification (spam detection, sentiment analysis, ticket routing): GPT-4o-mini or Claude Haiku handles these with near-identical accuracy to larger models at a fraction of the cost.
- Data extraction (pulling structured fields from documents): Smaller models are excellent at this, especially with well-structured prompts and output schemas.
- Summarisation: Unless you need nuanced, executive-quality summaries, mid-tier models produce perfectly adequate results.
- Complex reasoning, multi-step analysis, creative strategy: This is where frontier models earn their premium. Use them here and save money everywhere else.
A practical approach is to build a model routing layer. Classify incoming requests by complexity and route them to the appropriate model. Simple tasks go to cheap models. Complex tasks go to powerful ones. We have seen this single change reduce costs by 40-60% with no user-visible quality degradation.
Prompt Optimisation
Tokens cost money, and most prompts are far longer than they need to be. Every extra token in your system prompt is multiplied by every request, which adds up fast at scale.
Trim your system prompts: We regularly audit client prompts and find they can be reduced by 30-50% without any change in output quality. Remove redundant instructions, eliminate examples that do not improve performance, and use concise language. A 2,000-token system prompt that could be 800 tokens is costing you 2.5x more than necessary on every single request.
Use structured outputs: Instead of asking the model to generate free-text responses that you then parse, use JSON mode or function calling to get structured responses directly. This reduces output tokens (which are more expensive than input tokens on most models) and eliminates the need for post-processing.
Batch similar operations: If you are processing multiple items, batch them into a single prompt where possible. Processing 10 customer reviews in one call is significantly cheaper than 10 separate calls, because you pay for the system prompt only once.
Caching and Batching
Caching is the most underused cost reduction technique in AI systems. If you are making the same or similar API calls repeatedly, you are paying for the same work multiple times.
Exact-match caching: Store the response for every unique prompt and return the cached result for identical requests. This is trivially easy to implement (a simple key-value store keyed on a hash of the prompt) and immediately eliminates duplicate calls. We typically see 15-30% cache hit rates even without semantic matching.
Semantic caching: For more sophisticated savings, cache based on meaning rather than exact text. If a user asks "What is your refund policy?" and another asks "How do I get a refund?", these can serve the same cached response. This requires embedding the queries and matching against cached embeddings, but the cost savings can be substantial for customer-facing applications with repetitive queries.
Batch APIs: Both OpenAI and Anthropic offer batch processing endpoints that run asynchronously at significantly reduced rates (typically 50% off). If your workload does not require real-time responses — think nightly report generation, bulk classification, or scheduled data processing — batch APIs can halve your costs immediately.
Prompt caching: OpenAI and Anthropic now offer automatic prompt caching, where repeated system prompts are cached server-side and charged at reduced rates. If you have long system prompts that are consistent across requests (which you should), this provides a meaningful discount with zero implementation effort.
Using Smaller Models for Simple Tasks
This deserves its own section because it is the highest-impact change most organisations can make.
The AI industry has a bias towards using the most capable model available. But capability has a cost, and for many production tasks, the most capable model is massive overkill.
A concrete example: one client was using GPT-4o to classify incoming emails into five categories. The accuracy was 97%. We switched to GPT-4o-mini. The accuracy was 96.5%. The cost dropped by 95%. That 0.5% accuracy difference was completely invisible in practice but the cost saving was £2,000 per month.
Another pattern: use a small, fast model as a first pass, and only escalate to a larger model when the small model is uncertain. For classification tasks, this means routing to the frontier model only when the smaller model's confidence score is below a threshold. In practice, 80-90% of requests are handled by the cheap model, and the expensive model only processes the genuinely difficult cases.
For on-premise or privacy-sensitive deployments, open-source models running locally eliminate API costs entirely. Models like Llama 3.3, Mistral, and Qwen are now capable enough for many production tasks, and the inference cost is just the electricity to run your GPU.
Monitoring Costs: What Gets Measured Gets Managed
You cannot optimise what you do not measure. Every production AI system should track:
- Cost per request: Broken down by model, endpoint, and use case. This tells you where to focus optimisation effort.
- Token usage per request: Input tokens and output tokens separately. This reveals whether your prompts are bloated or your outputs are unnecessarily verbose.
- Cost per outcome: The truly useful metric. What does it cost to classify a ticket, process an invoice, or generate a report? This lets you calculate ROI and compare against the manual cost of the same task.
- Cost trends: Weekly and monthly trends that alert you to cost increases before they become budget problems. Set alerts for anomalies.
We build cost dashboards into every AI system we deploy. The visibility alone drives better decisions. When a team can see that their weekly AI spend just increased by 40%, they investigate and optimise. Without visibility, costs drift upward silently.
For a comprehensive approach to managing AI in production, see our AI operations services.
If your AI costs are higher than you expected or you want to optimise before scaling up, get in touch for a free cost review. We'll analyse your current usage and identify the quickest wins.