AI cost optimization: token cost review
Already running AI agents or a chat assistant? A token cost review shows where the model spend goes, and what to change to spend less without worse answers.
AI bills creep up quietly. A prompt that never gets cached, a long document sent on every call, a large model doing a small model’s job, a second pass nobody needed. Each one is small; together they add up. A token cost review finds them in your actual usage, not in guesses.
What I look at
- Spend by model and by feature, so you can see which parts of your product cost the most
- Prompt caching: how much of your input is cached, and what isn’t that could be
- Context size: what gets sent on every call, and how much of it the model needs
- Model choice: whether each step uses the right model for the job
- Extra calls: retries, second passes and checks, and whether each one earns its cost
- Cost per conversation, and the few conversations that cost the most
Where your AI bill goes
Every call to a model is billed by the token. Size up one typical call from your product and see which part costs the most.
- Instructions and documents
- History
- New message
- Reply
An illustration using Anthropic’s published price ratios: a reply token costs five times an input token, and input read from the cache costs a tenth of normal input (the one-off cost of writing the cache is left out). Your bill depends on your provider and models.
What you get
A written report with changes ranked by likely savings. Each one names the number in your usage that shows the problem, what to change, and any risk to answer quality. Changes that cost nothing in quality, like caching and trimming what you send, come before trade-offs like a cheaper model. You can make the changes yourself, or I can make them with you.
What I need from you
Read-only usage data: an export of your model usage, or admin-level access to your provider’s usage reports. I don’t need your customers’ conversations.
I run the same review on my own assistant, Finn, every month.
How we start
The price depends on how many systems and providers are involved, so we go through it on a 30-minute discovery call first, where I also tell you where I’d look first.