Where your AI API bill goes, and how to reduce AI API costs

If your business runs a chatbot or an AI agent, you’ve probably looked at the monthly bill and wondered whether it’s normal. Honestly, there’s no normal. But there’s almost always a reason the number is higher than it needs to be, and it’s rarely the one people expect.

Most people assume the cost is in the answers. Usually it’s in the questions: everything your system sends the model before it writes a single word back.

What you’re actually paying for

AI providers charge by the token, which is roughly a piece of a word. You pay for what goes in (input) and what comes out (output). Output costs more per token, but input is where the volume is, because on every single call your system might send:

• A long set of instructions, the same ones every time
• The whole conversation so far, getting longer with each message
• Documents or search results pasted in so the model has something to work from
• Tool descriptions, if the agent can take actions

Then the model writes a two-line reply. You paid for all of it.

Where the waste usually is

Nothing is cached

If the start of every request is the same (your instructions, your product documents), most providers will cache it and charge a fraction of the normal price to read it again. It has to be switched on and set up right, though, and a lot of systems never do it. This is the first thing I check, because it saves money without changing a single answer.

Too much context

Whole documents get sent when one paragraph would do. A twenty-message history gets sent when the last five matter. Search results come back long and nobody trims them. Each of these is small on one call and large over a month.

The wrong model for the job

Not every step needs the biggest model. Sorting an email into one of four folders is a simple job; writing a careful reply to an upset customer isn’t. Plenty of systems use one expensive model for everything because that’s what the first prototype used.

Calls nobody remembers adding

Retries that fire too eagerly, a second “check your work” pass on every message, a summary that gets regenerated when nothing changed. These creep in during a build and stay.

The order I’d fix things in

Changes that don’t touch quality come first: caching and trimming what gets sent. Then the trade-offs, like moving a step to a cheaper model, and only after you’ve checked the answers still hold up. Saving money by making your assistant worse is a bad deal, and it’s easy to do without noticing.

Whatever you change, measure the cost per finished job (a resolved ticket, a processed invoice, a booked call) rather than just the cost per token. A cheaper model that needs three tries isn’t cheaper.

What I do with my own

Finn, the assistant on this site, logs every call he makes to the model: which model, how many tokens, what it cost. Twice a month an agent reads that log and writes me a short report on where the money went and what to change. It’s the same review I offer clients, just pointed at my own bill first.

A review like this only needs read-only usage data, such as an export from your provider’s dashboard. It never needs your customers’ conversations.

When it’s worth a look

If your AI bill has grown faster than your usage, if nobody on the team can say which feature costs the most, or if you built something quickly and never went back to it, it’s worth an afternoon. The token cost review page explains what I look at. If you’d rather talk it through, book a 30-minute discovery call and bring the bill.