Here's what I learned on reducing token spend in your agent after spending $500 on iterations:
Reduce the number of loops. Every loop (or step) injects all of the tokens again, dramatically increasing costs. While caching helps, it's better to reduce the number of loops.
The easiest way to do this is to look for tools that are always called in a particular order, and collapse them into a single tool.
I collapsed 5 tools into 1 this way and saved 80% in token consumption for the same output.
Ensure your cache is stable. Caching unlocks massively reduced token pricing, but can be hard to get stable.
Place as much static content as you can at the start of the message history to maximize what can be cached, and make sure it's stable between loops.
Importantly, tool schema changes during a conversation can will invalidate the cache completely, so try to keep this to a minimum.
Tweak the history between loops. Removing unnecessary content from the history between loops based on what the model needs can help reduce the amount of wasted input tokens.
But changing the history too much can mess with your cache, so try to keep the message ordering as stable as possible, even if you truncate it.
Alternatively, accept that the cache will be invalidated, but the tweaked history will be cached separately.
Keep an eye on complex tool schemas. Large, complicated tool schemas contribute to input tokens, and might be increasing consumption unexpectedly.
Try reducing the size of these schemas, or consider tool searching to load them on-demand.
Use evals to measure your progress. Set up a good set of evals that measure token cost, and benchmark your output.
Measuring token cost is better than just measuring tokens, as it lets you test different models. Sometimes higher token consumption is necessary to achieve quality results.
Scoring your output will vary depending on your use-case, but an LLM judge can be very helpful in measuring subjective output.
These changes helped me reduce the cost of UI generation in Puck AI by 90% with 10% increase in quality as I moved from gpt-5.4 to gpt-5.6-luna. Happy savings!