Idea to reduce AI token use at large orgs
There must be many near-identical repeats of high-token tasks.
Perhaps for high-token tasks, first an automated search of prior work within the firm, parsing what's already done or not done, and only prompting AI for the new components, then assembling the components before serving to the user.
If lengthy components were already done. the overall output might be faster.
Is this an add-on to OpenRouter, or standaloone?
1 thread
This sounds more like a semantic caching layer on top of OpenRouter. The real challenge is knowing which parts of previous work are still reusable and only sending the delta to the model.