Show HN: Pi-Infinite-Context
github.comHi HN! Since I created this extension and heavily refined its core design, I'm not worrying about context size anymore. It just works so well. So I wanted to share. It is stress-tested with OpenAI and Anthropic models.
Since HN is sensitive to good readmes, I wanted to do it well here. I started writing it by hand, but noticed that it was not great. then I decided to go back to LLM help, but iterate on it until it was short, simple and readable. I hope you like it.
In this process I had an insight about AI generated content. Slop is not only the fact that AI was used, but that the creator did not care about a high quality result. I'm interested to hear what you think about this take.
To those of us who can barely keep up with the proliferation of homonyms (words with the same name and different meanings):
> Pi is a minimal agent harness.
Not 3.14159 and so on
Not a computer.
I just switch to pi from OpenCode (another polynomer), and have no intentions of going back
I won't touch anything that touches my context without very compelling evals. The readme is too minimal, imo
Which evals would you be interested in specifically? And what are you missing in the readme?
Isn't this going to completely slaughter cache hit rates?
valid question. I thought a lot about this. The prefix before every fold stays intact (cache hit). But for the messages after the fold there is always a cache miss. Folds don't happen too often, so in practice this turned out to be worth it, because the cache miss comes with a big token reduction in general. I don't have a solid cost analysis for this yet, though.