Editing Files at 1000 tokens/s with llama-70B
cursor.comhistory on this: this is something that aman was sort of challenged to do in a weekend as a sort of bet. it didnt end up working in that weekend but ended up working out overall! the post is quite fun as it walks through why the problem is hard.
its also important because it is a key primitive for doing good reliable multi-step code systems :)
what is the LLM doing to the files? Is it making edits based on a natural language command?
It edits the file based on the "plan" laid out by a smarter language model
That is clever, great idea.
Just downloaded again :-)