For about two weeks now, my default client to headless Pi has been uni03C0, which I built in response to my own frustrations with the default Pi TUI, and use, among other things, to build a Web engine.
I’ve been improving on this Pi client to the point where I felt like writing a follow-up post to the last one, focusing on the responsive UI part (the sandbox part I haven’t touched since).
Agentic coding UIs pose a unique challenge: large content that is (almost) constantly updated.
This challenge cannot be solved by applying design patterns from chat UIs, because no matter how chatty you or your friends are, that is peanuts in comparison to the ever expanding wall of text that is an agentic coding session.
This core UI problem can be broken-up into the following parts:
- How to display the session history in an efficient and stable manner.
- How to search the session history.
- How to paste large input into the prompt editor.
- How to make search and paste instantly undo-able or at least abort-able.
All of the above questions can be answered from the following over-arching principle: never block the main rendering thread. Here block means anything that can take a significant amount of time. So for example decoding incoming data is a no-go; on the other hand, acquiring a lock, can be ok, if you can be fairly certain that it will rarely be contended.
So all expensive operations needs to either be avoided, or moved off the main rendering thread. For example re-processing the entire session history at each render tick can obviously be avoided: only process what is necessary for what is part of the current viewport.
The “move off the main rendering thread” thing is more subtle: not only should you move expensive operations out of the main thread, but you also need to ensure they are not part of any critical section (the stuff you do when holding a lock) on data shared with the main thread (because otherwise you can end-up blocking it even though work is “off main thread”).
For a concrete example of such an optimization, see this PR where I moved some expensive decoding out of a critical section: making critical sections as fine-grained as they can be is the low hanging performance fruit.
More complicated optimizations require turning one large chunk of work — one which for some reason cannot be done efficiently outside of a critical section — into multiple chunks of work that can be interleaved with other important stuff (like processing new incoming content).
For example, searching needs, by definition, to be applied to the entire session history. So if the session history is shared data that can only be operated on from within a critical section, you have two options: clone the history and search the clone, or, break the search up into batches.
Note: using reader writer locks can solve the easy problem of multiple concurrent readers; batching is rather a way to interleave writers to ensure smaller operations can make progress by being interleaved with a larger operation broken into batches.
I like the second option: batching the search, because what I care about is not “time to full results” but rather “time to first result”. So that gives you search over the entire session, where results will be updated in nearly imperceptible batches, and you can almost instantly look at the first result and then also quickly after that start cycling results as they come in.
The search was optimized to keep the UI responsive, using various methods, in this PR.
The same thinking can be applied to processing a large paste: show the tail of the paste first (in the prompt input box), then process the rest in batches in the background and update the view incrementally.
This then brings us to the last part: how to make everything always undo-able or abort-able? Well if every large operation is broken up into incremental batches, that gives you a wedge in between each batch allowing you to apply an abort, and then undo the operation.
For example, if the user starts typing a new search while a previous one was still ongoing, you abort it at the next batch and undo the UI effects. Control-c should not only clear the prompt input, it should also abort the processing of a large previous paste.
All of the above is easily written, but hard to actually get right in practice (and yes, the coding agent does mess it up; copy and pasting this blog is not enough); but when doing extensive daily QA yourself, you can catch problems and address them as they come up.
And that’s it, you can see it for yourself by building and running https://github.com/gterzian/uni03C0