How LLMs accelerated Cloudy development

LLMs do help when writing code (we finally know how to center divs in all circumstances). But when editing large, non-trivial projects, they need a lot of guidance to avoid slop; they relieve developers of typing the code, not of thinking it through. So unless one is a very slow typist or using terribly verbose languages, not much gain will come from that direction.

Cloudy is typed on the best keyboards on the market and almost all its code from the agents to the web front-end is written in OCaml, a language that’s everything but verbose.

Still, LLMs are making the development much faster. How come?

Obsoleted by LLM: UI, documentation, onboarding, programmers

You would think that in developing a very technical, low level product such as Cloudy, most of the time is spent doing system programming, network analysis and infrastructure. But you would be wrong: most of the development time is spent in implementing, testing and debugging the UI. Yes, like any run-of-the-mill e-commerce website.

Here are some rough stats on Cloudy’s git repositories:

• 60% of lines of code are for the web UI (same language throughout, so the comparison holds)
• 50% of files touched by any commit are for the web UI

So it seems reasonable to estimate that half the work has been the UI.

Part of that is due to the accidental complexity brought by web technologies (let’s not get started on that), but also because many UI problems are genuinely hairy. One of them is editing arbitrary, deeply nested data structures with many specialized data types, such as the abstract syntax tree Cloudy uses to represent its testing programs. The need to edit a small DSL (Domain-Specific Language) arises frequently in UI work, and I’ve never seen a good solution for it; I came to consider this the Holy Grail of user interfaces.

Even with a good reactive layout and responsive validators, most users will find such a form intimidating:

In addition to the UI itself, one needs to write:

• a contextual help,
• a comprehensive manual,
• a tutorial or two…

And then one needs users who are willing to make the investment of learning your software. The most precious of them, those who are just giving your app a 15-minute test drive, don’t fit that description.

The fundamental issue is that a lot of concepts in tech come naturally in the form of some formal language, and humans are not good at that. They need explanations and training that they have no patience for.

But guess who is good with formal languages? LLMs. Give an LLM a formal syntax with its specification, a couple of examples, and half a second later it is a power user.

Compare the above screenshot with this Claude-Code session:

And since an LLM is also proficient in natural languages, it can easily convert users’ requests from plain English into whatever formal language the app uses. And not only convert them: it also adds its own expertise to refine the task and avoid the pitfalls:

• checking the probes’ actual locations rather than trusting their names;
• being careful with Cloudflare’s anycast;
• picking the right protocol for the traceroute, in accordance with the unstated goal of the test;
• sequencing the httping first for better performance.

A form would do none of that.

That’s a win-win situation:

• For the author: no need to write anything more than a concise reference that stays within the MCP server, and no need to encode in the UI all that knowledge the LLM already has.
• For the user: no need to learn an application’s details, nor how to navigate its complex UI.

The LLM has become the user interface, and that’s where the real gains are coming from.

It’s nuanced.

Program editor? As we have just seen, the LLM is so much better!

Timeseries? Cloudy has a large variety of those of course, but the LLM also has good built-ins it can reach out to; or it could use a more powerful BI tool, which would make it easier to correlate that data with other sources. Still, since there is not yet a convention to pass a live stream of data, the interaction would be poor.

Traceroute graphs? The LLM can present the user with a makeshift world map with clustered hosts and links between them, but that costs a lot of tokens, it has to resort to Leaflet and Python, and the result is neither consistent nor interactive.

So for now Cloudy’s UI is going nowhere, but the few specialized widgets that are truly unique are being made available in isolation, so that the LLM can direct the user at them. The other, more trivial widgets (most of the UI, honestly) might fall into disuse.

Objections

A UI pushes important things in front; without one we are in the dark

Cloudy’s UI tries hard to make anything unusual visible, using visual codes such as:

• Data in charts and graphs are color coded to attract the attention to bad data;
• Timelines feature markers when bad data is present;
• Bad data such as failing probes, BGP flapping, etc will be displayed prominently in flashy colors;
• Buttons opening a panel that has bad data will blink to attract user’s attention.

It’s doable, but it’s hard work.

On the other hand, the LLM will spontaneously report anything that looks unusual or bad, and will add context to it. Conversely, it will not freak out over a probe that fails for a trivial reason, whereas the UI is incapable of such nuance.

Following up on the previous example, notice how it rightly ignores a failing probe without losing its calm:

That’s an area where the fuzzy logic of LLMs defeats automatic behavior.

Without a UI, how can users discover what’s doable?

Discoverability is only a requirement because the human had to drive. Users don’t want to know what’s doable, they want to state a goal.

Cloudy is just another tool in the agentic SRE’s toolbox, there to expand its investigative power. Here is all the user needs to know: Cloudy gives the SRE a cost efficient way to continuously perform many network tests, and to consult the historical results of those tests, from high level trends down to individual network packets. The MCP descriptions will tell the LLM all the details. The LLM already has the background knowledge required to not only make sense of the tools and their output, but also to know in which situations to reach for them.

A tool might look cool and powerful yet be useless, whereas another might look obscure but come in handy in many real world scenarios. The real measure of Cloudy’s usefulness is not easily assessed from a quick tour of the UI; it is best measured by how often the agentic SRE reaches for that toolbox.

A UI gives a quick, free and private access to the app, unlike an LLM

This is undeniable, and it is precisely why Cloudy still has a UI: for those cases when clicking the right button is faster than describing it in plain English, for those situations where no third party can be admitted in the loop regardless of anonymisation, or for when the agent itself is down.

But let’s be honest: this objection has had very little influence over the evolution of IT infrastructure for the last two decades.

So, how to be more productive with AI

Based on all the above, here are a few recommendations:

1. Design your UI for the LLM:

• The MCP is the main access point and the only documentation.
• The navigation boils down to: human user is sent by the harness to a custom widget with a specific selection of data, then leaves.
• Do not manage users, sessions or access control: by the time a request reaches your service, the harness has already authenticated the user upstream.

2. Don’t shy away from formal languages:

• They have a bad rep with humans, but not with LLMs.
• Once the UI is out of the way, it is very cheap to expand the reach of a service by making it programmable.
• Provide a dry-run mode, or a playground for the LLM to experiment in.

3. LLMs make user acceptance testing cheap:

• Connect an LLM to your MCP early, and save time on experimentation.
• Ask the LLM what it needs: more often than not, it has a good idea of what it could do with your service.
• Watch what it does: a useless tool call is a clear sign that something is ambiguous or missing. (for instance, Claude used to ask for program statuses for seemingly no reason. It turned out to be confused by some important fields that were omitted from the answer when they had their default value; an easy fix, once identified.)

Prediction: harnesses are the operating system of the future

Natural language will become the main interface to applications.

Applications will become mere data providers.

Data from one app will be seamlessly combined with data from another, because all of it ends up in the LLM context, which mixes it to build the data that users actually want.

The temporary era of walled gardens, when data belonged to specific applications, will go out of fashion.

When the answer is not best expressed in words, LLMs will hand off streams of data with well-known shapes and semantics (ontologies) to generic visualizers, either built-in or provided by third parties.

So users won’t see applications as applications any more, only widgets summoned for a moment; and applications won’t see users: no more profiles, accounts or sessions - all that is the LLM context.

The operating system of the future could be just that: a harness that connects LLMs, MCP servers providing either services, data or widgets, and local, mostly transient files.

What will remain unchanged are the networks connecting all this and the SREs dedicated to take care of them, except that now the SREs are seconded by agents equipped with a variety of tools.