Extensible Software in the age of LLMs

> It’s become readily apparent that LLMs are really quite excellent at building Software for One. Personal apps that side-step all of the complexity and accountability of enterprise software and are custom fit for a single person’s workflow.

> …

> However most of our existing examples of pluggable software are local software: AI agents, developer IDEs, mods for video games, Blender add-ons, CAD extensions. These tend to be professional tools with a high barrier to entry. The web is the most successful software distribution system in the world. It shouldn’t be left behind. My hypothesis is that there is a new opportunity for Extensible Software on the web.

I don't follow. If this is really supposed to be "software for one", why would it need to be on the Internet? Why does it need a client/server model? Why do I care about "distribution"? People develop for web because native development gets painful when there are many flavours of "native". But you only use one of them yourself, and the LLM isn't bothered by its quirks.

Why not just work on designing pluggable local software that isn't so "professional"?

This is the right general idea, but it does read as an ad for Cloudflare OS.

Every tech company is scrambling to be the stable foundation for people in enterprise to build cute little one-off apps safely. It's a perfectly fine pattern, but it's hard to imagine a world where Cloudflare becomes the default. Much easier to imagine Google or Microsoft adopting whatever UI/UX patterns work well and tying into enterprise data natively.

I've been saying it to anyone who cares to listen: Salesforce is basically the biggest "Smalltalk" deployment out there. Super malleable, has a lot of ... decent rules for how to get software out into the world, every environment is basically its own universe at one point.

Someone should really attack SFDC from the "malleable Business OS" side of things. Unfrotunately everyone seems to attack it from the CRM side of things and lose the plot a bit.

Def agree that "have the thing I just came up with running in some environment that is sharable" is still a pretty gnarly problem, and LLM stuff doesn't... I don't think they've really made much progress on it.

At one point someone really needs to make an easy to manage box metaphor for all the stuff. It's not Docker. Google scripts is almost there but you really want to have the database be a thing you can drag and drop as well (SQLite people: get excited!)

Dropbox for business apps. If anyone runs with this at least send me an email thanking you for the metaphor.

I see a different future. A future where software developers are approached by clients with requirements in the form of an LLM generated program. They do it because they are at a point where LLM fails to make new changes without breaking existing stuff.

Maybe it won't be a program, but just some LLM context as some data dump.

This program or context will take up the role of a PM. Developers will refer to the program, or ask context for clarifications and the developer will build the actual program with or without the help from LLMs.

The long tail of unmet features is real. Most apps serve the common cases well, and LLM-generated extensions could fill that gap without bloating the core product.

As I said elsewhere: The future of tools like github is a platform for manual testing, where you write a prompt, the AI proposes a change and you can experiment with the UI and attach notes for the next iteration. AI can take user requests, prioritize, aggregate into tickets, and turn them into pull requests.

For a lot of end users, this may be enough, no programmers will be needed to get software built and shipped. For the rest, it lets programmers fill the remaining gaps, doing the manual testing to make sure the system works correctly, test for regressions and make sure the LLMs add those to the test suite, and then manage monitoring the rollouts. The bulk of development work going forward is manual verification that the LLM understood the user request correctly.

I'm not as sure that this idea of plugins will pan out; AI will want to make changes to support what it produces.

I’m building one of these Podda [1], though pointed at households/small communities rather than companies, so ordinary people can keep the apps they’ve made by talking to Claude or ChatGPT and share them with their friends.

Passing code only the things it’s allowed to use works on the server because you start from zero, so our generated code holds no credentials at all and its only way out is a proxy that allows exact origins and methods.

You can’t really do that in the browser. CSP only restricts which origins the code can reach, not the method or the path, so approving one destination means anything the code can read can go anywhere there. The difficult part is that if we're writing an honest consent prompt to our users then it has to say that, and it sounds a lot worse than "allow network access?". This is hard especially when our target users are non/less-technical. There are other versions of the same problem everywhere, like revoking an origin not actually taking effect until a refresh.

[1] https://podda.app

Sandboxed execution is definitely one aspect... But IMO, this is still not secure enough for vibe coders. They will want to have data-driven apps to share among small groups of people, then the security of the sandbox doesn't matter if they expose some external endpoints and if the access control logic which guards data is flawed.

Even if each user gets their own sandbox, they will still want to configure different access rules for different kinds of data which they host.

That said the idea that each user could control and host their own data is interesting and could work. I imagine you could have apps which link data from many different user sandboxes via remote foreign keys.

You could have a centralized data schema controlled by the application owner but the data itself would be held/scattered across a large number of sandboxes.

I really enjoy working on my inhouse orchestration tool because it brings me back to the days of working in Smalltalk image where I can tweak my development IDE as I go along.

Me and a Cofounder are building something along these lines on top of K8s. The wisdom that "if give the LLM clear boundaries and you get better results" definitely holds IME, even with frontier models like fable. It's impossible to specify all the guardrails you need to keep models from violating priors without writing the code yourself, so the only move is to remove their ability to do so, or to even recognize that the option exists.

I built a Lisp interpreter and hosted it on an OCaml http core, and it does exactly this for me.

I've been using it for the last three months for basically all my computing. I always wanted a Smalltalk type environment, and this finally scratches this itch.

Creating an endpoint is just defining a lisp function.

There's still a billion things to clean up but it works really well.

My LLMs say that if you followed this same pattern in pure clojure or racket it would be comfortably <10k lines of code.

That said, I do enjoy implementing my own lisp and I think the OCaml core gives it some stability and security.

We're building a collaborative IDE but for notes/tasks as a local-first web app [1], built to be malleable with plugins. In terms of tradeoffs we're taking a middle road I guess: the app is malleable, but plugins stay contained within the web app.

Because of end-to-end-encryption and privacy/ownership concerns, we can't run plugin code on a server node somewhere. But for some types of apps, server-side extensions alone also feel too restrictive. An advantage of a malleable app over just one-shotting a completely custom app (and to be clear I think both will be popular!) is that extending and customizing something is often exactly what you want vs starting over, but you can't really do that when you can't do things like deeply customizing the UI. So you can just ask a (local) agent "add me this functionality" and it will likely be possible.

It's still sandboxed inside the browser though, so this way you can do things like permissions, allowing admins to restrict who installs team plugins and so on (which is also important for malleable multiplayer apps).

[1] https://thymer.com

> It took me thirty minutes with Codex and GPT-Sol. The thing is fast, it does not ask me for anything, and it does the one job I wanted: it launches applications.

This is quite funny, actually. When I wanted something similar (just play MP3s, without playlists, special indexes, etc), it also took me 30 minutes.

In 2002.

I still use it daily - a wish application displays entries matching the filter using the locatedb to get a list of all MP3s to find them and mpg123 to play them. This is the application that I have used the longest in my life, unchanged and unmodified from day 1, and it still tickles me that it took 30m.

I'm pretty certain, had LLMs not existed, that I can make an application finder using locatedb (perhaps store a private index as well) and a GUI in about 30m using wish as the front-end.

(Just for reference, here's my no-frills MP3 player) ---------------------------------------------

    #!/usr/bin/wish
    # Copyright Lelanthran K. Manickum 2002, provided under BSD license
    #

    set version "1.0"

    proc playSong {songname} {
       set rc [catch { exec killall -9 mpg123 }]
       if {$songname=="\[Stop Playback]"} {
          .midFrame.lblPlaying configure -text "Stopped"
          return 0;
       }
       set rc [catch { exec mpg123 --loop -1 "$songname" & }]
       if {$rc==0} {
          .midFrame.lblPlaying configure -text "Playing: $songname"
       }
    }

    proc locateSongs {pattern} {
       .lstResults delete 0 999999999
       set tmpvar [split [exec locate -i "*$pattern*.mp3"] "\n"]
       .lstResults insert 0 "\[Stop Playback]"
       foreach title $tmpvar {
          .lstResults insert 1 $title
       }
    }

    frame .topFrame 
    frame .midFrame

    entry .topFrame.entSearchString -text "Search String"
    button .topFrame.btnSearch -text "Search" -command {locateSongs [.topFrame.entSearchString get]}
    listbox .lstResults -yscrollcommand {.sby set} -xscrollcommand {.sbx set}
    scrollbar .sby -orien vert -command {.lstResults yview}
    scrollbar .sbx -orien horiz -command {.lstResults xview}
    label .midFrame.lblPlaying -text "Stopped"

    locateSongs ""
    focus .topFrame.entSearchString

    bind .topFrame.entSearchString <Return> ".topFrame.btnSearch invoke"
    bind .lstResults <Double-B1-ButtonRelease> {playSong [.lstResults get active]}
    bind .lstResults <Return> {playSong [.lstResults get active]}


    grid .topFrame -row 0 -column 0 -sticky nsew
    grid .topFrame.entSearchString -column 0 -row 0 -sticky nsew
    grid .topFrame.btnSearch -column 1 -row 0 -sticky nsew
    grid .midFrame -column 0 -row 1 -sticky nsew
    grid .midFrame.lblPlaying -column 0 -row 0 -sticky nsew
    grid .lstResults -column 0 -row 2 -sticky nsew 
    grid .sby -column 1 -row 2 -sticky nsew 
    grid .sbx -column 0 -row 3 -sticky nsew 

    grid rowconfigure .topFrame 0 -weight 1
    grid columnconfigure .topFrame 0 -weight 1
    grid rowconfigure . 2 -weight 2
    grid columnconfigure . 0 -weight 2

    wm protocol . WM_DELETE_WINDOW {
        set rc [catch { exec killall -9 mpg123 } ]
        destroy .
    }

    wm title . "Simple Music Player - v$version"
    wm geometry . 700x500

> "code can only take actions via the references it has been passed"

The cool part is a move to capabilities composed in code brings back everything great about software itself: composability, encapsulation, type systems.

I've been working on this for SQL: instead of giving clients a handle to endpoints, you give them scoped query builders they can compose with (e.g., hand out a scoped `users` object and client can do `users.where(u => u.isActive()).orderBy(u => u.createdAt)`).

The result is you define a data-model (schema, computed columns, relations) and expose that instead of an endpoint per desired client query.

(if interesting: https://typegres.com/)

I doubt that future. Traditionally, only hardcore power users are interested in extending software. Normal end users want reliable software that does what they want it to do. Some of them also want a certain amount of shiny buttons and UI effects to look at. I'm not saying there is no market for heavily personalized software, just that it is not going to be big. Of course, people also want AI that acts like a friend or expert and does amazing things. However, I don't believe the two worlds mix well. IMHO, AI substitutes a person working for you whereas traditional software is a tool that persons and AI use.

Why is lovable not enough here by the way?

Extending software makes it more complex. As you keep adding extensions, the complexity grows until the app is unusable. Doesn't matter if it's built-in or a plugin, result is the same. But there is a proven alternative that works well.

Write completely separate small apps, and it's a different result. You get reliable functionality without an increase in complexity. Don't add a "file search extension" to your application (consider that if any other application wants this functionality, now they have to implement their own extension). Instead you make one completely separate app, called 'grep'. You then call that one 'grep' app, from any application.

It's a very old-fashioned idea to programmers who have only ever known custom-integrated REST APIs and microservices and giant monolithic frameworks. But this old-fashioned idea is the reason AI agents are even useful at all. They call those old-fashioned single-purpose external tools, and suddenly the agent has useful features, no custom extension needed.

Another thing you don't need: "a platform for platforms". There's already a platform designed to run interoperable applications. It's called an Operating System. It runs these little independent things called applications. They all have data object storage input-output access. Even internal communication between processes. And they're all compatible.

This is exactly what I did a few days ago: I created my own "small software" to solve my own need: https://news.ycombinator.com/item?id=49330939. Let me tell you: it works, it works really well, and I don't feel like, "Hey, the options out there are better."

In fact, the options available are outdated in terms of frameworks and bloated with features to please all kinds of customers. They try to do everything but end up doing everything poorly.