We’re open sourcing Resident, our library for running AI-authored code on microcontrollers – with no compile step and no firmware flashing.
It’s our twist on vibe coding firmware, built for instantly loading new device functionality coded by end users. It’s aimed at device developers, like us. We use Resident in all our work.
(We = Inanimate.)
Resident gives you a code sandbox on ESP32 devices and a driver API to provide hardware control and events. So an end user can push an app over Wi-Fi that instantly turns their clock into an interactive pill timer (for example) but the app can’t run probes on the home Wi-Fi network.
It comes bundled with a set of Claude skills. I told Claude about the capabilities of a compatible dev kit and asked it to push a simple app. Here it is:
At Inanimate, we believe that on-device sandboxes are an essential low-level primitive for AI agents in the real world.
The announcement is over on Lab Notes but I want to take a moment to connect this to some previous themes…
Look, I want to bring software into my room.
Like: if I’m working with Claude Code and I step away from my keyboard, it should be able to give me updates on what it’s working on by taking over my desk clock, and ping its permission requests to the gumstick device I have in my pocket for me to accept/decline while I’m making tea. Untether me from my desk! (btw I built this, it was awesome.)
Or - more prosaically - why can’t I yell at my stove as I’m leaving the house in the morning oh I forgot I’m roasting a chicken that needs to be ready for 7pm and have it look up timings and push me a notification when it’s pre-heating, and show a custom basting timer app on its 14-seg LED display.
Why can’t I point at a lamp and say ‘on’ and the light come on?
– I was asking in 2020.
So half of the solution here is AI: LLMs are super good at translating intent into action. I use regular language and the computer will Do What I Mean (2025).
But the other half is a problem: how does this actually work? Where does the software run?
Every device needs to be able to run user code, that’s the answer.
I’ve been bouncing off this since 2023 when I put an LLM in charge of a smart home (using tech that we now call agents, but then was my implementation of the ReAct pattern from a paper out of Princeton and Google Research).
In a nutshell, let’s say you press a button on your desk clock and your AI agent has to decide in the moment what to do with that. It won’t work, it’s too slow. Cognitively, an interface has to respond inside 150ms or it is no longer instant – but the AI is a network hop away.
Ok so let’s remove the network hop. Let’s pretend we have edge AI: a GPT-4-equiv AI as a component in every physical thing; ubiquitous intelligence is coming and a Feynmann-level light switch could guess your intentions pretty well,
that was was my guess in 2023.
It’s not an outrageous extrapolation! Taalas is baking LLMs into silicon and delivers 17k tokens per second per user on Llama 3.1 8B
(try it here, e.g. 4,000 words on Hamlet as a space opera, it’s wild it’s so instant). So GPT-4-equiv is a matter of time.
Even with that speed, it turns that inference is not enough. To Do What I Mean, you still need to import user context for personalisation and grounding. But memory, Gmail, Wikipedia and the rest are still mostly in the cloud. That network hop again.
So AI can’t be inside the event loop, not if you want really great on-device interactions.
Instead – let the AI write device code. That’s the approach we’ve found.
Take that toy example of a roast chicken basting timer on the display of my stove. My AI agent should be able to dynamically write code for that interactive app, and have that app code executed on the stove itself.
I’m not saying that the AI needs to vibe-code firmware.
Firmware is the code that runs on device microcontrollers: you author it and compile it and you flash it and from that point on, it never changes. AI agents can vibe firmware (very happily). But I’m not sure I want to load code onto my stove that has boundless control over the heating element and the network stack and whatever other low-level capabilities are managed by the firmware itself. We’ve already had smart fridges sending spam email (BBC News, 2014), no more thanks.
Instead of firmware, the AI can run code in sandboxes.
Cloudflare made the case for sandboxes in March this year when they introduced dynamic workers:
Last September we introduced Code Mode, the idea that agents should perform tasks not by making tool calls, but instead by writing code …
You can’t just eval() AI-generated code directly in your app: a malicious user could trivially prompt the AI to inject vulnerabilities.
You need a sandbox: a place to execute code that is isolated from your application and from the rest of the world, except for the specific capabilities the code is meant to access.
Sandboxing is a hot topic in the AI industry …
We asked ourselves:
What if, when we allow an AI agent to spread its arms and stretch its legs in a room, it could inhabit devices by writing code that runs in an on-device sandbox, a sandbox that gives access to buttons and screens but not the network stack, and the app code could load and run instantly?
That’s Resident. It provides a code sandbox for ESP32 devices, and a toolchain for AI agents to write apps that target that sandbox.
At Inanimate, we use Resident for all our product prototyping.
And it will be at the heart of our future products.
Look, it may seem wildly disproportionate to write code to turn on a lamp.
But what are computers for? They do the hard work to make it easy for us. So this approach scales well from basic on/off control to… well, it turns out that, now we have this sandbox, we can compose all kinds of useful experiences that take over an entire room – and also weird and wonderful interactive ones that still work when you pull the network cable.
(An app arrives over the network but then the network is no longer required. Resident is built on our messaging library Courier which includes UDP multicast for local inter-device messaging even when internet connectivity drops.)
We’re opening Resident today as an alpha (v0.5.0) and open sourcing it under the highly permissive MIT license. Just include our copyright notice with any modifications.
Technically, we’re adding a Lua runtime to your ESP32 device. (Lua is a language designed to be embedded.)
We love Espressif’s ESP32 microcontroller family because it has a unique span in the ecosystem: it is used by individual makers, new hardware startups, and in production at real scale. It has built-in Wi-Fi and supports its native framework esp-idf and Arduino too, which is great for quick prototyping.
Resident gives you an API to add extensions to that Lua runtime: hardware drivers. Those managed capabilities are what makes it a sandbox. So the apps in the sandbox can respond to events that your button driver injects, and they can write to the display via a module added by your display driver.
The apps can be hot loaded at runtime. The way we have it wired up, you push app code down a websocket to the device and it run immediately in the sandbox.
Adding Resident is straightforward during development: bring up your new device as normal, then point your coding agent at docs/start-building.md. It’ll walk you through adding the sandbox and writing the drivers.
Resident comes bundled with:
- Connectivity: websockets, JSON messaging, and easy Wi-Fi config
- A default back-end server at
resident.inanimate.tech so you can easily push new apps and events – use the example code to build out your own back-end when you’re ready
- A collection of Claude skills to create, validate and push new apps to your devices (and even write device documentation)
- Example projects.
Want to try Resident now?
Ahead of developing your new device, pick up an M5StickS3. These are made by M5, I talked about them when we announced Courier.
The M5Stick is a dev kit that has an ESP32 with a screen, battery, couple of buttons, buzzer and IMU. You can bring it up using Arduino pretty simply. Then add Resident and start writing apps. We have an example project.
Want to get going even faster?
You don’t even need hardware…
The Try it now section of the Resident homepage has a M5Stick simulator, running the Resident sandbox in-browser.
Drag and drop an app onto the simulator to see it run.
Or even: install the Claude skills, tap the button on the webpage to make the simulator live, and create apps from your local Claude Code session. The in-browser simulator will update live and run your app.
Hey, deep cut reference alert:
Back in 2021 I went on a dive into files: Golems, smart objects, and the file metaphor.
What would it mean to drag and drop a file onto a lightbulb? I asked. Do I literally mean that the lightbulb needs a little slot like the golem’s mouth, into which you insert your instructions stamped on microfiche?
This is my answer haha
Resident is what we’ve been using to prototype products and use cases at Inanimate.
We believe that one day all products will work this way.
For more info, the GitHub, and to try it now: Resident.
For updates: subscribe to Lab Notes.
We’re open sourcing Resident, our library for running AI-authored code on microcontrollers – with no compile step and no firmware flashing.
It’s our twist on vibe coding firmware, built for instantly loading new device functionality coded by end users. It’s aimed at device developers, like us. We use Resident in all our work.
(We = Inanimate.)
Resident gives you a code sandbox on ESP32 devices and a driver API to provide hardware control and events. So an end user can push an app over Wi-Fi that instantly turns their clock into an interactive pill timer (for example) but the app can’t run probes on the home Wi-Fi network.
It comes bundled with a set of Claude skills. I told Claude about the capabilities of a compatible dev kit and asked it to push a simple app. Here it is:
At Inanimate, we believe that on-device sandboxes are an essential low-level primitive for AI agents in the real world.
The announcement is over on Lab Notes but I want to take a moment to connect this to some previous themes…
Look, I want to bring software into my room.
Like: if I’m working with Claude Code and I step away from my keyboard, it should be able to give me updates on what it’s working on by taking over my desk clock, and ping its permission requests to the gumstick device I have in my pocket for me to accept/decline while I’m making tea. Untether me from my desk! (btw I built this, it was awesome.)
Or - more prosaically - why can’t I yell at my stove as I’m leaving the house in the morning oh I forgot I’m roasting a chicken that needs to be ready for 7pm and have it look up timings and push me a notification when it’s pre-heating, and show a custom basting timer app on its 14-seg LED display.
– I was asking in 2020.
So half of the solution here is AI: LLMs are super good at translating intent into action. I use regular language and the computer will Do What I Mean (2025).
But the other half is a problem: how does this actually work? Where does the software run?
Every device needs to be able to run user code, that’s the answer.
I’ve been bouncing off this since 2023 when I put an LLM in charge of a smart home (using tech that we now call agents, but then was my implementation of the ReAct pattern from a paper out of Princeton and Google Research).
In a nutshell, let’s say you press a button on your desk clock and your AI agent has to decide in the moment what to do with that. It won’t work, it’s too slow. Cognitively, an interface has to respond inside 150ms or it is no longer instant – but the AI is a network hop away.
Ok so let’s remove the network hop. Let’s pretend we have edge AI: a GPT-4-equiv AI as a component in every physical thing; ubiquitous intelligence is coming and that was was my guess in 2023.
It’s not an outrageous extrapolation! Taalas is baking LLMs into silicon and delivers (try it here, e.g. 4,000 words on Hamlet as a space opera, it’s wild it’s so instant). So GPT-4-equiv is a matter of time.
Even with that speed, it turns that inference is not enough. To Do What I Mean, you still need to import user context for personalisation and grounding. But memory, Gmail, Wikipedia and the rest are still mostly in the cloud. That network hop again.
So AI can’t be inside the event loop, not if you want really great on-device interactions.
Instead – let the AI write device code. That’s the approach we’ve found.
Take that toy example of a roast chicken basting timer on the display of my stove. My AI agent should be able to dynamically write code for that interactive app, and have that app code executed on the stove itself.
I’m not saying that the AI needs to vibe-code firmware.
Firmware is the code that runs on device microcontrollers: you author it and compile it and you flash it and from that point on, it never changes. AI agents can vibe firmware (very happily). But I’m not sure I want to load code onto my stove that has boundless control over the heating element and the network stack and whatever other low-level capabilities are managed by the firmware itself. We’ve already had smart fridges sending spam email (BBC News, 2014), no more thanks.
Instead of firmware, the AI can run code in sandboxes.
Cloudflare made the case for sandboxes in March this year when they introduced dynamic workers:
We asked ourselves:
What if, when we allow an AI agent to spread its arms and stretch its legs in a room, it could inhabit devices by writing code that runs in an on-device sandbox, a sandbox that gives access to buttons and screens but not the network stack, and the app code could load and run instantly?
That’s Resident. It provides a code sandbox for ESP32 devices, and a toolchain for AI agents to write apps that target that sandbox.
At Inanimate, we use Resident for all our product prototyping.
And it will be at the heart of our future products.
Look, it may seem wildly disproportionate to write code to turn on a lamp.
But what are computers for? They do the hard work to make it easy for us. So this approach scales well from basic on/off control to… well, it turns out that, now we have this sandbox, we can compose all kinds of useful experiences that take over an entire room – and also weird and wonderful interactive ones that still work when you pull the network cable.
(An app arrives over the network but then the network is no longer required. Resident is built on our messaging library Courier which includes UDP multicast for local inter-device messaging even when internet connectivity drops.)
We’re opening Resident today as an alpha (v0.5.0) and open sourcing it under the highly permissive MIT license. Just include our copyright notice with any modifications.
Technically, we’re adding a Lua runtime to your ESP32 device. (Lua is a language designed to be embedded.)
We love Espressif’s ESP32 microcontroller family because it has a unique span in the ecosystem: it is used by individual makers, new hardware startups, and in production at real scale. It has built-in Wi-Fi and supports its native framework esp-idf and Arduino too, which is great for quick prototyping.
Resident gives you an API to add extensions to that Lua runtime: hardware drivers. Those managed capabilities are what makes it a sandbox. So the apps in the sandbox can respond to events that your button driver injects, and they can write to the display via a module added by your display driver.
The apps can be hot loaded at runtime. The way we have it wired up, you push app code down a websocket to the device and it run immediately in the sandbox.
Adding Resident is straightforward during development: bring up your new device as normal, then point your coding agent at
docs/start-building.md. It’ll walk you through adding the sandbox and writing the drivers.Resident comes bundled with:
resident.inanimate.techso you can easily push new apps and events – use the example code to build out your own back-end when you’re readyWant to try Resident now?
Ahead of developing your new device, pick up an M5StickS3. These are made by M5, I talked about them when we announced Courier.
The M5Stick is a dev kit that has an ESP32 with a screen, battery, couple of buttons, buzzer and IMU. You can bring it up using Arduino pretty simply. Then add Resident and start writing apps. We have an example project.
Want to get going even faster?
You don’t even need hardware…
The Try it now section of the Resident homepage has a M5Stick simulator, running the Resident sandbox in-browser.
Drag and drop an app onto the simulator to see it run.
Or even: install the Claude skills, tap the button on the webpage to make the simulator live, and create apps from your local Claude Code session. The in-browser simulator will update live and run your app.
Hey, deep cut reference alert:
Back in 2021 I went on a dive into files: Golems, smart objects, and the file metaphor.
What would it mean to drag and drop a file onto a lightbulb? I asked.
This is my answer haha
Resident is what we’ve been using to prototype products and use cases at Inanimate.
We believe that one day all products will work this way.
For more info, the GitHub, and to try it now: Resident.
For updates: subscribe to Lab Notes.