SIMD-accelerated computer vision on a $2 microcontroller
shraiwi.comBTW, I find it pretty amazing that you can buy complete ESP32-S3-Zero boards on AliExpress for $5.33. That's for a little board with the chip, a ceramic antenna, a USB connector and an LED.
ESP32-C3-Zero are $2.80 each.
These prices are absolutely ridiculous for what you get. Also, the ease of use of these boards is incredible. Just connect them via USB to your computer and off you go.
Anecdotal, but believable, warning for those in the US
https://www.reddit.com/r/GlobalEntry/comments/1x4c062/us_ord...
The original post and the following discussion at https://www.reddit.com/r/esp32/comments/1x3bmjh/ordering_esp... clarify that the devices ordered were likely counterfeit and one had a fake FCC ID, so it was illegal to buy them irrelevant to the new law and likely why they were seized.
So the conclusion is not necessarily "don't buy ESP", but "verify what you're actually buying".
Dumb question: How do you verify what you're buying?
What if they revoke the ability to import an FCC ID (like if it was for DJI) in the time between you ordered it and it arrived? It's not an entirely improbably scenario, and I'm guessing they won't care about the nuance. This administration is incredibly fickle and arbitrary.
The whole point of most of those esp32s being in that metal can module is that they're already tested and have FCC certs. That is, the intentional transmitter is already certified, making intgrators' lives a lot easier.
I bet there's either something the poster isn't telling us, or CBP made a bad call.
Likely the not so well trained CBP folks applying things incorrectly.
No, the discussion at https://www.reddit.com/r/esp32/comments/1x3bmjh/ordering_esp... clarifies that the FCC ID was faked and the product was counterfeit.
Or you can buy a three-pack of Xiao ESP32-C3s for $20 from Amazon in the US and not worry? Seeed also ships domestically, though more slowly. If I was trying to build and sell a product around them paying 3X as much would matter to me, but for the Home Assistant etc. screwing around I do with them, it's fine to pay the upcharge.
I am a bit skeptical. IT caught my attention because I recently ordered some parts, but then I recalled that all my existing gear from that supplier already has FCC certification. And indeed so does the base technology:
https://documentation.espressif.com/en/documentList?t=Certif...
It could be there's more to the story of what the person ordered, or it could be that CBP are misapplying the rules (quite likely under this administration), but it seems like some very overstated assumptions as written.
They are pretty hard to beat. I just bought some RP2040 for a project but am replacing them with ESP32-C3 for bluetooth support. 5 pack was like $20-30
what do you do with them?
Mine works around a bad design decision in my gate’s GSM intercom and stops endless missed deliveries.
The intercom does ring-around: a visitor presses the button, it calls the first number on its list (my mobile), and if nobody answers it tries the next (my wife's mobile). Except it counts voicemail as an answer. If the first phone goes to voicemail, the intercom decides the job is done and numbers two and three never ring. So the list is effectively one number long if my phone has no reception. I assume nobody tested it against a phone that was switched off or travelling on the London tube.
I gave up on the chain calling in a the first week of several failed delivery attempts. Now an ESP32 and a SIM7600 sit first on the list instead. The modem picks up after 3 seconds, sends a # DTMF tone (the intercom’s open command), hangs up, and texts me that someone came in.
Since this is a mass used car security gate it is a weak security control and more to hinder casual drivers parking where they are not supposed to - and now I get my packages too :P
Personally it made it very easy to build custom sensors/actuators for my Home Assistant setup. I have a folder with a bunch of esphome configs and just point Claude Code at it and tell it to add new entries for the stuff that I need, it's really good at that it can even write drivers for specific sensors autonomously. Granted you still need to do the wiring/soldering by hand (for now!)
Just around the home I have:
- a board that captures the audio output from my vinyl player and streams it on my LAN (so I can play my records on speakers across the house)
- one esp32 in each of my DIY speakers around the house driving the DAC+amplifier
- smart plant sensors that measure soil humidity, temperature, etc.
- a camera inside of a bird house, solar powered
- a 8x8 lidar ToF distance measurement board for detecting sleep phase
I'm curious about the sleep phase detection. What kind of data can you extract from the array beyond body movement? Do you find it better reflects how you feel than a wearable?
> smart plant sensors that measure soil humidity, temperature, etc.
Hah, that was one of my first projects long time ago before ESP32s even existed (PIC FTW!). Feels like this is one of those rights of passage projects for many people.
How do you deal with the audio latency / missed packets / etc?
I am currently working on a Bluetooth Mesh home network allowing me to control lights, gather sensor data from various places, etc (think interfacing with a home alarm system for presence detection).
Also, a home air quality monitor, both with a display and reporting to my mesh network.
Or you could build a "plane radar" like this one :-) https://github.com/ironicbadger/ESP32-Plane-Radar
Cheap MCU dev kits in general, for me?
A somewhat-elaborate fan speed controller for a wonderful old pro audio power amp that also needed a good bit of other rehab
A window fan controller, for summer use, that only turns on the fan when it's cooler outside than it is inside -- so it only cools the indoors, seeks never to heat it
A sensor for automating the color temperature of my indoor lighting based on outdoor conditions
A macro pad that ultimately works kind of like a Streamdeck, but without the built-in display features (that was a download-and-print project)
Other random stuff, like an ESPHome widget that talked to a window aircon via infrared commands (mostly just a download-and-run project)
They're just cheap little programmable low[ish]-power computers on a board, with some flash storage and RAM, and a fair bit of low-level IO. Often with wireless network connectivity as a built-in. They do stuff. :)
If the code is tight-enough, they can do a remarkable amount of that stuff -- in terms of raw compute, they present something near-ish to the Pentium era of personal computing for a few dollars.
For me, they make sense for simple projects. When something like a disused laptop or a Raspberry Pi-like device [and the accoutrements, like a whole-ass operating system] seems like overkill, then: It's probably overkill.
With a dev board, I can breadboard a circuit and maybe even leave it that way forever, or get a custom PCB drawn up with KiCad and made inexpensively by JLC PCB or Osh Park or whatever, or just solder wires and parts to the dev board like a caveman.
All methods work, and the cost of screwing it up is usually very low.
After that, the thing gets powered up with one of the featureless USB wall warts that we all have accumulated too many of and it just sits there and does its job.
(And honestly, these Pi Pico and random ESP32 dev boards are also overkill for what I do with them. Most of what I've done could be accomplished with an MCU chip that costs pennies instead of dollars, but my quantities are low-enough that I just don't care much about a few dollars for occasional one-off projects.)
Personally, I just finished up a project with one to poll outdoor and indoor weather sensors and notify the household over discord when it's warm enough outside to probably need air conditioning, when it's cooled off enough to probably not need AC, when the AC goes on, and when the AC goes off. It also parses and displays and notifies information from the National Weather Service when there are relevant alerts in my area.
a microphone that streams to birdnet-go in my home assistant with that firmware https://github.com/Sukecz/esp32-birdnet-mic
Similar class of chip, but I use the rp2350 to perform computer vision with LiDAR to detect when people leave dishes in a sink and beep angrily at them.
Wait until you get into developing your own PCBs and discover the BK7238 for $0.81 :)
Oh, I do design my own PCBs -- but the price and accessibility of these modules is hard to beat. Especially given that, if you're doing anything that has a radio, you probably want a pre-designed board or an integrated module rather than the chip itself. Baluns are not obvious to design on a board that you do not fully control.
Also worth noting that most (all?) ESP32s have dual cores.
I recently implemented a beam-forming microphone with LLM transcription (offloaded to a browser model)[1] and was able to get all the beamforming and noise cancellation DSP code running on the second core along with the UI, while the and webserver that streams audio uses core 0.
I don't use SIMD myself, but it uses esp-dsp which is already using it.
> For its price, the ESP32-S3 is a powerhouse of a microcontroller.
Indeed, don't forget it is more powerful than computers like this one,
That was my exact first computer with the CGA graphics, GEM Desktop, Locomotive Basic etc. All the software came in 4 360KB diskettes. The diskettes had each it's own beautiful color. There were 3 sample Basic programs that seemed to me quite advanced. One sample was a document management system. Another was a drawing program where you could draw with a mouse. Didn't know that GEM Desktop was made by Digital Reasearch. It was a very good windowing system. I believe it was based on MS-DOS. There was also a CP/M OS. So much software in only 1.2MB.
Exactly, thanks for sharing.
I dearly remember the keyboard's warm red Caps Lock light and my sessions of running Print Master and writing BASIC programs on it.
You can quite literally run Doom on it. This isn't new either; here's a video from 8 years ago demonstrating that, and the author mentions not being the first person to do this: https://www.youtube.com/watch?v=y6PP_IBbOTY
Newer models like the ESP32-P4 are substantially more powerful, running dual cores and with all sorts of computational acceleration built in.
We will see the day your toothbrush has more compute than the original IBM PC.
Pretty sure we have, e.g. [0], "TI's MSPM0Cx MCUs featuring Arm® Cortex®-M0+ core with 24 MHz CPU, 32-bit structure, 4 to 16 KB Flash provide cost-effective designs for electric toothbrush and shaver applications." - compared to the 5150[1], it's 4x the bus width, 5x the CPU clock, albeit with fewer RAMs.
That insane new Dyson toothbrush definitely. It has on-device AI analyzing 28 fps video from its camera.
At 8mhz? I'd imagine it rivals at least the Pentium III
Well, probably not, but maybe a plain Pentium?
The Pentium III was a superscalar core that scaled up to 1.4 GHz. This microcontroller is max 240 MHz and does less work per cycle.
Probably, I was use this example because I feel youngers can't imagine how much we could do with such MS-DOS class computer, even within its limitations.
So naturally an ESP32 can do so much more.
The CH32V407/CH32V467 is a bit more expensive, was $2.4 per unit two month ago, but supports the embedded subset of the RISC-V vector extension.
So you basically get a full AVX-512/SVE level SIMD ISA, minus FP support and with 64-bit SIMD width. But you still have LMUL, which will be great here to maximize utilization.
nit: 5.1MP/s to 11.2MP/s is not an increase of 220%, it's an increase of 120%. it's 2.2x faster, but percentage increase starts at 0%, not 100%. 100% faster is double, 200% faster is triple.
nit²: 2.2x as fast, or 2.2 times the speed; not 2.2x faster. same issue you pointed out.
using faster with x/times is just begging for misunderstandings
2.2x the speed is 1.2x faster.
This sounds interesting, Is there a way to port opencv code to this? I know there is a way to write opencv code to run on fpga but it would be nice if you can do it on low powered microcontrollers. Or did you write the algorithm from scratch by hand?
ESP32-S3 SIMDs are super difficult to use unfortunately
Practice makes perfection
true, but nothing is perfect
I find it easier than x86 SIMD[^1], but the lack of support for vector operations for 32 bits makes me very grumpy.
[^1] Though, to be honest, x86 has great automatic SIMD tools so I never had to try too hard.
Just LLM it
IME, they’re still pretty bad at getting specific implementation details of SIMD correct. The best I’ve done so far is to harness them to an actual piece of hardware and have them iterate until they see performance improvements but 1) Goodhart’s law, they tend to overfit the benchmark suite and 2) the resulting SIMD often becomes illegible.
For a lot of SIMD applications, overfitting isn't really a concern. In my use case I need to calculate the sum of absolute difference between two vectors for autocorrelation, there's no real possibility for overfitting, either it gets the correct answer or it doesn't. LLMs are good at reading through the CPU manual and trying different optimizations.
The resulting SIMD being illegible isn't an issue in my eyes, assembly isn't a language known for it's readability in general. If you want to really understand what the assembly is doing, ask your LLM to whip up a visualization and explanation tailored to your input data where you can see how each operation leads you to the final result.
Have you tried including the instructions documentation to the project?
Yes, of course. It doesn’t seem to recall the specific details very well, or confuses them with x86 or ARM instructions that do similar things. And yes, my AGENTS.md says ONLY USE INSTRUCTIONS IN blah.txt.
I dunno, clearly I need an LLM psychiatrist.
Haha, maybe try different agent/harness which would the closest thing to that.
Which model and at which effort?
The engineering seems impressive, but how useful is the result? What can you detect exactly with it in real time?
Detecting corners, but to what resolution/detail, a picture with a result would have been helpful to understand the capability.
edit: missed this link, this is what this is about
https://en.wikipedia.org/wiki/Features_from_accelerated_segm...
mitchellh's post about SIMD: https://mitchellh.com/writing/everyone-should-know-simd
Expecting lots of traffic to this blog post from russian and ukrainan ip-s.
Interesting. Since this was implemented on the S3, which uses the ISA XTensa LX instead of the RISC-V of most newer ESP32 series chips, I looked into how Espressif is handling SIMD on the latter ISA. Apparently it is only present in limited fashion as a processor extension on the P4 and S31 [0]. The S31 datasheet claims SIMD [1] and the P4 datasheet [2] files them as "Processor Instruction Extensions."
0. https://bitbanksoftware.blogspot.com/2026/04/esp32-p4-simd-e...
1. https://documentation.espressif.com/esp32-s31_datasheet_en.p...
2. https://documentation.espressif.com/esp32-p4_technical_refer...
It's all kinda sketchy with Espressif. They do not hesitate to ship hardware prior to publishing comprehensive documentation. The complete P4 TRM documentation for the PIE extension didn't arrive until June 2026, although it's been possible to buy P4 dev boards for roughly two years.
They also don't hesitate to signal "mass production" availability while they're still working out substantial errata. Espressif will ship the moral equivalent of "engineering sample" silicon (no, they don't call it that) for early use by their supported customers, and eventually iron out the bulk of the issues in later silicon.
Espressif applies the "release early, release often" model of open-source software, but for silicon, and I love them for it. But it does create pitfalls for early adopters that don't understand the game.
So the big hole in ESP32-S31 SIMD documentation (literally "Coming soon"[1]) isn't surprising, despite the fact that you can buy official and third party dev boards right now.
[1] https://esp32-s31.espressif.com/en/docs/9-Technical%20Refere...
Bits of it exist, not much about the CPUs yet - I get the impression they have something different from a normal RISC-V vector unit, and only on one of the two cores
https://documentation.espressif.com/esp32-s31_technical_refe...
OTOH it has a full MMU and will run linux
OTOOH it comes in a new WROOM package that's much harder to hand solder (not castelated so they can have lots more pins
Why is it still called SIMD-accelerated, if these are now just normal instructions. You just have to take a little bit more effort into restructuring your algorithms to make use of them.
The compiler does not generate SIMD instructions on this platform. So they're not "normal" instructions in that sense.
Or perhaps you want to argue that everything is running "normal" instructions. Be it a hardware video decoder or a GPU.
How does using instructions that operate on multiple data elements, which make your software 3 times faster, not fit the definition of SIMD-accelerated?
Because these are normal instructions.
If you do multiplication using mul instead of loop and add, is it acceleration too?
Yes.
The article would then be "MUL-accelerated multiplication". Obviously any modern compiler should not require a developer to make that optimization.
What does "normal instructions" mean? I don't see (or missed) a heading with "Normal" in the ESP32 docs.
Edit: Or are you saying the instruction called out by the author is not SIMD?
What does "normal" have to do with it? SIMD means single instruction, multiple data, which is how these instructions are designed.