Show HN: Agent Draw: An agent draws while you talk, built on TLDraw
techstackups.comWe recently built 2draw, a Drawful-style game where players draw on a shared canvas and race to guess each other's drawings, on tldraw, an infinite-canvas SDK for React.
We started wondering what it would take to put an agent in that loop, as an opponent or a rival guesser, and dug into how an agent could read and draw on a tldraw canvas. That research turned into:
Agent draw, a tool that lets an agent draw to the canvas for you while you present.
You can try it right now, or grab the source:
- Live demo: https://tldraw-agent-draw-demo.james-664.workers.dev
- Source: https://github.com/ritza-co/tldraw-agent-draw-demo
Here you can see the agent assisting me in my demo presentation of a third grade chemistry class:
- https://www.youtube.com/watch?v=iIH2hJAxxm8
This is sick! I built a similar tool at https://shimmerdiagrams.com
I really enjoyed reading your blog post, it was pretty informative about the techniques. I had built mine as a prototype and specifically interested in running all models local and in browser.
Tldraw seems to be a better medium for this that mermaid.
This is awesome! I love mermaid but have struggled to get my agents to be any good at it. Maybe I just need to context engineer a little better
A very interesting way of interacting with an AI agent. I tried out the demo for creating mind maps, and it was an interesting experience to see it grow and be able to explore different branches. An interesting concept for exploring AI-supported visual thinking.
Yes I would love to see some work on the design, because I think it could be useful, just needs some speed and accuracy improvements, and maybe some design towards specific use cases
Good idea! It could be fun to try creating an RSA-esque visual explanation video instead of using canvas selection (https://youtu.be/zDZFcDGpL4U).
That’s awesome! My inspiration was the IBM YouTube channel where they write on the board in front of them, maybe you could use the white canvas as like a green screen or something to get a similar effect?
Heh. I do something like this when I'm at work in meetings, basically dynamically trying to map out what people are saying so that I can build mental models.
Funny that soon the AI can define my own mental model on the fly soon.
It would be great if it could have examples of your own diagrams in its context, so it knows what points you like to map out and how you like to visualize them
This is a great idea! I see so many use cases, esepically in education. I could've really used something like back when I was tutoring in grad school.
The potential use cases was my favorite part of designing it, it’s not perfect now but it’s so fun to think what people might do with these kinds of designs in the future
I asked it to draw "two circles connected by an arc" and it crashed before drawing the arc :-(
Eish, probably needs some bug hunting. During testing I had some issues with less capable models, if you have your own key for a more capable model it should perform better.
Steve is a treasure <3
Facts!
Very cool. Takes some getting used to but its powerful
Yes it could get a lot better with some design improvements, maybe live streaming the transcription, and maybe after some trial and error testing the best + fastest model for the job