After 600 experiments, here's what I discovered about JEPA world models
haddyalchaer.comMCTS also solves push-T problems without training: https://github.com/olivkoch/gumbel-mcts
Hands-on learning is an awesome thing - especially in the age of AI!
But maybe worth reframing away from “discovering” - hierarchical planning, sub-goals, action chunking etc. are well understood - this is more of a… transcript of your personal learning journey?
Which is still cool to see. But I read through expecting there to be some kind of novel results. We probably need to categorize these kinds of posts a bit better.
There's weird subtle moving rainbow color hues on this webpage on mobile
Let me know if you have any questions!
I knew close to nothing about JEPA before reading your post, but i was under the impression this was supposed to be a contender for LLM's limitations, a way to do things "properly".
However, based on this article, i have the feeling that on the contrary, JEPA is filling a super niche problem, and is pretty much only trying to improve robotic motions in space.
Which of the two ideas is it ? Can JEPA technics be applied to a much more wide range of problems ? (like LLM and transformers)
JEPA can be applied to text as well in the sense you can use it to build text embedding models, freeze your JEPA text embedder, then train a decoder over the top of that.
OF course your decoder model could end up doing a lot of the heavy lifting if your decoder loss was based on text reconstruction methods (which almost all LLMs use for eval purposes)
For certain tasks, you do need pixel/token level reconstruction fidelity to do a lot of things. Like image generation or text generation.
For other tasks, having a coherent representation of the world that can be computed quickly and intuitively is more salient. I suspect the human mind works more like JEPA than like an autoregressive LLM. Human children train on far less data to achieve extreme fidelity on a lot of tasks generative reconstruction loss based AI/ML techniques struggle with.
I wanted to know how you made these animations, they look gorgeous!
Seconding that, gorgeous animations
javascript and webgl magic ;)
there's no webgl on the page though afaict?
In a world where I'm increasingly being triggered by the fonts default Claude uses, I really liked the font used for the title here - it's Faculty Glyphic FYI.
AI; DR
If you’re going to use AI to generate text, keep it short and concise and meaty. I don’t hate reading AI, but I can’t read large walls of text for presumably some useful nugget of information in it.
https://www.pangram.com/history/2dd2b019-221a-4bee-bb6a-6b7e...
as someone with an oled monitor that background effect gave me a small heart attack
Website is borderline unusable on my phone. I suspect because of the animations? Either way not a great state of affairs
For anyone considering skipping because of this, it works flawlessly on my several year old iPhone.
There is a faint somewhat distracting background animation though I think…
Thanks! Glad to know it works for most people. I reduced the bg animation intensity.
Works perfectly fine for me on iOS 27.3 (with the exception of some glitching that was solved by disabling adaptive dark mode)
Sorry about that- will investigate. What OS/browser are you using?
Older intel macs initially have quite large blank areas of scroll while the figures later in the page render.
Much better after scrolling through and then reloading.
Looks perfectly fine for me