Free download. No account, no subscription. Apple Silicon, macOS 15 or later.
Train your own language model
One of the best ways to understand technology, especially when it seems like magic, is to build a small version yourself. Language Model Builder explains the foundations behind mainstream AI tools and lets you build models from scratch. Choose a tokenizer and training data, watch the loss fall during pre-training, teach the model to answer questions, and end up chatting with something that you made.
Interactive textbook
A short textbook with interactive playgrounds
I often find myself explaining how language models are built and trained to friends and family. I finally wrote the introduction I always wanted: It’s friendly to non-programmers, contains many interactive examples, and covers all the fundamentals without treating the reader like a child or a machine learning expert.
Interactive from the first page
Run the experiments yourself: build a tokenizer, move through embedding space, inspect attention, and watch a tiny model learn.
Go as deep as you like
No coding, machine learning, or math experience required. Where useful, I’ve added opportunities for those curious to dive a little deeper.
No artificial additives
The educational content is only as long as it needs to be, without generated filler content.
Native training workbench
MLX
Train a GPT on your Mac.
Run the entire model creation pipeline directly on your Mac, with hardware-accelerated MLX, without installing any additional tools. Every major step comes with beautifully crafted tools that you’ll love.
The whole training pipeline
Pre-train with a data set of your choice, sample the base model, run supervised fine-tuning and direct preference optimization, then chat with the model you made.
See the run as it happens
Follow loss, throughput, validation, saved samples, and checkpoint history without stitching together scripts. It’s really fun to watch your model become more and more capable.
Fully local & free
Resume from a saved checkpoint tomorrow. Your models, datasets, and training history stay on your Mac. No accounts, no cloud, no bill.
Contents
Language modelsnext-word prediction, n-grams, RNNs
Tokenizationhow text becomes numbers
Embeddingshow numbers acquire meaning
Attentionhow words take in their context
The transformerthe architecture inside every modern model
Training dataweb crawls, filtering, deduplication
Loss & gradient descenthow a model actually learns
Fine-tuningteaching by example (SFT), then by preference (DPO)
ToolsAccelerated training with Apple’s MLX
01 · Training
A training run you can watch.
Pre-training, supervised fine-tuning, direct preference optimization, and sampling — all in-process, on your Mac. Watch the loss curve fall, teach the model your taste and preferences, then compare different model settings.
02 · Data
A catalog of good data.
Curated datasets for both pre-training and fine-tuning, each explained and one click away: bigger corpora for pre-training; friendly chats, human-written instructions, and math problems for fine-tuning. Also, obviously, full support for your own data sets.
03 · Chat
Talk to your creation
If you’ve never had the pleasure, it’s hard to describe just how fun it is to talk to a model that you’ve made. Language Model Builder allows you to chat in a pretty environment that also allows deep inspection - like token probabilities and tokens not chosen, for instance.
What kind of model can I actually build with this?
Using the default settings, you’ll get a model that writes coherent, grammatical multi-paragraph text in as little as a day. On a MacBook Pro M5 Max you could train a GPT-2-small-class model (~100–150M parameters on a few billion tokens) in about a week. It might be obvious, but to avoid disappointment: you will not train a Claude Fable 5 or ChatGPT 5.6 Sol in your garage.
Can I stop training and come back later?
Yes. Training runs save checkpoints as they go, so you can stop at any point and resume from a saved checkpoint tomorrow — or keep an earlier checkpoint around to compare against.
Does anything leave my Mac?
No. Your models, datasets, and training history stay on your Mac. There is no cloud component and no account — the model you chat with at the end exists nowhere but your own machine.
Can I share the models I made?
Yes! Language Model Builder creates files in the standardized “safetensors” format. Your friends can either open it in an app of their choice or directly with Language Model Builder.
Why is this app free?
Because I like building things and this app was fun to build. I am fortunate enough that I don’t need to sell it. In any case, I hear there’s very little money to be made by building educational software anyway. If you want to repay me, just send me a quick thanks, it’ll make my day.
When will you release a version for Windows, Linux, or Intel Macs?
Language Model Builder is a native macOS app that heavily relies on Apple’s machine learning framework MLX, which requires an Apple Silicon chip. I do not expect this app to be very popular, but if it somehow does, I might make ports to other platforms.
