GitHub - ninjahawk/bongochat

Getting started

git clone https://github.com/ninjahawk/bongochat.git
cd bongochat
pip install -r requirements.txt
python download_weights.py     # ~320MB from the release
python bongochat_tui.py

That is the whole setup. It runs on CPU if you have no GPU — slower, but it works. Inference needs only torch, tiktoken and numpy: no Triton, no Rust toolchain, no CUDA-specific build.

Put the repo directory on your PATH to call it by name from any terminal — bongochat on Windows, the bongochat bash script on macOS and Linux.

bongochat                              interactive TUI
bongochat -p "why is the sky blue?"    one-shot, prints and exits
bongochat -t 0.9 -k 100                temperature / top-k
bongochat -m 512                       longer replies
bongochat -i base -p "Once upon a"     the raw pre-SFT base model

In the TUI: /help /clear /temp /topk /max /stats /retry /exit. Ctrl-C stops a reply mid-stream without killing the session.

Overview

bongochat is a complete language model — tokenizer, pretraining, supervised finetuning — trained end to end with nanochat on a single RTX 5070 (12GB) running Windows. Total cost: about two and a half hours of a desktop GPU and no cloud spend at all.

It writes fluent English and invents facts with total confidence. That is what a model this size does, and this README will not pretend otherwise.

Benchmarks

Every bongochat score is measured locally on the complete test set — no sampling, no caps, no cherry-picked subset. Frontier scores are published figures, cited and dated.

The raw column is misleading and you should ignore it

Four of these benchmarks are 4-way multiple choice, so guessing scores ~25% for free. Always answering "D" on MMLU scores 26.89% — bongochat's 30.10% beats a rock by 3.2 points. Normalized for guessing it reaches 6.8%, against ~92.8% for a frontier model. The honest gap is 13.6×, not the 3.1× the raw numbers suggest.

The generative benchmarks do not move between those two charts, because there was never anything to guess between. Those bars were always the truth:

• GSM8K 0.15% (2 of 1,319). It emits calculator tool calls but cannot carry a multi-step arithmetic chain. Asked 17 + 25, it answered 28.
• AIME 2025 0.00% (0 of 30).
• SWE-bench ≤0.60%. Only 3 of 500 outputs were even syntactically valid diffs. A patch that cannot apply cannot pass, so that bounds the true score at essentially zero.
• GPQA 14.65%, below the chance floor — because it emitted no answer at all 37.4% of the time and picked "A" for 88% of the answers it did give.

Base model, before finetuning: val bpb 0.9397, CORE 0.1034. GPT-2 (1.6B) scores 0.2565 CORE — so this reaches roughly 40% of GPT-2 with 8% of the parameters.

What it sounds like

Asked an AIME problem — find all integer bases b > 9 where 17 in base b divides 97 in base b — bongochat opened with:

"To find the sum of all integer bases b>9 for which 17ᵦ is a divisor of 97ᵦ, we can use the method of Lagrange multipliers."

It is a divisibility problem solvable in four lines of integer arithmetic. Lagrange multipliers are a continuous-optimization technique requiring derivatives — nothing is being optimized, and divisibility cannot be differentiated. The phrase "find the sum of all X for which Y" merely has the grammatical shape of a constrained optimization problem, so the model reached for the most authoritative-sounding method attached to that shape.

That is the model in one sentence. It has learned the register of expertise — the cadence, the vocabulary, the confident "we can use the method of" — with none of the machinery underneath.

Why the weights aren't in the repo

GitHub rejects any file over 100MB and the model is 320MB. Git LFS's free tier serves 1GB of bandwidth per month, so it would stop working after three clones. The weights are release assets instead — 2GB limit, unlimited bandwidth — and download_weights.py fetches them.

Files

bongochat_tui.py      the terminal UI (stdlib only)
bongochat / .cmd      launchers for unix / windows
download_weights.py   fetch weights from the release
nanochat/             vendored inference runtime (MIT, karpathy/nanochat)
tools/                scripts that regenerate every number and chart here
MODEL.md              full model card: training config, provenance, all metrics
media/                demo recording and benchmark charts
models/               weights land here after download (gitignored)

Every figure in this README is reproducible from the published weights — tools/README.md has the commands, the runtimes, and the method caveats.

Training your own

This repo ships inference only. To train, use karpathy/nanochat directly. Reproducing this model:

python -m scripts.tok_train
python -m scripts.base_train --depth=8 --window-pattern=L --device-batch-size=16
python -m scripts.chat_sft   --model-tag=d8 --device-batch-size=16

--window-pattern=L matters on any GPU without FlashAttention 3 kernels (including all RTX 50-series): SDPA has no sliding-window support and utilization collapses without it.

Running nanochat on Windows

Upstream is Linux-first and breaks in four places on Windows. The vendored nanochat/ here already includes these fixes; if you use upstream for training you will need them:

1. torch.compile fails — no Triton on Windows. Install triton-windows (needs MSVC build tools). Without it the Muon optimizer cannot run at all.
2. execution.py imports resource (Unix-only) and scrubs the subprocess env down to a Unix PATH; on Windows Python won't start without SYSTEMROOT.
3. engine.py uses signal.SIGALRM for the calculator timeout, which does not exist on Windows. Needs a thread-based fallback.
4. Peak-FLOPS table has no entry for most consumer cards, so MFU reports as 0%.

Limitations

• Not loadable in Ollama or llama.cpp. nanochat's architecture uses per-layer value embeddings, learned resid_lambdas/x0_lambdas, and a mid-layer residual "backout". llama.cpp has no graph for any of it, so there is no GGUF conversion path without implementing a new architecture in C++.
• 2048-token context. Long documents do not fit.
• English only, and it will confabulate freely about anything factual.
• -i base does not chat. The base checkpoint never saw conversation tokens; it continues text rather than answering.
• SWE-bench was not run through the official harness — no Docker, no test execution. The reported figure bounds the real score rather than measuring it.

Acknowledgements

Built on nanochat by Andrej Karpathy (MIT). All the hard parts — the architecture, the Muon optimizer, the training loop, the eval harness — are his. This repo is a trained checkpoint, a terminal UI, and the Windows fixes needed to get there. Frontier comparison figures via TokenCalculator, LM Council and PricePerToken, July 2026.

MIT licensed. Weights included. Do whatever you want with it.