Remember "Boot to Gecko?" Well, this is "Boot to Llama."
I'm sorry.
Llanux is a minimal Linux distribution that boots directly to llama.cpp. There's a built-in GGUF model downloader, a simple TUI, and all the local inference your CPU can handle. (CUDA support is a work in progress; for now, prepare yourself for heavily quantized intelligence at 300 baud.)
Included scripts will build an ISO that you can run in a VM. You could even go full hardcore and run this on the metal: wipe Windows, install Llanux, delete Facebook, etc. (Note: do not do this.)
Obviously, this is not a product. This is a silly experiment. But it's also an "AI OS" that was vibe coded in less than a day. Discuss.
--Stephen @ Telepath
Overview
- Build System: Buildroot
- Target: x86_64 (bootable ISO, VM-compatible)
- Acceleration: CPU-only (AVX2/AVX512 optimized, runtime detection); CUDA support is WIP
- Init: BusyBox init → auto-login → TUI launcher
- Default Model: TinyLlama 1.1B Q4_K_M (~700MB)
Quick Start
Build Locally (Linux only)
# 1. Clone with submodules git clone --recursive https://github.com/yourusername/llanux.git cd llanux # 2. Download a model (optional - can download in VM later) ./scripts/download-model.sh tinyllama # 3. Setup and build ./scripts/build.sh setup ./scripts/build.sh build # 4. Test in QEMU ./scripts/run-qemu.sh
Architecture
┌─────────────────────────────────────────┐
│ Llanux ISO │
├─────────────────────────────────────────┤
│ Linux Kernel (minimal config) │
│ BusyBox (shell, coreutils) │
│ llama.cpp (llama-cli + llama-server) │
│ TinyLlama model (Q4_K_M) │
│ TUI launcher │
└─────────────────────────────────────────┘
Project Structure
llanux/
├── .github/workflows/ # GitHub Actions CI/CD
│ └── build.yml # Automated ISO builds
├── buildroot/ # Buildroot (git submodule)
├── board/llanux/
│ ├── rootfs_overlay/ # Files copied to root filesystem
│ │ ├── etc/inittab # Auto-login configuration
│ │ ├── usr/bin/
│ │ │ ├── llanux-login
│ │ │ └── llanux-tui # Main TUI interface
│ │ └── models/ # Bundled GGUF models
│ ├── post_build.sh # Post-build script
│ └── isolinux.cfg # Boot menu
├── configs/
│ └── llanux_defconfig # Buildroot configuration
├── scripts/
│ ├── build.sh # Main build script
│ ├── download-model.sh # Model download helper
│ └── run-qemu.sh # QEMU testing
├── external.desc # Buildroot external tree descriptor
├── external.mk # External packages makefile
├── Config.in # External packages config
└── README.md
Build Requirements
Buildroot requires a Linux host.
On Linux, install:
# Debian/Ubuntu sudo apt install build-essential patch wget cpio rsync bc file # Fedora sudo dnf install @development-tools wget cpio rsync bc file
Build Commands
./scripts/build.sh setup # Configure Buildroot ./scripts/build.sh build # Build the ISO ./scripts/build.sh clean # Clean build artifacts ./scripts/build.sh menuconfig # Open Buildroot menuconfig ./scripts/build.sh rebuild # Clean and rebuild
Testing in QEMU
# Basic graphical boot ./scripts/run-qemu.sh # Serial console (no GUI) ./scripts/run-qemu.sh --serial # With more resources ./scripts/run-qemu.sh -m 8G -c 8 # With KVM acceleration (Linux only) ./scripts/run-qemu.sh --kvm -m 8G # Mount host models directory ./scripts/run-qemu.sh --model ~/models/
QEMU Requirements
# Debian/Ubuntu sudo apt install qemu-system-x86 # Fedora sudo dnf install qemu-system-x86
Available Models
Download models to bundle with the ISO:
./scripts/download-model.sh tinyllama # 669 MB - Fast, good for testing ./scripts/download-model.sh phi2 # 1.6 GB - Small but capable ./scripts/download-model.sh mistral # 4.1 GB - Great balance ./scripts/download-model.sh llama2 # 3.8 GB - Meta's foundation ./scripts/download-model.sh qwen2.5 # 4.7 GB - Strong multilingual # List downloaded models ./scripts/download-model.sh list # Custom URL ./scripts/download-model.sh https://huggingface.co/.../model.gguf
TUI Features
When Llanux boots, you'll see a menu with:
- Interactive Chat - Chat directly with the LLM
- Start API Server - OpenAI-compatible API on port 8080
- Select Model - Choose from available models
- Download Model - Download models from Hugging Face
- System Shell - Drop to bash for debugging
- Shutdown/Reboot
API Server
The API server is compatible with OpenAI's API format:
# From host machine (port forwarded in QEMU) curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "messages": [{"role": "user", "content": "Hello!"}], "temperature": 0.7 }'
Implementation Status
- Project structure
- Buildroot external tree
- llama.cpp integration (Buildroot built-in)
- Boot configuration (isolinux)
- Auto-login to TUI
- TUI launcher script
- Build scripts
- QEMU test script
- Model download script
- First successful build
- Tested in QEMU
- Tested on bare metal
Future Enhancements
- NVIDIA CUDA support
- Vulkan GPU acceleration
- ARM64 builds (Raspberry Pi 5, Apple Silicon VMs)
- Persistent storage for downloaded models
- Web UI via llama-server
- Multiple ISO variants (minimal/full)
- Pre-configured system prompts
Troubleshooting
Build fails with missing dependencies
Run ./scripts/build.sh setup and check the error output. Install missing packages.
QEMU shows black screen
Try ./scripts/run-qemu.sh --serial for text-mode console.
Model not found after boot
Models must be in board/llanux/rootfs_overlay/models/ before building, or downloaded after boot using the TUI menu.
Slow inference
This build is CPU-only. For better performance:
- Use a quantized model (Q4_K_M recommended)
- Allocate more memory to QEMU
- Use KVM acceleration on Linux
License
MIT