MLC LLM: Universal LLM Deployment with GPU Acceleration
github.comSupported platforms include:
- Metal GPUs on iPhone and Intel/ARM MacBooks
- AMD and NVIDIA GPUs via Vulkan on Windows and Linux
- NVIDIA GPUs via CUDA on Windows and Linux
- WebGPU on browsers (through companion project WebLLM).