Show HN: PicoLM v1.0-rc2 ("Yura Kana"). Run an LLM on Digital Unix
github.comPicoLM is an LLM inference engine written in C99. It currently supports llama-2, GPT-2, Qwen 3.6/3.8(+MoE) and Gemma-3n models. Significant amount of work went into CPU SIMD acceleration/testing/correctness, and wide cross-platform availability with constant testing to never lose portability (from DOS through OS/X 10.4 to modernity). CUDA/HIP is supported, and accelerated IMMA kernels are available. More work needs to be done on prompt processing speed, but text generation is quite fast already.
With v1.0-rc2, the list of tested and supported platforms became even more obscene (iPhone 1! Tru64!), and some amount of work has gone into having a Vulkan backend.
No comments yet.