Settings

Theme

Tabby v0.1.1: Metal inference and StarCoder supports

tabby.tabbyml.com

4 points by wsxiaoys 2 years ago · 1 comment

Reader

wsxiaoysOP 2 years ago

Previously, Tabby ran exclusively on CUDA devices, posing a significant barrier for developers looking to effectively utilize LLMs in their day-to-day coding.

The Tabby team made a significant contribution by enhancing support for the StarCoder series models (1B/3B/7B) in llama.cpp. This enhancement allows these models to run on Metal, providing comparable performance to that of an NVIDIA GPU.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection