leon-se/gemma-3-27b-it-FP8-Dynamic
27B • Updated •
29.7k •
20
Models that run well on RTX 5090
Note Runs well on vLLM, will not fit standard sglang. Very good model for the size.
dddsaty/phi-4-GPTQ-8bit
Text Generation • 15B • Updated •
3 •
1
Note Very fast with sglang, can improve even more with draft easiest-ai-shawn/Phi-4-EAGLE3-sharegpt-unfiltered
cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit
Text Generation • 5B • Updated •
96.2k •
35
Note Very good at tool call and instruction following, prone to unexpected hallucinations.
gaunernst/gemma-3-27b-it-int4-awq
Image-Text-to-Text • 6B • Updated •
25.9k •
37
Note Good balance of size and ease of running
google/gemma-3-12b-it
Image-Text-to-Text • 12B • Updated •
1.35M • •
643
Note On the smaller side, I recommend larger models