leon-se/gemma-3-27b-it-FP8-Dynamic
27B • Updated •
1.4k •
20
Models that run well on RTX 5090
Note Runs well on vLLM, will not fit standard sglang. Very good model for the size.
dddsaty/phi-4-GPTQ-8bit
Text Generation • 15B • Updated •
169 •
1
Note Very fast with sglang, can improve even more with draft easiest-ai-shawn/Phi-4-EAGLE3-sharegpt-unfiltered
cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit
Text Generation • 5B • Updated •
506k •
26
Note Very good at tool call and instruction following, prone to unexpected hallucinations.
gaunernst/gemma-3-27b-it-int4-awq
Image-Text-to-Text • 6B • Updated •
145k •
36
Note Good balance of size and ease of running
google/gemma-3-12b-it
Image-Text-to-Text • 12B • Updated •
1.46M • •
588
Note On the smaller side, I recommend larger models