leon-se/gemma-3-27b-it-FP8-Dynamic
27B •
Updated
•
6.08k
•
20
Models that run well on RTX 5090
Note Runs well on vLLM, will not fit standard sglang. Very good model for the size.
dddsaty/phi-4-GPTQ-8bit
Text Generation •
15B •
Updated
•
1
•
1
Note Very fast with sglang, can improve even more with draft easiest-ai-shawn/Phi-4-EAGLE3-sharegpt-unfiltered
cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit
Text Generation •
5B •
Updated
•
137k
•
45
Note Very good at tool call and instruction following, prone to unexpected hallucinations.
gaunernst/gemma-3-27b-it-int4-awq
Image-Text-to-Text •
6B •
Updated
•
23.9k
•
39
Note Good balance of size and ease of running
google/gemma-3-12b-it
Image-Text-to-Text •
Updated
•
2.15M
•
•
687
Note On the smaller side, I recommend larger models