Fast (20+ tok/s)

Qwen3.5 0.8B 120+DeepSeek-R1 1.5B 120+Qwen 1.8B 120+Llama 3.2 1B 120+Qwen 3 1.7B 120+Granite 3.1 moe 1B 120+minicpm v4.6 1B 120+Gemma 2 2B 120+codeGemma 2B 120+Gemma 2B 120+smollm 2 1.7B 120+starCoder 2 3B 120+falcon 3 1B 120+Qwen2.5 3B 120+Qwen2.5 Coder 3B 120+Qwen3.5 2B 120+Qwen 3-VL 2B 120+Llama 3.2 3B 120+orca mini 3B 120+Granite 3.1 moe 3B 120+falcon 3 3B 120+Granite 4.1 3B 120+Granite 4.2 3B 120+Phi 3 3.8B 120+Qwen 4B 120+Qwen 3 4B 120+Qwen2.5-VL 3B 120+Gemma 3 4B 120+Qwen 3-VL 4B 120+translateGemma 4B 120+Qwen3.5 4B 120+Llama 2 7B 120+Code Llama 7B 120+DeepSeek Coder 6.7B 120+Llama 2 uncensored 7B 120+orca mini 7B 120+Mistral 7B 120+Qwen2 7B 120+starCoder 2 7B 120+OLMo 2 7B 120+falcon 3 7B 120+DeepSeek-R1 7B 120+Qwen2.5 7B 120+Llama 3 8B 120+Qwen2.5 Coder 7B 120+LLaVA 7B 120+Llama 3.1 8B 120+Qwen 7B 120+dolPhin 3 8B 120+Gemma 7B 120+codeGemma 7B 120+lfm 2.5 8B (A1B) 120+DeepSeek-R1 8B 120+Qwen 3 8B 120+Granite 4.2 8B 120+Granite 4.1 8B 120+Gemma 2 9B 120+minicpm v 8B 120+LLaVA Llama3 8B 120+ornith 9B 120+Gemma 3n E2B 120+Qwen2.5-VL 7B 120+minicpm v4.5 8B 120+Qwen 3-VL 8B 120+falcon 3 10B 120+ornith 1.5 9B 120+Qwen3.5 9B 120+Mistral nemo 12B 120+Gemma 4 E2B 120+Llama 2 13B 120+Code Llama 13B 120+Gemma 3n E4B 120+Gemma 4 12B 120+Llama 3.2 Vision 11B 120+Phi 3 14B 120+orca mini 13B 120+LLaVA 13B 120+Gemma 3 12B 120+translateGemma 12B 120+gpt-oss 20B 120+Qwen 3 30B (A3B) 120+Gemma 4 26B (A4B) 120+Qwen3 Coder 30B (A3B) 120+Qwen 3-VL 30B (A3B) 120+Qwen3.5 35B (A3B) 120+Qwen3.6 35B (A3B) 120+Nemotron 3.5 lightning 30B (A3B) 120+gpt-oss 120B 120+Qwen 14B 120OLMo 2 13B 117DeepSeek Coder v2 16B 111DeepSeek-R1 14B 109Qwen2.5 14B 109Qwen2.5 Coder 14B 109Phi 4 14B 108Qwen 3 14B 106Gemma 4 E4B 102starCoder 2 15B 99mixtral 8X7B 82Mistral small 22B 76Mistral small 24B 70Mistral small3.2 24B 66Gemma 2 27B 61Granite 4.1 30B 58Gemma 3 27B 58Qwen3.5 27B 58Qwen3.6 27B 58translateGemma 27B 58Granite 4.2 30B 55Qwen3.8 27B 55muse glimmer 30B 55Qwen 32B 55Code Llama 34B 52DeepSeek Coder 33B 52DeepSeek-R1 32B 49Qwen2.5 32B 49Qwen 3 32B 49Gemma 4 31B 49Qwen2.5 Coder 32B 49LLaVA 34B 49qwq 32B 49Qwen3.5 122B (A10B) 48ornith 35B 47Qwen 3-VL 32B 47Qwen2.5-VL 32B 47ornith 1.5 35B 43Nemotron 3 super 120B (A12B) 38Nemotron 3 33B 35Llama 2 70B 25Code Llama 70B 25orca mini 70B 25Llama 2 uncensored 70B 25Llama 3 70B 25Qwen 72B 24Qwen2 72B 24Llama 3.1 70B 23DeepSeek-R1 70B 23Llama 3.3 70B 23Qwen2.5 72B 21Qwen2.5-VL 72B 20

Comfortable (10-20 tok/s)

Llama 3.2 Vision 90B 18Qwen 110B 16

Slow but usable (5-10 tok/s)

Mistral medium 3.5 128B 7.4mixtral 8X22B 7.4

Won't fit

Qwen3.8 flash next 125B (A6B)