Fast (20+ tok/s)
Qwen3.5 0.8B 120+DeepSeek-R1 1.5B 120+Qwen 1.8B 120+Llama 3.2 1B 120+Qwen 3 1.7B 120+Granite 3.1 moe 1B 120+lfm 2.5 8B (A1B) 120+minicpm v4.6 1B 112Gemma 2 2B 112codeGemma 2B 112Gemma 2B 105smollm 2 1.7B 100starCoder 2 3B 100falcon 3 1B 100Qwen2.5 3B 94Qwen2.5 Coder 3B 94Qwen3.5 2B 94Qwen 3-VL 2B 94Llama 3.2 3B 90orca mini 3B 90Granite 3.1 moe 3B 90falcon 3 3B 90Granite 4.1 3B 85Granite 4.2 3B 82Phi 3 3.8B 82Qwen 4B 78Qwen 3 4B 69Qwen2.5-VL 3B 56Gemma 3 4B 54Qwen 3-VL 4B 54translateGemma 4B 54Qwen3.5 4B 53Qwen 3 30B (A3B) 52Qwen3 Coder 30B (A3B) 52Qwen 3-VL 30B (A3B) 50Llama 2 7B 47Code Llama 7B 47DeepSeek Coder 6.7B 47Llama 2 uncensored 7B 47Qwen3.5 35B (A3B) 47Qwen3.6 35B (A3B) 47orca mini 7B 44gpt-oss 20B 44Mistral 7B 41Qwen2 7B 41starCoder 2 7B 41OLMo 2 7B 40Nemotron 3.5 lightning 30B (A3B) 40falcon 3 7B 39DeepSeek-R1 7B 38Qwen2.5 7B 38Llama 3 8B 38Qwen2.5 Coder 7B 38LLaVA 7B 38Llama 3.1 8B 37Qwen 7B 37dolPhin 3 8B 37Gemma 7B 36codeGemma 7B 36Gemma 4 26B (A4B) 36DeepSeek-R1 8B 35Qwen 3 8B 35Granite 4.2 8B 34Granite 4.1 8B 34Gemma 2 9B 33minicpm v 8B 33LLaVA Llama3 8B 33ornith 9B 32Gemma 3n E2B 32gpt-oss 120B 31Qwen2.5-VL 7B 30minicpm v4.5 8B 29Qwen 3-VL 8B 29falcon 3 10B 28ornith 1.5 9B 27Qwen3.5 9B 27Mistral nemo 12B 25Gemma 4 E2B 25Llama 2 13B 24Code Llama 13B 24Gemma 3n E4B 24Gemma 4 12B 24Llama 3.2 Vision 11B 23Phi 3 14B 23orca mini 13B 23LLaVA 13B 22Gemma 3 12B 22translateGemma 12B 22Qwen 14B 22OLMo 2 13B 21DeepSeek Coder v2 16B 20
Comfortable (10-20 tok/s)
DeepSeek-R1 14B 20Qwen2.5 14B 20Qwen2.5 Coder 14B 20Phi 4 14B 20Qwen 3 14B 19Gemma 4 E4B 19starCoder 2 15B 18mixtral 8X7B 15Qwen3.5 122B (A10B) 15Mistral small 22B 14Mistral small 24B 13Mistral small3.2 24B 12Nemotron 3 super 120B (A12B) 11Gemma 2 27B 11Granite 4.1 30B 11Gemma 3 27B 11Qwen3.5 27B 11Qwen3.6 27B 11translateGemma 27B 11Granite 4.2 30B 10Qwen3.8 27B 10muse glimmer 30B 10Qwen 32B 10