benchr
What this guide covers
This guide covers the open-weight tier in 2026: the frontier-class open models (Llama 4, Mistral Large 3, DeepSeek-V4, Qwen3.6), the small-model tier (Phi-4 mini, Gemma 3, the new Phi variants), and the hardware question — what does it take to run any of these yourself, and when is that the right call instead of paying for an API.
The frontier-open tier
-
The open-weight tier right now: Llama 4, Mistral, Qwen, DeepSeek
Where open weights have caught up to closed models, and what's left of the gap after the April 2026 refresh. License clarity, code quality, multilingual range, and the cost-performance frontier.
The small-model tier
-
Small language models, in working use
Phi-4 mini, Gemma 3, and the workloads where sub-10B parameter models quietly win. 94% classification accuracy on a 1,200-email test set, at one-tenth the cost of the frontier.
Running them yourself
-
Running models on your own machine
Hardware, software, real tokens-per-second on three quantizations. When local is worth it versus paying for an API. 220 tok/s for Phi-4 mini on an M3 Max.
Which open model should you use?
For most production workloads where you need an open-weight model with a permissive license, Qwen3.6-27B is the default pick: Apache 2.0, broad multilingual range, 77.2% SWE-bench Verified on its official card, and it fits on a single GPU.
If you need the best open-weight math and code performance and your data-residency story can accept the MIT license terms, DeepSeek-V4 beats Qwen on those specific benchmarks and is the cheapest hosted endpoint in the field — V4-Flash starts at $0.14 per million input tokens.
For small-model workloads (classification, extraction, routing), start with Phi-4 mini (3.8B, MIT). It fits in 16GB of RAM, runs at 100+ tokens per second on a consumer laptop, and hits 94% accuracy on the email-classification test against a 96% Sonnet 4.6 baseline.
For the open-vs-closed cost question, see the AI costs guide. Since the April 2026 DeepSeek-V4 and Qwen3.6 releases, the open tier sits within a few benchmark points of closed flagships at under a tenth of the token price — the remaining gap is in long agent loops and extreme-scale retrieval, not the headline scores.