Section 1
How quality is scored
After a model finishes a scenario, its output is graded from 0 to 100 by an LLM judge (currently openai/gpt-5.4-mini) against a rubric written for that scenario. Because every score comes from one judge model, treat them as a consistent relative signal between models rather than an absolute measure of quality.
Only models with at least three runs under the current scenario set are ranked, so no single lucky score can top a chart. That leaves 24 models across 178 runs, drawn from the 253 runs recorded since the scenarios were finalized on 13 April 2026.
- Agent Workflow — planning and tool use, weighted on tool selection and coherence (35 percent each).
- Code Generation — a complete browser game in one shot, with correctness weighted heaviest (40 percent).
- Role Play and Narrative — staying in character across an emotional scene, with immersion weighted highest (35 percent).
- Research and Analysis — reasoning over data, led by insight (35 percent) and depth (30 percent).
Section 2
The winner in each scenario
No model wins everything. The mid-size Qwen3.6 family dominates planning and analysis, a large mixture-of-experts coding model takes roleplay, and code generation stays stubbornly hard for all of them.
The roleplay result is the surprise: qwen3-coder-next, a model built for code, produces the most immersive and in-character narrative in the data. The same model scores far lower on research, a reminder that strength in one scenario does not carry to the next.
- Agent Workflow — Qwen3.6-27B (MLX 4-bit): quality 86.6 over 5 runs, about 20 tokens per second.
- Code Generation — qwen3.6:35b on Ollama: quality 77.0 over 3 runs, about 16 tokens per second, the highest score in the hardest scenario.
- Role Play and Narrative — qwen3-coder-next (79.7B) on Ollama: quality 92.9 over 4 runs, about 31 tokens per second.
- Research and Analysis — Qwen3.6-27B (MLX 4-bit): quality 83.8 over 5 runs, about 20 tokens per second.
Section 3
The best all-rounder
If you would rather run one model for everything, Qwen3.6-27B at 4-bit is the clearest pick: a combined score of 83.2 across the four scenarios, topping or tying the field in three of them and never dropping below 75.
If you want quality and speed together, the Qwen3.6-35B-A3B mixture-of-experts build is the better trade: it gives up only about three combined points while running roughly three times faster, because it activates only a few billion parameters per token.
- Qwen3.6-27B (MLX 4-bit) — combined 83.2 (agent 87, code 75, roleplay 88, research 84), about 20 tokens per second.
- qwen/qwen3.6-27b — combined 81.0, about 16 tokens per second.
- Qwen3.6-35B-A3B (MLX 8-bit) — combined 79.9 at about 69 tokens per second: nearly the same quality, far more speed.
- Qwen3.6-27B (MLX 8-bit) — combined 79.5, about 10 tokens per second.
- qwen3.6:35b — combined 78.9, about 16 tokens per second.
Section 4
Which scenarios are hardest
The same judge is far stricter on some scenarios than others, which is a useful signal about where local models genuinely hold up. Creative and planning work scores highest; one-shot code generation scores lowest by a wide margin.
The practical takeaway: if your use case is coding, weight the quality scores heavily and test on your own workloads before committing, because even the best local models here produce competent rather than flawless code in a single pass.
- Role Play and Narrative — mean quality 75.8, the most generously scored scenario.
- Agent Workflow — mean quality 73.0.
- Research and Analysis — mean quality 68.3.
- Code Generation — mean quality 56.4, the hardest scenario by a clear margin.
Section 5
The hardware behind the numbers
Every result here comes from consumer hardware — the machines contributors actually own. The most common GPU is a 24GB AMD Radeon RX 7900 XTX, followed by Apple Silicon Macs (M5 Max and M4 Pro, with up to 64GB of unified memory) and a handful of NVIDIA RTX 40 and 50-series cards.
That ceiling defines what got tested: almost every ranked model is between 3B and 36B parameters, the range that fits in about 24GB of VRAM or an Apple unified-memory budget. No datacenter GPUs and no large full-precision dense models appear in the data. Read these rankings as the best models you can run at home today — add bigger models on more powerful hardware and the standings would very likely shift, since larger models tend to win on quality, especially on code generation.
- Most common GPU: 24GB AMD Radeon RX 7900 XTX; also Apple M5 Max and M4 Pro (up to 64GB unified) and some RTX 40/50-series.
- Almost every ranked model is 3B to 36B parameters — what fits a 24GB GPU or an Apple unified-memory budget.
- No datacenter GPUs or large full-precision dense models are represented, so adding them could change the standings.