Guides

Benchmark-guided recommendations for local LLMs

Use these guides when you want a direct answer, not a table of numbers. Each guide translates benchmark data into a practical choice for coding, agent workflows, or consumer hardware.

Guide

KV cache quantization halves the cache — and on the 7900 XTX it made generation faster

At 65.5k context a 27B model wants nearly 6 GB for its KV cache, on top of 16 GB of weights. llama.cpp and other tools let you quantize it. Thirteen runs on one card at f16, q8_0, q5_0 and q4_0: the cache halves, generation gets faster, and no quality cost shows up at all.

  • At 65.5k context the KV cache is 5.9 GB — more than a third the size of the model's own 16.2 GB of weights. q4_0 halves it to 2.9 GB.
  • Measured VRAM fell from 24.8 GB to 22.1 GB across the sweep, so the saving lands on the card and not just in the estimate.
  • No measurable quality cost. Overall medians land at 86.1 (f16), 88.7 (q8_0), 87.4 (q5_0) and 87.6 (q4_0), and every range overlaps every other. The untouched default scores lowest of the four.
  • Generation is faster, and this one holds up: 40.4 tok/s at f16 against 51.7 at q8_0, with the f16 runs (38.8-42.4) not overlapping q8_0 (50.5-52.1) or q4_0 (50.2-54.9) at all.
Read guide

Guide

Qwen3.8-27B has the best coding ceiling you can run at home, and it ships on maximum effort

Qwen3.8's chat template sets reasoning effort to xhigh unless you change it. On one Mac that is the difference between an answer in eighty seconds and one in fourteen minutes. We ran all three levels to find out what the slow one buys.

  • Qwen3.8-27B posted the highest single coding score in our data from anything that fits on consumer hardware, a 90.7, and at IQ3/Q4 it needs just over 13 GB.
  • The default is the ceiling: the chat template resolves reasoning_effort to xhigh when nothing is set, so an untouched install runs every answer at maximum effort.
  • Low and medium are the same setting in practice, 4,984 tokens against 4,792 and eighty-four seconds against seventy-seven.
  • xhigh costs eight times the tokens and eleven times the wall clock for half a point of median coding score, which is well inside normal run-to-run variation.
  • A 12,000-token thinking budget halves the wait without truncating anything, and is the one lever we have tested directly.
Read guide

Guide

The Best-Quality Local LLM for Each Scenario

Which local model has the best output quality — not speed — in each of the four benchmark scenarios: agent workflow, code generation, roleplay, and research. Based on community benchmark data.

  • Best all-rounder: Qwen3.6-27B (MLX 4-bit) holds up across every scenario at roughly 20 tokens per second.
  • Winners differ by scenario: a 79.7B coder model writes the best roleplay, while one-shot code generation is the hardest scenario for every model.
  • All results come from consumer hardware — a 24GB Radeon RX 7900 XTX and Apple Silicon Macs — so bigger models on more powerful hardware could shift the standings.
Read guide

Guide

Best Local LLM for Apple M5 Max (64GB)

Find the best local LLM for a 64GB Apple M5 Max, with benchmark-backed guidance on why the MLX runtime, MoE models, and unified memory decide your real-world speed and quality.

  • On Apple Silicon the runtime matters more than the model size: MLX mixture-of-experts models reach 67-85 tok/s, while dense GGUF models on Ollama sit at roughly 5-20 tok/s on the same chip.
  • 64GB of unified memory is the real advantage — it lets a 79.7B coding MoE (qwen3-coder-next, ~51GB) run fully on the GPU, which a smaller Mac simply cannot hold.
  • The quality ceiling (~84) comes from the Qwen3.6 27B/35B family; choose based on whether you want raw speed or the very top scores.
Read guide

Guide

Best Local LLMs for Coding

Pick a local coding model based on quality, speed, and whether it fits your GPU without constant swapping.

  • Prioritize code quality and instruction following before raw token speed.
  • The ~20B parameter class, led by gpt-oss:20b, dominates the quality charts while retaining lightning-fast speeds.
  • Use the benchmark pages to compare the same model across different hardware and tools.
Read guide

Guide

Which Model to Use for Agent Workflows

Choose a model for agent workflows based on planning quality, robust tool usage, and enough context length to keep a long task coherent.

  • Robust tool usage and sufficient context length are critical for preventing agent hallucinations.
  • Agent planning and tool selection matter more than tiny speed differences.
  • These guidelines hold for any autonomous loop, regardless of which agent framework runs it.
Read guide

Guide

Consumer Hardware Performance Guide

Understand how VRAM tiers shape local LLM performance on consumer GPUs and why the best card depends on the models you want to run.

  • VRAM capacity usually sets the real ceiling before raw compute does.
  • Each VRAM tier opens or closes different model sizes and quantization choices.
  • The hardware page is the quickest way to compare practical GPU options.
Read guide

Guide

Best Local LLM for RX 7900 XTX

Find the best local LLM for RX 7900 XTX with benchmark-backed guidance on speed, output quality, context size, and VRAM usage.

  • 24GB VRAM makes 20B-class models like gpt-oss:20b the practical sweet spot on the RX 7900 XTX.
  • The gpt-oss:20b model achieves ~60-70+ tokens per second with very strong quality assessments.
  • Use the benchmark, hardware, and model browser pages to verify the recommendation you pick.
Read guide

Start with the data

These guides stay grounded in the same benchmark pages the rest of the site uses, so you can move from the recommendation to the raw results without leaving the app.

Benchmark resultsBrowse modelsHardware browser