Model benchmarks
prism-ml/Ternary-Bonsai-2-27B-gguf local LLM performance
As of October 2026, prism-ml/Ternary-Bonsai-2-27B-gguf runs at up to 61.7 tok/s for local inference (best of 11 community benchmark runs across 2 GPUs).
Model size
27B
Peak speed
61.7 tok/s
Average speed
40.5 tok/s
Avg PP
649.2 tok/s
Min memory
9.9 GB
Max context
65,536 tokens
Avg output / run
52,751 tokens
Avg runtime / run
28m 22s
Avg quality
68.2
Benchmark runs
11
GPUs tested
2
Quality by task
Average LLM-judged quality (0–100) with the run-to-run spread shown as a P5–P95 band, overall and for each benchmark task, across all 11 runs. The low and high columns show how much the judge’s score varies between runs, and need at least two runs to display.
| Task | P5 (low) | Avg | P95 (high) |
|---|---|---|---|
| Overall | 45.6 | 68.2 | 83.6 |
| Agent Workflow | 73.9 | 83.5 | 91.4 |
| Code Generation | 0.0 | 39.7 | 81.0 |
| Role Play & Narrative | 68.6 | 84.1 | 90.7 |
| Research & Analysis | 8.0 | 65.4 | 89.6 |
Performance by hardware and tool
Every hardware/tool/quantization combination prism-ml/Ternary-Bonsai-2-27B-gguf has been benchmarked on, ranked by peak token generation speed. Last updated October 2026.
| Hardware | Tool | Quant | Best tok/s | Avg tok/s | Memory | Context | Quality | Runs |
|---|---|---|---|---|---|---|---|---|
| NVIDIA GeForce RTX 4070 | Unsloth Studio | Q4_K_M | 61.7 tok/s | 61.7 tok/s | n/a | 65,536 tokens | 60.5 | 1 |
| NVIDIA GeForce RTX 4070 | Unsloth Studio | — | 47.9 tok/s | 46.9 tok/s | 11.6 GB | 50,000 tokens | 67.7 | 3 |
| NVIDIA GeForce RTX 4070 | Unsloth Studio | TQ1_0 | 46.4 tok/s | 45.8 tok/s | n/a | 50,000 tokens | 69.2 | 3 |
| NVIDIA GeForce RTX 4070 | Unsloth Studio | PTQ1_0 | 43.5 tok/s | 43.5 tok/s | 9.9 GB | 65,536 tokens | 82.8 | 1 |
| NVIDIA GeForce RTX 4070 | Unsloth Studio | PQ2_0 | 42.9 tok/s | 42.9 tok/s | 10.9 GB | 65,536 tokens | 79.7 | 1 |
| Intel(R) Arc(TM) 140V GPU (16GB) | Unsloth Studio | PTQ1_0 | 9.9 tok/s | 9.9 tok/s | 16.9 GB | 16,384 tokens | 57.9 | 2 |
Benchmark runs
All 11 prism-ml/Ternary-Bonsai-2-27B-gguf runs submitted so far — expand one for its hardware, quality breakdown and per-scenario detail.
Frequently asked questions
- Is prism-ml/Ternary-Bonsai-2-27B-gguf good for coding?
- In our benchmarks, prism-ml/Ternary-Bonsai-2-27B-gguf scores 39.7/100 for coding. It runs at about 40.5 tok/s, so if you want more speed, ornith-1.5-35b-a3b-apex-i-quality is faster (~59.4 tok/s) and still scores well for coding (82.8/100).
- Is prism-ml/Ternary-Bonsai-2-27B-gguf good for agentic (tool-using) tasks?
- In our benchmarks, prism-ml/Ternary-Bonsai-2-27B-gguf scores 83.5/100 for agentic workflows. It runs at about 40.5 tok/s, so if you want more speed, swift-iq2_xs is faster (~91.5 tok/s) and still scores well for agentic workflows (91.4/100).
- How fast is prism-ml/Ternary-Bonsai-2-27B-gguf for local inference?
- Across 11 community benchmark runs, prism-ml/Ternary-Bonsai-2-27B-gguf reaches up to 61.7 tok/s and averages 40.5 tok/s, with the fastest results on NVIDIA GeForce RTX 4070.
- How much memory does prism-ml/Ternary-Bonsai-2-27B-gguf need?
- The leanest observed configuration used about 9.9 GB of memory (quantizations tested: PQ2_0, PTQ1_0, Q4_K_M, TQ1_0).
- Which tools have been used to run prism-ml/Ternary-Bonsai-2-27B-gguf?
- Benchmarks were submitted using Unsloth Studio. Results are community-contributed and updated as new runs arrive.