Benchmark result

Muse-Glimmer-30B-4bit on Apple M5 Max — 30.7 tok/s

Measured with oMLX 0.6.0.dev1 on August 13, 2026.

ShareRedditX

LLM / Model

Model Size
30B
Memory Kind
model
Tool
oMLX v0.6.0.dev1
Scenarios

LLM Quality Assessment

Agent WorkflowopenclawCOMPLETED
Overall Quality Score
87.90
coherence
88
decomposition
86
error_handling
84
tool_selection
90

The plan is logically structured and stays within the 4-step constraint, with appropriate use of search_web and execute_code and a sensible backup strategy. It correctly excludes send_email, though step 4 is somewhat redundant and could be more explicitly tied to the final report assembly. Overall, a strong and coherent plan with good tool choices and reasonable failure mitigation.

Code Generationcoding_agentCOMPLETED▶ Play Game
Overall Quality Score
67.00
correctness
62
performance
74
code_quality
68
completeness
70

A mostly functional single-file Breakout game with canvas rendering, scoring, lives, restart, and basic mouse/touch/keyboard support, but it misses several spec details and has some collision and layout inaccuracies that reduce fidelity and robustness.

Role Play & NarrativeroleplayCOMPLETED
Overall Quality Score
87.60
dialogue
84
immersion
90
consistency
88
narrative_arc
86

A vivid, emotionally credible response that captures Aldwyn's guarded grief and mercenary caution well. The atmosphere, bodily reaction, and final commitment are especially effective, though the trust-building could be stretched slightly to better satisfy the required pacing.

Research & AnalysisresearchCOMPLETED
Overall Quality Score
79.65
depth
84
clarity
88
insight
79
interpretation
72

Strong, well-structured response with clear task-specific comparisons and practical recommendations. However, the scaling claims are somewhat overstated relative to the sparse data, and the extrapolation is weak because it relies on an unvalidated log assumption and a ceiling effect. The recommendations are actionable, but the methodological grounding is limited by unknown benchmark variance and uncertain parameter estimates.

GPU Acceleration

GPU Model
Apple M5 Max
VRAM
64 GB

Performance Metrics

Generation Speed
30.7
tokens/sec
TTFT
1167
milliseconds
Memory Usage
18.83
GB
Memory (Raw)
19286
MB

Context Information

Max context used
5990
Context Window
65,536

Throughput

Total Runtime
7m 49s

CPU & Memory

CPU Model
Apple M5 Max
CPU Threads
18
Physical Cores
18
CPU Frequency
N/A
Total RAM
64 GB

Operating System

OS Name
macos
OS Version
26.5.2
Kernel Version
25.5.0

Submission Details

Submitted:
8/13/2026, 5:00:27 PM
Client:
v0.4.36+97
Benchmark Result ID:
cmsrrkwzk000001l8wcdl90te