Benchmark result

NVIDIA-Nemotron-3.5-Lightning-30B-A3B-8bit on Apple M5 Max — 110.1 tok/s

Measured with oMLX 0.5.7 on August 12, 2026.

ShareRedditX

LLM / Model

Model Size
30B
Memory Kind
model
Tool
oMLX v0.5.7
Scenarios

LLM Quality Assessment

Agent WorkflowopenclawCOMPLETED
Overall Quality Score
90.40
coherence
92
decomposition
95
error_handling
84
tool_selection
88

The plan is logically structured, uses appropriate web search tools for sourcing benchmarks, and cleanly decomposes the task into exactly four steps. It also identifies a non-essential tool and a fallback strategy, though the error handling could be stronger by specifying concrete backup sources and how to resolve missing speed/accuracy metrics.

Code Generationcoding_agentCOMPLETED▶ Play Game
Overall Quality Score
66.00
correctness
62
performance
72
code_quality
68
completeness
64

A mostly functional Breakout game is produced, but it has several implementation issues around input coordinate mapping, restart loop handling, and collision robustness. It meets many core requirements, yet the output format and responsiveness are weaker than requested.

Role Play & NarrativeroleplayCOMPLETED
Overall Quality Score
89.65
dialogue
87
immersion
89
consistency
92
narrative_arc
90

Aldwyn is portrayed convincingly as guarded, haunted, and slowly humanized by credible proof. The sensory detail and emotional beats are strong, and the character arc is coherent, though the meta-thinking structure makes it feel less like pure in-character narrative.

Research & AnalysisresearchCOMPLETED
Overall Quality Score
79.20
depth
82
clarity
88
insight
78
interpretation
74

Strong structure and clear score-based recommendations with generally correct arithmetic and useful comparative observations. However, the scaling characterization and 4T extrapolation are only loosely justified, and the analysis lacks formal modeling and explicit uncertainty treatment.

GPU Acceleration

GPU Model
Apple M5 Max
VRAM
64 GB

Performance Metrics

Generation Speed
110.1
tokens/sec
TTFT
421
milliseconds
Memory Usage
31.70
GB
Memory (Raw)
32457
MB

Context Information

Tokens Used
19090
Context Window
65,536

CPU & Memory

CPU Model
Apple M5 Max
CPU Threads
18
Physical Cores
18
CPU Frequency
N/A
Total RAM
64 GB

Operating System

OS Name
macos
OS Version
26.5.2
Kernel Version
25.5.0

Submission Details

Submitted:
8/12/2026, 7:41:39 AM
Client:
v0.4.36+97
Benchmark Result ID:
cmsps6gj1000701nvhrax204m