How fast is MiniCPM-V-4_7-35B-A3B-IQ4_NL for local inference?

Across 2 community benchmark runs, MiniCPM-V-4_7-35B-A3B-IQ4_NL reaches up to 33.7 tok/s and averages 33.4 tok/s, with the fastest results on NVIDIA GeForce RTX 3070.

How much memory does MiniCPM-V-4_7-35B-A3B-IQ4_NL need?

The leanest observed configuration used about 23.3 GB of memory.

Which tools have been used to run MiniCPM-V-4_7-35B-A3B-IQ4_NL?

Benchmarks were submitted using llama.cpp. Results are community-contributed and updated as new runs arrive.

Model benchmarks

MiniCPM-V-4_7-35B-A3B-IQ4_NL local LLM performance

As of October 2026, MiniCPM-V-4_7-35B-A3B-IQ4_NL runs at up to 33.7 tok/s for local inference (best of 2 community benchmark runs across 1 GPU).

llama.cpp

ShareReddit X

Model size

35B

Peak speed

33.7 tok/s

Average speed

33.4 tok/s

Avg PP

377.2 tok/s

Min memory

23.3 GB

Max context

128,000 tokens

Avg output / run

11,337 tokens

Avg runtime / run

5m 54s

Avg quality

n/a

Benchmark runs

GPUs tested

Quality by task

Average LLM-judged quality (0–100) with the run-to-run spread shown as a P5–P95 band, overall and for each benchmark task. The low and high columns show how much the judge’s score varies between runs, and need at least two runs to display.

Task	P5 (low)	Avg	P95 (high)
Overall	—	n/a	—
Agent Workflow	—	n/a	—
Code Generation	—	n/a	—
Role Play & Narrative	—	n/a	—
Research & Analysis	—	n/a	—

Performance by hardware and tool

Every hardware/tool/quantization combination MiniCPM-V-4_7-35B-A3B-IQ4_NL has been benchmarked on, ranked by peak token generation speed. Last updated October 2026.

Hardware	Tool	Quant	Best tok/s	Avg tok/s	Memory	Context	Quality	Runs
NVIDIA GeForce RTX 3070	llama.cpp	—	33.7 tok/s	33.4 tok/s	23.3 GB	128,000 tokens	n/a	2

Benchmark runs

All 2 MiniCPM-V-4_7-35B-A3B-IQ4_NL runs submitted so far — expand one for its hardware, quality breakdown and per-scenario detail.

Open run page →

Frequently asked questions

How fast is MiniCPM-V-4_7-35B-A3B-IQ4_NL for local inference?: Across 2 community benchmark runs, MiniCPM-V-4_7-35B-A3B-IQ4_NL reaches up to 33.7 tok/s and averages 33.4 tok/s, with the fastest results on NVIDIA GeForce RTX 3070.
How much memory does MiniCPM-V-4_7-35B-A3B-IQ4_NL need?: The leanest observed configuration used about 23.3 GB of memory.
Which tools have been used to run MiniCPM-V-4_7-35B-A3B-IQ4_NL?: Benchmarks were submitted using llama.cpp. Results are community-contributed and updated as new runs arrive.