MiniCPM-V 4.7: OpenBMB Claims a 35B Vision MoE Runs at 13 Tokens a Second in 3 GB on an iPhone

Data graphic: a large 3 GB figure labelled as OpenBMB's vendor claim for MiniCPM-V 4.7 running at 13+ tokens per second on an iPhone

Alex Kim
Threat intelligence editor · Published Oct 14, 2026, 12:51 PM EDT

OpenBMB says its 35B-A3B vision MoE hits 13+ tok/s in about 3 GB on an iPhone 17 Pro Max. No quantisation is stated; the Q4_K_M GGUF is 21.2 GB.

OpenBMB has published MiniCPM-V 4.7 35B-A3B under the Apache-2.0 licence. The Hugging Face repository was created on 7 October 2026. It is a mixture-of-experts vision-language model with about 35 billion total parameters, the first MoE in the MiniCPM-V line. The headline claim is 13+ tokens per second in about 3 GB of memory on an iPhone 17 Pro Max.

That claim comes from OpenBMB's own model card and deployment document. Neither states the quantisation, context length or prompt behind the figure, and the demo clips are described as unedited screen recordings. Treat it as an unreproduced vendor number.

What the model is

The language backbone comes from Qwen3.5 and Qwen3.6, with a SigLIP2 vision encoder and mixed 4x/16x visual token compression. It handles single images, multiple images and video. The weights are BF16 safetensors, with FP8, int4, AWQ (W4A16), GPTQ (W4A16), BNB and GGUF variants alongside. OpenBMB lists iOS, Android and HarmonyOS as targets.

What the phone claim depends on

Three techniques are named, and none comes with a published speed breakdown.

Lookahead Expert Scheduling (LES). Cross-layer routing prediction prefetches experts the next layer may need. Predictions only steer loading. Computation still follows the model's real token-level routing, and any missed expert is loaded normally. This matters because a 35B model cannot sit in 3 GB, so the experts must stream from storage.
Fine-grained layer-wise quantisation of the expert weights, to cut storage and memory traffic. No bit-widths are given.
DSpark speculative decoding. A separate draft model, about 389M parameters over 5 layers, proposes 15 tokens per pass. The model card reports acceptance lengths (math 6.687, code 7.387, general 3.501) but no wall-clock speedup.

Data graphic: flow from 35B-A3B weights through layer-wise quantisation, LES expert prefetch and a 389M DSpark draft model to the claimed 13+ tokens per second

The pipeline OpenBMB credits for the phone result. The 13+ tok/s figure is the vendor's claim; no stage has a published speed breakdown.

LES links to OpenBMB's llama.cpp-omni fork. That repository's README does not mention V4.7, LES, DSpark or iPhone, so we could not confirm from it where the phone path lives.

The desktop numbers look different

The mainline GGUF route is far heavier than the phone claim. The GGUF repo (architecture qwen35moe) ships Q4_0 through F16: Q4_K_M is 21.2 GB, Q8_0 is 36.9 GB and F16 is 69.4 GB. Against that, the phone figure is about 3 GB. Nothing published shows the GGUF files being run with LES or the draft model, so the two sets of numbers describe different setups and should not be compared as like for like. A 789 MB DSpark F16 draft file sits beside the GGUFs.

What runs where

llama.cpp. OpenBMB's cookbook says mainline supports the model through PR #29416, with no release number.
vLLM. Listed as supported upstream via PR #58674. The DSpark instructions are vLLM-only, using method dspark with 15 speculative tokens.
SGLang. Only through the tc-mb/sglang fork, not upstream.
Transformers. Version 5.18.0 or later.
Ollama. The sources conflict and we could not resolve it. The cookbook says the model code is ready but the runtime awaits a llama.cpp bump. The Ollama library page lists 35b-a3b tags anyway: Q4_K_M at 22 GB, Q8_0 at 38 GB and F16 at 70 GB, with a 256K context window and no stated minimum Ollama version. Test it on your own install before relying on either page.

Benchmarks

No scores are published as text. The model card shows charts only, and claims that in Instruct mode the model scores higher overall than Qwen3.6 35B-A3B in less inference time. No numbers back that up in text.

What to do

Do not plan around 13 tok/s until someone reproduces it. On a desktop, the standard GGUF route needs roughly 22 GB for Q4_K_M. Check that the llama.cpp build you run includes PR #29416 before pulling the model, and confirm the Ollama version before relying on it.