A stability-first SGLang deployment serves DeepSeek V4.1 Flash across three NVIDIA DGX Sparks: 750k KV cache, ~1600 tok/s prefill, ~38 tok/s single-stream decode.
DeepSeek V4.1 Flash now runs on a cluster of three NVIDIA DGX Spark units behind an OpenAI-compatible endpoint. The deployment repo ships conservative defaults chosen for stability rather than peak numbers, and the throughput figures below come from that default profile.
Default settings
The defaults are deliberately conservative so the cluster stays stable under sustained load:
- 500k context, 750k KV cache
- Prefill of roughly 1600 to 1700 tok/s at every context length
- Around 38 tok/s of prose decode in a single stream
- Around 79 tok/s of aggregate prose decode across 4 concurrent streams
How the cluster is built
The serving stack is SGLang with FlashInfer kernels. Expert weights are stored in MXFP4 and the dense weights in FP8, with DSpark speculative decoding enabled. The model runs tensor-parallel across three GB10 nodes, each with 121.7 GiB of unified memory, joined in a ConnectX-7 RoCE triangle. The Engram n-gram tables, about 189 GiB, are offloaded to local NVMe on every node, and the Engram cache is disabled by default because reuse measured close to zero. A separate profile supports a four-Spark TP4 layout.
Decode throughput by stream count
Aggregate decode climbs as streams are added, while per-stream speed falls. Measurements are with TP=3 and DSpark speculation on:
Aggregate decode throughput by concurrent stream count, TP=3 with DSpark speculation.
| Streams | Aggregate tok/s | Per stream |
|---|---|---|
| 1 | 37.9 | 37.9 (TTFT 248 ms) |
| 2 | 58.9 | 30.5 |
| 3 | 71.2 | 24.5 |
| 4 | 78.6 | 20.9 |
Long-context behaviour was verified with a single 208k-token prompt and with four concurrent 46k-token prompts.
Getting started
Bring the cluster up with the bundled launcher, in order:
./start.sh doctor # validate nodes and links
./start.sh build # build the image on every node
./start.sh share # export the checkpoint over NFS
./start.sh pack # cache Engram shards to local NVMe
./start.sh serve # start workers, then the head node
A full boot takes 12 to 13 minutes.
Get it here: MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks