Edge0-145B parks its experts in host RAM to fit a 24 GB RTX 5090 D. The catch: 31 to 38 GiB of system RAM and a custom licence. All figures are vendor-reported.
Edge0 has put a 145-billion-parameter mixture-of-experts (MoE) model, Edge0-145B, on Hugging Face and says it runs on a single 24 GB consumer card, an RTX 5090 D, at 79.6 tokens a second in chat. The trick is expert offload: the GPU holds what fits and the experts sit in host RAM. Every figure in this article is vendor-reported, taken from Edge0's own model cards, and none has been independently reproduced. If you want to try it, the number to check first is not your VRAM but your free system RAM: the NVFP4 build used 38.27 GiB of it on top of the card.
What Edge0-145B is
The model has 145B parameters in total and activates 10.12B per token. Edge0's card lists 48 layers, 512 experts per layer (10 routed plus 1 shared per token) and a 256K context window.
It is built on Qwen/Qwen3.8-Flash-Next. Edge0 says it is not a re-quantisation of that model: the routed experts were replaced with a latent MoE and the result was post-trained. It ships in two builds, NVFP4 (47.79 GiB of weights) and a Q3_K GGUF (40.81 GiB). If the format names are new to you, our NVFP4 vs FP8 vs MXFP4 guide walks through the bytes-per-parameter math.
The memory you actually need
With expert residency on, Edge0 measured this on the RTX 5090 D 24 GB:
| Build | Weights | VRAM used | Host RAM used |
|---|---|---|---|
| NVFP4 | 47.79 GiB | 23.25 GiB | 38.27 GiB |
| Q3_K | 40.81 GiB | 23.14 GiB | 31.28 GiB |
So a 24 GB card is necessary but not sufficient. Plan for the card to be nearly full and for 31 to 38 GiB of system RAM to be taken by the model, plus whatever your OS and other apps need.
Edge0 also ran a tighter setup: 16 GiB of VRAM and a 20 GiB RAM cap (enforced with a cgroup MemoryMax=20G, no swap). On Q3_K it reports 36.8 tok/s chat decode there, against 18.5 for Qwen3.8-Flash-Next under the same budget. The card calls 16 GiB VRAM plus 20 GiB RAM test limits, not minimums, so do not read them as a supported floor. For machines short on RAM, a --resident-budget-gib 20 option (it requires --expert-profile) reads uncached experts from SSD instead.
Speed by machine
Edge0 reports two decode figures per machine: chat and agent. The headline 79.6 is chat decode on NVFP4; agent decode on the same card is 98.3. The faster number on the 5090 D comes from the smaller Q3_K build, not NVFP4.
Chat decode in tokens per second by machine and build, as reported on Edge0's model cards. The headline 79.6 is the NVFP4 build; the Q3_K build is faster on the same RTX 5090 D.
| Machine | Build | Chat decode (tok/s) | Agent decode (tok/s) |
|---|---|---|---|
| RTX 5090 D 24 GB | NVFP4 | 79.6 | 98.3 |
| RTX 5090 D 24 GB | Q3_K | 99.7 | 121.2 |
| RTX 4090 24 GB | Q3_K | 69.9 | 91.7 |
| RTX Spark N1X 60 GiB | NVFP4 | 40.1 | 46.0 |
| RTX Spark N1X 60 GiB | Q3_K | 37.7 | 36.7 |
| Apple M2 Ultra (Metal) | Q3_K | 60.9 | 57.0 |
Prefill on NVFP4 on the 5090 D is 1839.6 tok/s at 4K and 3165.7 at 16K. The M2 Ultra manages 755.6 and 800.3 on Q3_K. Apple Silicon gets the Q3_K build only: the card says NVFP4 is CUDA-only.
For how this compares with models that fit entirely in memory, see our picks for 16 GB cards and the Mac mini M4 and for one to four DGX Sparks.
Benchmarks: a claim to check
Edge0 reports 86.5 on Terminal-Bench 2.1 against 83.3 for Qwen3.8-Flash-Next, and 64.6 on DeepSWE 1.1 against 58.7. Both are self-reported on the model card, with no independent evaluation or methodology given. Treat them as a claim, not a ranking.
The licence catch
Edge0-145B is under the Edge0 Community Model License 1.0, a custom licence, not open source in the OSI sense. The Qwen materials stay under the Qwen Community License 1.0.
It allows personal use, research, modification, noncommercial sharing, internal use and commercial use of the model's outputs. Selling or bundling the model, or running it as a commercial model service, needs a separate written licence from Edge0. The framework code is Apache-2.0 and Strata is MIT.
What to do before you download
- Check free system RAM first: about 31 GiB for Q3_K or 38 GiB for NVFP4, on top of a 24 GB card.
- Use Edge0's own engine (edge0-infer with edge0-strata), on CUDA 13 under Linux or Windows, or Metal on macOS for Q3_K. It serves OpenAI- and Anthropic-compatible APIs on 127.0.0.1:8000.
- Ignore the Hugging Face sidebar metadata: it shows 5B params and :BF16 snippets that do not match the 145B model.
- Read the licence before you offer the model as a hosted service or bundle it in a product.
- Run your own agent tasks before trusting the Terminal-Bench figure.
Sources
- Edge0-145B-NVFP4 model card (Hugging Face): architecture, NVFP4 memory residency, throughput table, licence, base model, NVFP4 CUDA-only
- Edge0-145B-q3k model card (Hugging Face): Q3_K sizes and residency, test method, Apple, Spark and 4090 numbers, Terminal-Bench 2.1 and DeepSWE comparisons, flags, licence detail