Recursive Flint on Local Hardware: Which Quant Fits 16, 24 and 128 GB

Recursive released Flint on 21 October 2026. It is an Apache 2.0 coding model with 39.8B total parameters and about 2.5B active per token, and it shipped with llama.cpp support merged the same day and an Ollama build (0.40.8) that day as well. The file sizes are small enough for a 24 GB machine, but two tool-call bugs are open and the Ollama build is macOS-only for now. The headline benchmark numbers come from the vendor alone.

What Flint is

Flint is a decoder-only mixture-of-experts model: 49 layers (2 dense, 47 MoE), 256 routed experts per MoE layer with 6 selected per token plus one shared expert. The model card lists a 262,144-token context shared between prompt and output. The announcement post says up to 512k with 256k recommended, so treat 262,144 as the documented figure.

Only 14 of the 49 layers use full attention. The other 35 use a 513-token sliding window, with 4 KV heads of dimension 128. That matters for sizing, because the KV cache grows only on the full-attention layers. By our own arithmetic from the card (not a vendor figure), an f16 cache costs about 28 KB per token: roughly 1 GB at 32k tokens, 4 GB at 128k and 7.5 GB at the full 262k. That assumes llama.cpp uses its sliding-window cache, which is why Recursive's NVFP4 instructions say to leave --swa-full unset.

Which quant fits which machine

Sizes below are file sizes from the ggml-org and Unsloth repositories. Add the KV cache above plus operating-system headroom. Unsloth's own guidance is 13-15 GB for 2-bit, 16-20 GB for 3-bit, 26 GB for 4-bit, 38 GB for 6-bit and 44 GB for 8-bit.

16 GB. Nothing in the 4-bit class fits. The realistic choices are the 2-bit files (UD-IQ2_XXS at 11.9 GB or UD-Q2_K_XL at 13.6 GB), and UD-IQ3_XXS at 14.5 GB only with a very short context and little else running. This is our inference from the file sizes; none of the sources benchmarks Flint on a 16 GB machine, and no source reports how much quality the 1- and 2-bit files lose. Treat them as a way to try the model, not as a daily coding setup; our 16 GB VRAM guide has dense models that fit that class more comfortably.

24 GB. UD-Q3_K_XL (18.5 GB) is the quant Unsloth says fits 24 GB setups at short contexts. Q4_K_M at 23.4 GB and the 22.9 GB NVFP4 file leave almost nothing for the cache and the OS, so on a 24 GB Mac they are unlikely to be usable; Unsloth's guide lists 26 GB for 4-bit. Again, this is sizing arithmetic rather than measured results.

128 GB. Everything fits, which makes Flint an easy addition to the models we sized for a DGX Spark. Q8_0 (about 42 GB) or UD-Q6_K_XL (36.3 GB) leave room for a long context and other tools. Even BF16 (79.6 GB) plus a full 262k f16 cache (about 7.5 GB by our estimate) fits. Because only about 2.5B parameters are active per token, the large machine is a quality and context choice, not a speed one.

The 48 GB case is worth a note: Recursive recommends at least 48 GB of GPU or unified memory for its 22.9 GB NVFP4 file, which it tested on B300, RTX PRO 6000 Blackwell, RTX Spark and DGX Spark. The page does not say Blackwell is required.

Runtime status, as of 23 October

Runtime status as of 23 October. Sources: llama.cpp and Ollama on GitHub.

• llama.cpp. PR #30928 (commit a37cc46) merged on 21 October with conversion, tokenizer and attention support. It reuses an activation-scaling workaround that keeps Q8 activation sums within F16 range. The author reports a 10-30% slowdown from it and says follow-up PRs will address it. Most validation used existing GGUF files rather than freshly converted ones.
• Ollama. v0.40.8 supports Flint on macOS (ollama run flint, a 23 GB download). The notes say other GPU support "will be enabled in a subsequent release", so Windows and Linux GPU users should use llama.cpp for now.
• Open tool-call bugs. Flint is pitched at agentic coding, and both open issues hit that path.
  • llama.cpp #31000 (opened 22 October): on the NVFP4 GGUF with llama.cpp b11823 and OpenCode 2.1.2, some turns end with literal <tool_call> XML inside the reasoning and a stop finish, so the tool never runs. The reporter counted 8 such messages in one session, cannot reproduce it deterministically, and used a customized system prompt. No maintainer reply or fix is linked.
  • Ollama #19191 (opened 21 October): integer tool-call arguments in history are re-rendered as floats (30000 becomes "30000.0"), so schema-validating clients reject later calls. The reporter hit it in 3 of 3 runs with a prior tool call on Ollama 0.40.8 with the flint:40b-a2b-qat-int4 build on macOS, and it blocked every shell command for the rest of one agent session. A fix, PR #19195, is open.

Both bugs involve different quants and runtimes, so a fix in one place will not cover the other. Recursive's model card and Unsloth's guide both say to keep reasoning_content in the conversation history, which is worth checking in any custom client.

How far to trust the benchmarks

The model card reports Terminal-Bench 2.1 at 54.9%, SWE-bench Verified at 74.2% and SWE-bench Pro v2 at 89.7%. All are Recursive's own runs: mini-swe-agent for SWE-bench, Terminus 2 for Terminal-Bench, average pass@1 over at least five attempts, temperature 1.0. We found no independent reproduction.

The Pro v2 number is the oddity. It is 15.5 points above Verified from the same harness, and 89.7% would sit just below the 89.88 listed for Inkling in tenth place on a third-party aggregator's copy of the Scale leaderboard, with no Flint entry on that page. Reading the gap:

• SWE-bench Pro v2 is a September 2026 revision of a public set (642 tasks, 11 codebases, 89 invalid questions removed). The aggregator warns that public-set scores may reflect training-data exposure, so a high score does not show generalization to unseen code. We cannot say whether that applies to Flint.
• Verified has its own problems. An OpenAI audit reported on 8 July found 59.4% of the 138 hardest problems it examined had material test-design or description issues, and OpenAI stopped reporting Verified scores.
• A May audit by Datacurve, reported secondhand and unconfirmed by Scale, says Pro's graders mis-graded about a third of trials.

The two SWE-bench numbers come from different task sets with different difficulty and flaw profiles, and neither is independent. Recursive's own claims also carry small inconsistencies: 512k versus 262,144 context and "about 2.5B" versus "approximately 2B" active parameters on the Unsloth page. The speed claims (above 125 tokens/s on an M4 Pro, above 150 on a 64 GB RTX Spark) are for short contexts, so do not expect them in a long agent session.

What to do

1. Pick the quant by memory first: UD-Q3_K_XL on 24 GB, Q8_0 or Q6_K_XL on 128 GB, a 2-bit file on 16 GB only to experiment.
2. On Windows or Linux, use llama.cpp rather than Ollama until the GPU release lands, and pin a build that includes a37cc46.
3. If you run an agent harness, test tool calls on your own repository before trusting it. Watch for turns that end in stop with <tool_call> text in the reasoning (#31000), and for integer arguments turning into "N.0" strings on Ollama (#19191).
4. Use Recursive's sampling settings (temperature 1.0, top-p 0.95, top-k 20), keep reasoning_content in history, and budget at least 32,768 output tokens for hard problems.
5. Run your own eval on your own code before relying on 89.7%.

Sources

• RecursiveSI/flint model card
• Introducing Recursive Flint
• llama.cpp PR #30928
• llama.cpp issue #31000
• Ollama v0.40.8 release
• Ollama issue #19191
• Ollama library: flint
• ggml-org/Flint-GGUF
• unsloth/flint-GGUF
• Unsloth Flint guide
• RecursiveSI/flint-nvfp4-gguf
• SWE-Bench Pro V2 leaderboard (DataLearner)
• AI Weekly on the SWE-Bench audits