Underdog's Saluki 27B packs Qwen3.8-27B into a 7.89 GB 2-bit GGUF. Vendor tests show tool calling intact but a large drop in competition maths.
Underdog (published as ConwayResearch on Hugging Face) has released Saluki 27B 1.0, a 2-bit GGUF of Alibaba's Qwen3.8-27B that weighs 7.89 GB against 54 GB for the full model. It is aimed at people with a 16 GB GPU who want local tool calling. Every score below is vendor-run, from Underdog's own model card, and has not been independently reproduced.
What it is
The card describes Saluki as "Qwen3.8-27B in under 8 GB, tuned to keep tool calling intact." The main file, Underdog-Saluki-27B-1.0-IQ2-mix.gguf, is an IQ2 mix with imatrix. It is quantized from the ISTA-DASLab GSQ-RCO GGUF of Qwen3.8-27B, and the card gives no further method detail. It is Apache 2.0 and runs in stock llama.cpp. The main file is text-only; vision needs a separate mmproj add-on of 928 MB (F16) or 629 MB (Q8_0). The card does not state a VRAM requirement, so the 16 GB fit is our inference from the file size plus context cache, not a vendor claim.
Recommended launch: llama-server --jinja -ngl 99 -fa on -c 32768. For general use the card suggests thinking on with temperature 0.6, top_p 0.95, top_k 20. For fast tool calls it suggests thinking off with temperature 0.
Vendor-run scores, and how comparable they are
The card's key caveat is the harness. Where Underdog ran the full model itself, it used the same harness and settings as for Saluki. Rows marked "(public)" use published figures from a different harness, and the card says those are not directly comparable.
| Benchmark | Saluki | Full model | Baseline source |
|---|---|---|---|
| Underdog Bench (120 BFCL v4-derived tasks, thinking off) | 88 | 84 | Underdog-run, same harness |
| Parallel tool calls (100 BFCL v4 tasks, official checker) | 42 | 35 | Underdog-run, same harness |
| SWE-bench Verified (issues fixed, of 50) | 30 | 33 | Per card; not marked public |
| AIME 2025 (avg@4) | 79.2 | 96.7 | Public, different harness |
| AIME 2026 (avg@4) | 80.0 | 94.6 | Public, different harness |
| IFEval (prompt-loose) | 93.5 | 91.5 | Public, different harness |
| MBPP+ | 78.0 | 83.9 | Public, different harness |
| MuSR | 67.5 | 79.6 | Public, different harness |
Underdog reports 96% average retention across nine benchmarks.
Saluki 27B vs the full Qwen3.8-27B. All scores vendor-reported; AIME full-model figures come from a different harness.
Read the table in two halves. The tool-calling rows are like-for-like and favor Saluki, but the task set is small: the card itself says 120 tasks is modest and that a few tasks of run-to-run variation matter, so 88 versus 84 is close to a tie, not a win. The 42 versus 35 parallel-call gap is larger, yet the card admits about a fifth of parallel replies carry small formatting slips. The SWE-bench figure is 30 against 33 fixed; we could not confirm from the card how the 50-issue sample was drawn, so treat it as a small-sample indication. The AIME gap is real and large: about 17 points on 2025 and 15 on 2026, with the card putting retention at roughly 82 to 85% of the full model's math score. Because those baselines come from a different harness, the true gap could be smaller or larger.
A separate caution: the ISTA-DASLab upstream card reports BF16 AIME25 of 100.00, while Saluki's card cites 96.7 as the public full-model figure. Different harnesses and settings produce different baselines, so do not mix numbers across cards.
Where upstream stands
ISTA-DASLab's own GSQ-RCO quants are self-reported too. Its smallest, IQ2_XS at 2.50 bits per weight and 8.4 GB, scores 96.67 on AIME25 against 100.00 for BF16, per that card, which does not name its harness or sampling settings. Saluki's 7.89 GB file is smaller than that, which suggests a more aggressive mix, but the cards do not say what was changed.
Who it suits
- Good fit: local agents and function-calling pipelines on a 16 GB card, where tool-call accuracy matters more than competition math or long reasoning.
- Reasonable fit: instruction following; IFEval and IFBench were slightly ahead of the (public) baselines.
- Poor fit: math-heavy or multi-step reasoning work (AIME, MuSR gaps), and tasks needing strict letter-level instruction compliance, which the card names as a weakness.
- Watch: with thinking on, the model often reasons at length, which costs time and context.
For hardware context see ThreatFrontier's best local LLM for 16 GB VRAM, RTX 5060 Ti and Mac mini M4 and the quantization format guide for the VRAM math.
What to do
Run your own tool-calling tests on your own schemas before swapping it in, especially parallel calls, and add output validation for malformed arguments. Compare against a 3-bit quant of the same model if you have headroom. Wait for independent reproductions before relying on the SWE-bench and AIME figures.