Laguna S 2.1 Coding Benchmark Performance: How Efficient Open Models Are Reshaping Enterprise AI

SAN FRANCISCO — Software engineering teams face a growing dilemma: relying on cloud-hosted frontier artificial intelligence models offers high capability but incurs steep API costs, latency bottlenecks, and strict data privacy constraints. That compromise shifted significantly with the release of Laguna S 2.1, an open-weight Mixture-of-Experts (MoE) coding model designed to deliver strong agentic performance within a fraction of the hardware footprint of massive cloud models.

Developed by artificial intelligence laboratory Poolside, Laguna S 2.1 features a total parameter count of 118 billion, yet activates only approximately 8 billion parameters per token during inference (~6.8% active). Released under the permissive OpenMDW-1.1 license, the model outscored open-weight competitors up to 14 times its total parameter size on several benchmark suites at launch, while ranking 11th overall on the Terminal-Bench 2.1 leaderboard Poolside published alongside it.

This article has been corrected. Two figures in the original version — a pair of local-inference throughput numbers and a claim about DeepSWE performance — were not supported by any published source. The correction note at the end of this article sets out what changed.

Benchmark Deep-Dive: Parameter Efficiency vs. Frontier Scale

The primary metric of interest for software architects is how a model with 8 billion active parameters competes against trillion-parameter deployments. The largest of those include Alibaba's open-weight Qwen 3.8-Max at 2.4 trillion parameters. In standardized evaluations, Laguna S 2.1 achieved notable scores across key software engineering and shell execution benchmarks, particularly when operating in its extended reasoning mode.

On Terminal-Bench 2.1, which measures multi-step command-line problem-solving, Laguna S 2.1 posted a score of 70.2, placing it 11th on the leaderboard Poolside compiled at launch. That put it ahead of the 1.6-trillion-parameter DeepSeek-V4-Pro-Max (64.0), the 975-billion-parameter Inkling (63.8) and the 550-billion-parameter Nemotron 3 Ultra (56.4), but behind Tencent's 295-billion-parameter Hy3 (71.7) and every frontier closed model on the board. On SWE-Bench Multilingual, its 78.5% topped Poolside's own published comparison set, edging Qwen 3.7 Max (78.3%); it was not the top of the public leaderboard, which Anthropic's Claude Mythos Preview leads at 87.3%. On DeepSWE v1.1, testing deep code comprehension and bug resolution, it scored 40.4% with reasoning enabled — well ahead of DeepSeek-V4-Pro-Max (9.0), and behind the 753-billion-parameter GLM-5.2 (44.0) in Poolside's own launch table.

Those are Poolside's launch-day figures, published 21 July 2026, and should be read as vendor numbers. Independent boards have since re-run several of the same models on their own harnesses and report different results — llm-stats, for example, now lists Claude Fable 5 at 84.3 on Terminal-Bench 2.1 rather than 88.0. The same caution applies to the leaked benchmarks for Anthropic's rumored Opus 5.

Almost two months on (leaderboard figures below are as of 13 September and were not re-pulled for this update), the public standings look materially different:

Two caveats apply to all of these numbers. Terminal-Bench 2.1 has been superseded twice, by 3.0 and then by 4.0 in August 2026; the maintainers describe the changes as breaking and state that trials must be re-run, so 2.1 and 4.0 scores are not comparable. Laguna S 2.1 has no published Terminal-Bench 4.0 result. Benchmark methodologies also vary across implementations: vendor evaluations on DeepSWE utilized custom agent harnesses, training runs required explicit filtering to prevent models from inspecting original pull-request patches online, and Poolside raised evaluation timeouts to five hours for Terminal-Bench 2.1, SWE-Bench Pro and SWE-Bench Multilingual, and six hours for DeepSWE v1.1.

Top-tier closed models retain a definitive edge on complex, high-reasoning tasks, and that frontier has moved since July. Claude Mythos 5.1 and GPT-6 Astra lead Terminal-Bench 4.0; Claude Fable 5.1 leads SWE-Bench Pro at 81.2%; Meta's closed-weight Muse Spark 1.3, released 3 September 2026, leads DeepSWE v1.1 at 75.4%. A head-to-head on the closed-model side, GPT-6 Astra vs Claude Fable 5.1 vs Opus 5.5 for coding, breaks down that frontier gap in more detail.

More consequential for Laguna's own argument is what has happened on the open-weight side. DeepSeek released DeepSeek-V4.1-Flash on 10 September 2026 under the MIT license: 552 billion total parameters activating roughly 8 billion per token on input — the same active-parameter budget as Laguna S 2.1 — and it now sits first on the Terminal-Bench 2.1 board at 90.6 and second on DeepSWE v1.1 at 74.2. That release is examined in more depth in DeepSeek-V4.1-Flash's asymmetric routing economics. Z.ai's GLM-5.3-Flash (320B total, 18B active, MIT, 25 August 2026) scores 84.3 and 63.4 on the same two boards. Alibaba's Qwen3.8-27B, a dense 27.8-billion-parameter Apache-2.0 model released 14 August 2026, scores 73.0 on Terminal-Bench 2.1 and 61.7 on SWE-Bench Pro at less than a quarter of Laguna's total footprint, and llm-stats' head-to-head now gives it all three shared benchmarks. The thesis Laguna S 2.1 was built to demonstrate now has better demonstrations.

Two late-September arrivals complicate the comparison without yet changing it, because neither ships open weights. Ant Group's Ling-3.1-flash (about 560 billion total, 25 billion active, announced 30 September) reports 65.39 on SWE-Bench Pro and 40.4 on Terminal-Bench 4.0, ahead of Laguna's 59.4 on SWE-Bench Pro. The SWE-Bench Pro figure is still vendor-only; the one independent Terminal-Bench 4.0 run, from Artificial Analysis, came in lower at 33.3%. Ant has so far only promised weights after a free trial. MiniMax's M3.1 Flash Preview, announced on 29 September, is available only through MiniMax's M Plan and MiniMax Code, with no published benchmark scores. Our write-ups cover Ling-3.1-flash and MiniMax M3.1 Flash. Moonshot's Kimi K3 remains the open-weight reference at the large end, at 2.8 trillion parameters, and scores 64.84 on SWE-Bench Pro in Ant's table. Three open-weight models sit above Laguna's 59.4 there: Kimi K3, Z.ai's GLM-5.3-Flash (63.06) and Qwen3.8-27B (61.7).

SWE-Bench Pro scores for eight models. Solid dots: public leaderboard (13 September 2026), the Qwen3.8-27B figure cited above, and Poolside's launch table; hollow dots: Ant Group's launch table. Harnesses differ, so this is not one controlled run.

One claim from July does still hold. Laguna S 2.1 remains the highest-scoring Western open-weight model on Terminal-Bench 2.1, ahead of Inkling-Small (64.7), Inkling (63.8), MAI-Code-1.1-Flash (62.9), Solar Pro 4 (57.0) and Nemotron 3 Ultra (56.4). Poolside has not shipped a successor; its model release notes still end at Laguna S 2.1.

Architectural Insights: MoE Design and Agentic Fine-Tuning

Laguna S 2.1 relies on a sparse MoE layout incorporating 256 routed experts and 1 shared expert, utilizing grouped-query attention and interleaved sliding-window layers across a native 1-million-token context window. Pre-trained in under nine weeks across a cluster of 4,096 Nvidia H200 GPUs, the model demonstrates that post-training environment exposure can outweigh raw pre-training volume.

Rather than altering the foundational dataset, developers built a post-training corpus spanning over 409,000 agentic and non-agentic environments; within that total, 83,000 setups are dedicated to terminal use cases and 168,000 target standard software engineering workflows. This process emphasized persistent verification habits—forcing the model to re-test code outputs against local test suites before finalizing changes.

The model operates with two explicit reasoning states: off and max, with no intermediate effort dial in between. Enabling max reasoning mode increases Terminal-Bench 2.1 performance from 60.4 to 70.2 and boosts DeepSWE v1.1 accuracy from 16.5% to 40.4%, albeit at an increased token generation footprint — on DeepSWE, a mean of ~249,000 completion tokens per trajectory with reasoning enabled versus ~99,000 without.

For autonomous tool interaction, the model issues structured JSON tool calls inside agentic frameworks:

# Sample agentic verification tool call issued by Laguna S 2.1
{
 "tool_name": "execute_shell_command",
 "arguments": {
 "command": "pytest tests/test_parser.py -k 'test_nested_json'",
 "timeout_seconds": 30,
 "environment": {"PYTHONPATH": "."}
 }
}

Enterprise Security Posture: Local Hosting and Supply Chain Sovereignty

For technology leaders in regulated sectors—such as finance, healthcare, defense, and public infrastructure—sending proprietary source code through third-party SaaS APIs introduces legal compliance and intellectual property risks. That's the same calculus driving interest in replacing ChatGPT with a self-hosted stack for regulated workloads.

+-------------------------------------------------------------------+
| Enterprise Internal Security Perimeter |
| |
| +-------------------+ Local +-------------------------+ |
| | Engineering Work- | Inference | Laguna S 2.1 (Self- | |
| | stations / CI/CD | --------> | Hosted 118B-A8B MoE) | |
| +-------------------+ (No Data)| +-------------------------+ |
+-------------------------------------------------------------------+
 | (NO DATA EGRESS)
 v
 [ External Public Internet ]

Deploying open-weight models within an enterprise private cloud or air-gapped infrastructure eliminates data egress vulnerabilities. Source code, API keys, and internal system architectures remain strictly within corporate boundaries. Published under the OpenMDW-1.1 license, Laguna S 2.1 provides an auditable Western open-weight baseline, offering an alternative for organizations seeking to avoid reliance on proprietary APIs or foreign open-source dependencies. To address benchmark transparency concerns, the model developers released unedited execution logs detailing shell outputs, reasoning chains, and code modifications for every trial in the final evaluation set, at trajectories.poolside.ai.

Real-World Economics and Deployment Trade-Offs

Deploying Laguna S 2.1 requires understanding the hardware dynamics of Mixture-of-Experts architectures. Although only 8 billion parameters participate in computation for any given token, all 118 billion parameters must reside in fast GPU memory for real-time serving. Sizing that kind of footprint against your own hardware is easier with a dedicated VRAM math for open-weight models across every quantization format.

Those figures come from Poolside's official vLLM serving recipe and cover weights only. A working DGX Spark deployment of the NVFP4 build loaded 67 GiB of weights alongside 27.11 GiB of FP8 KV cache — roughly 107 GB of the machine's 128 GB unified memory at 0.80 GPU memory utilization, enough cache for about 15 parallel requests at maximum context. The model does fit on a single 128 GB box, but with materially less headroom than a naive parameter-count calculation suggests.

Throughput on that class of hardware is modest. In a published DGX Spark run of the NVFP4 checkpoint, single-stream decoding measured 19.5 tokens per second, holding essentially flat at 19.39 tokens per second with a 262,144-token context. Pairing the model with Poolside's DFlash draft model for speculative decoding raised single-stream output to 31.15 tokens per second at a speculation depth of K=3, a 1.60x speedup; the recipe's suggested K=15 was slower at 29.06 tokens per second, because accepted length plateaus near two tokens regardless of depth. Under eight concurrent requests the same configuration reached 121.9 aggregate tokens per second against 91.0 without speculation. Teams sizing an internal deployment should plan around these measured figures.

For organizations that would rather not self-host, Poolside runs its own hosted inference (documented at docs.poolside.ai/api/overview) and the model is also listed on gateways. Poolside publishes no price page on poolside.ai; the launch price on OpenRouter was $0.10 per million input tokens and $0.20 per million output tokens. As of 8 October 2026 that endpoint carries a 10% discount, currently $0.09 and $0.18 (cache reads $0.009), with a 1,048,576-token context, which keeps it among the cheaper ways to run a 1-million-token-context coding model.

Plan for 31 October. OpenRouter lists both the paid and the free Laguna S 2.1 endpoints as going away on 31 October 2026. Teams that route through OpenRouter need to move to Poolside's hosted API or self-host before then.

Data terms differ by endpoint. OpenRouter's paid endpoint states that prompts are not used for training and are retained for 90 days. The free 256K endpoint says: "If you are using Laguna S 2.1 for free, we may use your inputs and outputs to train and improve our models." Keep proprietary code off the free endpoint.

However, deployment comes with clear operational caveats:

Key Operational Considerations:

• Grounding Under Pressure: Independent stress testing indicates that when forced to handle non-coding tasks under high ambiguity, the model can fabricate facts if its internal reasoning cutoff is triggered too early.
• Tool Schema Rigidity: While highly effective in native agent environments, Laguna S 2.1 occasionally distorts complex nested JSON arguments when interfaced with non-standard third-party agent schemas.
• Lack of Native Vision: The model does not include native multimodal image processing, requiring external OCR or vision preprocessing when interpreting UI design specs.
• Specialization: Optimized for software engineering and mathematics, general-knowledge tasks trail broader foundation models.
• Throughput Ceiling: On a single 128GB unified-memory workstation, expect roughly 20 tokens per second single-stream, or about 31 with speculative decoding—adequate for background agents, slow for interactive pair programming.

The Path Ahead for Enterprise Software Development

The release of Laguna S 2.1 marked a real shift in AI integration strategies for software development lifecycles: a model small enough to sit inside a corporate boundary that could hold its own against open-weight systems many times its size. What the weeks since have shown is that the shift was industry-wide rather than specific to one model. DeepSeek-V4.1-Flash now runs the same 8-billion active-parameter budget to a 90.6 on Terminal-Bench 2.1 against Laguna's 70.2, under an MIT license; a 27.8-billion-parameter dense Qwen model outscores Laguna on all three of their shared benchmarks at a quarter of the memory footprint. That model, Alibaba's Qwen3.8-27B, is covered on its own in Qwen3.8-27B's frontier agentic benchmark results.

For engineering organizations, the practical conclusion is unchanged but the shortlist is not. The choice is no longer restricted to paying high per-token API costs or accepting low-quality local code completion. But the open-weight field now turns over in weeks, and a model chosen in July on the strength of a leaderboard position will not hold that position by autumn. License terms, data sovereignty, quantized memory footprint and measured throughput on hardware the organization actually owns are the durable selection criteria; benchmark rank is not.

Correction

14 September 2026. Two claims in the original version of this article, published 22 July 2026, were not supported by any published source. Both have been corrected.

Inference throughput. The article stated that Laguna S 2.1 quantized to 4-bit NVFP4 achieved "single-stream generation speeds of up to 109 tokens per second at 256K context lengths," and that "single-user inference speeds reach up to 271 tokens per second" when paired with speculative decoding. Neither figure appears in Poolside's launch post, the Hugging Face model card, or the official vLLM serving recipe — none of which publish tokens-per-second numbers at all. The measured figures from a published DGX Spark run of the NVFP4 build are 19.5 tokens per second single-stream, 19.39 at a 262,144-token context, and 31.15 tokens per second with DFlash speculative decoding at K=3. The original numbers overstated real single-stream throughput by roughly five to nine times. The run is documented at dev.classmethod.jp/en/articles/dgx-spark-laguna-s-2-1-nvfp4/.

DeepSWE v1.1. The article stated that Laguna S 2.1's 40.4% on DeepSWE v1.1 was "significantly outperforming larger open-weight alternatives." Poolside's own launch table contradicted this on the day of publication: it listed GLM-5.2, a 753-billion-parameter open-weight model with 40 billion active, at 44.0% — above Laguna. The accurate statement is that Laguna S 2.1 scored far ahead of DeepSeek-V4-Pro-Max (9.0) and behind GLM-5.2 (44.0). As of 13 September 2026 it ranks 28th of 32 models on the public DeepSWE 1.1 board, behind five open-weight models.

Separately, the 4-bit memory footprint has been corrected from ~59 GB to ~72 GB to match Poolside's official serving recipe; ~59 GB was a naive parameter-count calculation rather than a measured or published footprint. Benchmark ranks, the SWE-Bench Multilingual framing, the description of Poolside's 409,000-environment training corpus and the competitive comparison set have all been updated to the position as of 14 September 2026.

Sources

• OpenRouter: Laguna S 2.1 pricing
• OpenRouter: Ling 3.1 Flash
• Artificial Analysis: Ling 3.1 Flash
• Poolside API overview
• MiniMax: subscription plan announcements
• MiniMax API docs: text generation

Related reading

• Xiaomi's MiMo-V2.6 Breaks the CyberGym Ceiling and Ships MIT Weights the Same Day
• Qwen 2.5-32B vs. Llama 3.3-70B: Enterprise Foundation Model Sizing, Benchmarks, and Local VRAM Economics
• Open-Weight Coding at the Frontier: Evaluating Qwen2.5-Coder-32B for Enterprise Software Engineering