Tag archive

Structured Output

Coverage tagged structured output.

Data-poster diagramming an AI agent control plane: an agent routed through interception, workload identity, a policy engine and guard models to a credential broker, an MCP server and a signed audit ledger.

Structured Output and Constrained Decoding: Making LLMs Return Valid JSON

Prompting for JSON gets you high-90s reliability; a grammar-driven token mask gets you a guarantee. The mechanism, the real costs, and where it still fails.

Related desks

AI Security

Latest

Sep 12, 2026, 12:37 PM EDT · Research · Mlx12 minMLX vs llama.cpp on Apple Silicon: Which Local LLM Runtime to Actually Use

One is a portable inference engine, the other an array framework that also trains — and that single difference settles almost every practical question.

Sep 12, 2026, 12:37 PM EDT · Operations · Llm Evaluation14 minHow to Build an LLM Evaluation Harness That Catches Real Regressions

Public benchmarks only shortlist a model; your own cases decide whether to ship it. Here is the dataset, the scorers and the CI gates that catch real drift.

Sep 12, 2026, 12:37 PM EDT · Research · Prompt Caching11 minPrompt Caching Explained: Cutting Agent Costs Without Touching the Model

Your agent re-sends the same long prefix on every step and pays to recompute it. Here is the KV mechanism, the ordering rule it forces, and the break-even math.

Sep 12, 2026, 12:37 PM EDT · Research · Speculative Decoding11 minSpeculative Decoding and Multi-Token Prediction: How LLMs Generate Faster

Decode is memory-bound, so checking several tokens in one pass is nearly free. Here is the mechanism, the arithmetic behind it, and why quality never moves.