Tag archive

Gpu Vram Requirements

Coverage tagged gpu vram requirements.

Protocol schematic poster for a 128GB Apple Silicon local AI stack led by Qwen3.6

Will It Fit? VRAM Math for Open-Weight Models Across Every Quantization Format

Weights are only one of four memory buckets, and past roughly 115k tokens the KV cache costs more than the model does. Here is the arithmetic in full.

Related desks

Research

Latest

Sep 12, 2026, 12:37 PM EDT · Research · Mlx12 minMLX vs llama.cpp on Apple Silicon: Which Local LLM Runtime to Actually Use

One is a portable inference engine, the other an array framework that also trains — and that single difference settles almost every practical question.

Sep 12, 2026, 12:37 PM EDT · Operations · Llm Evaluation14 minHow to Build an LLM Evaluation Harness That Catches Real Regressions

Public benchmarks only shortlist a model; your own cases decide whether to ship it. Here is the dataset, the scorers and the CI gates that catch real drift.

Sep 12, 2026, 12:37 PM EDT · Research · Prompt Caching11 minPrompt Caching Explained: Cutting Agent Costs Without Touching the Model

Your agent re-sends the same long prefix on every step and pays to recompute it. Here is the KV mechanism, the ordering rule it forces, and the break-even math.

Sep 12, 2026, 12:37 PM EDT · Research · Speculative Decoding11 minSpeculative Decoding and Multi-Token Prediction: How LLMs Generate Faster

Decode is memory-bound, so checking several tokens in one pass is nearly free. Here is the mechanism, the arithmetic behind it, and why quality never moves.