MLX vs llama.cpp on Apple Silicon: Which Local LLM Runtime to Actually Use
One is a portable inference engine, the other an array framework that also trains — and that single difference settles almost every practical question.
Author
Owen Park
AI security researcher
Covers AI application security, model risk, and agentic system abuse.
ai-securityresearchapplication-security
Coverage by Owen Park (28)
Prompt Caching Explained: Cutting Agent Costs Without Touching the Model
Your agent re-sends the same long prefix on every step and pays to recompute it. Here is the KV mechanism, the ordering rule it forces, and the break-even math.
Speculative Decoding and Multi-Token Prediction: How LLMs Generate Faster
Decode is memory-bound, so checking several tokens in one pass is nearly free. Here is the mechanism, the arithmetic behind it, and why quality never moves.
Structured Output and Constrained Decoding: Making LLMs Return Valid JSON
Prompting for JSON gets you high-90s reliability; a grammar-driven token mask gets you a guarantee. The mechanism, the real costs, and where it still fails.
Reranking Explained: Why Your RAG Pipeline Needs a Second Scoring Pass
Your retriever found the right chunk and buried it at rank 31. A cross-encoder second pass fixes ordering without touching recall, and here is how to size it.
Best Open-Weight Embedding Models 2026: Benchmarks, Dimensions and VRAM Cost
Dimension count, sequence length and model size decide your index memory, chunk limits and ingest hours. A framework that prices the retrieval quality you buy.
Chunking Strategies for RAG: How Document Splitting Decides Retrieval Quality
Most retrieval bugs are split bugs. A working reference on overlap, structure-aware and small-to-big splitting, with concrete defaults you can tune from.
Production RAG Architecture: Retrieval Pipelines That Actually Work
Ingestion, chunking, embeddings, hybrid search, reranking and context assembly, plus the tenancy leaks and injection paths each stage quietly opens up.