AI

Writing on AI engineering

Notes and how-tos on building with LLMs — prompting, agents, retrieval, and evaluation. Written to help other engineers ship AI features that hold up in production.

Portrait of Sachin Gupta rendered in binary

Articles

33 articles

Writing· Speculative Decoding
In the Machine Room
The last five parts were the recipe. This one is the kitchen at dinner rush: the two engineering tricks that get DSpark running inside a real serving system, and the speedups it delivers there.
Part six of the speculative decoding series, the finale. Parts one to five were the idea. This one is the engineering: two tricks that get DSpark running inside a live serving system. First, do not ship the whole dictionary of scores between machines while training the guesser, ship the compact summary and do the last step locally. Second, a scheduler that cannot wait, because real GPU speed rises in jagged steps and the pipeline cannot pause, so it predicts the batch size ahead of time. Plus the real production speedups. Plain words, an analogy, a diagram, and two auto-playing visualizers.
- Inference
By Sachin Gupta9 Jul 20267 min read
Writing· Speculative Decoding
The Expediter
DSpark's last machine, on its own: how it decides exactly how much of the draft to check, across everyone it is serving at once, like loading a delivery van to its sweet spot.
Part five of the speculative decoding series, and the last of three that open up DSpark one machine at a time. This one is the expediter, the paper's greedy scheduler. It decides how much of the draft the big model should check, across everyone it is serving at once, and the rule is neat: pool every guessed word, sort the most likely to pass first, and keep loading the batch while the words-per-second keeps climbing, then stop at the peak. Explained with a delivery-van analogy, a diagram, and an auto-playing visualizer. Short, one idea, and it wraps the three machines.
- Inference
By Sachin Gupta8 Jul 20265 min read
Writing· Speculative Decoding
The Honesty Check
DSpark's second machine, on its own: the AI's confidence scores are overconfident, like a forecaster who says 90% but is right 70% of the time, so DSpark corrects them to match reality.
Part four of the speculative decoding series, and the second of three that open up DSpark one machine at a time. This one is the honesty check. DSpark puts a confidence number on every guessed word, but those numbers run high, the same way a weather forecaster who says 90% is often right only 70% of the time. Calibration shaves them down until they match reality, without changing their order, which is what lets the next machine trust them. Plain words, a worked example, a diagram, and an auto-playing visualizer. Short, one idea only.
- Inference
By Sachin Gupta7 Jul 20265 min read
Writing· Speculative Decoding
The Cheat Sheet
DSpark's first fix, on its own: how a tiny cheat sheet, the same trick your phone uses, makes each guessed word fit the one before it.
Part three of the speculative decoding series, and the first of three that open up DSpark one machine at a time. This one is the cheat sheet, the paper's Markov head: the same trick your phone's predictive text uses, where the word you just typed nudges the next one. It explains what a score even is, why the fast guesser is blind, and how a small nudge breaks the tie, with a worked example, a diagram, and an auto-playing visualizer. Short, one idea only.
- Inference
By Sachin Gupta6 Jul 20265 min read
Writing· Speculative Decoding
Draft Better, Verify Smarter
A fast drafter gets sloppy toward the end, and checking its whole guess wastes the big model's time. DSpark makes two small changes that fix both, and the output never changes.
Part two of the speculative decoding series, in plain language. Part one left a dilemma: a draft model is either accurate but slow or fast but sloppy, and checking a long shaky draft wastes the big model's time. This part walks through DSpark, an open method from DeepSeek, one small change at a time. First, let each drafted word see the word before it, so the fast drafter stops contradicting itself. Second, score each word's chance of surviving, multiply those scores into a running number that only falls, and stop checking the moment it stops being worth it. Two interactive visualizers, one of them auto-playing through the whole cycle, and the production numbers, with no change to the output.
- Inference
By Sachin Gupta5 Jul 20268 min read
Writing· RAG series
Vectorless RAG
When a document has a structure, stop shredding it. Navigate it.
Part six of the RAG series. The default RAG recipe chops a document into chunks, embeds them, and retrieves by vector similarity. For a long, structured document that throws away the one thing that made it readable: its structure. Vectorless RAG keeps the document whole, builds a table-of-contents tree, and has a model reason its way to the right section, the way a human expert uses a table of contents. This covers why similarity is not relevance, how tree-navigation retrieval works, where it fails, when to reach for it, and when plain vector search is still the right tool. With an interactive navigator.
- RAG
By Sachin Gupta4 Jul 20268 min read
Writing· Speculative Decoding
Draft, Then Verify
Writing one token at a time makes a large model slow. Speculative decoding speeds it up by letting a small model guess ahead.
Part one of a series on speculative decoding, the trick that lets a large language model generate text faster with no change to the output. This part covers why one-token-at-a-time is slow, the lossless draft-and-verify rule that fixes it, and the real difficulty: the draft model. It walks through the two families of draft models and the flaw in each, autoregressive drafters that are accurate but slow, and parallel drafters that are fast but decay down the block. With two interactive visualizers. Part two covers how DeepSeek's DSpark gets the best of both.
- Inference
By Sachin Gupta4 Jul 20266 min read
Writing· Essay
The Harness Is the Agent
Why agent reliability lives in the scaffolding, not the model.
When an agent works in production, we credit the model. When it fails, the cause is almost always in the harness around it: the loop, the tools, the context, the guardrails, the checks. The harness is the part you actually own.
- AI Agent Harness
By Sachin Gupta2 Jul 20266 min read
Writing· MCP series
How MCP Grew
Four spec revisions, an official registry, the Linux Foundation, and the ecosystem that adopted it.
The closing piece of the MCP series: the four revisions that shaped the protocol through its first year and a half, the official registry and MCP Apps, the move to the Linux Foundation and open governance, how it went from one release to an industry standard, and where the 2026 roadmap points.
- MCP
By Sachin Gupta25 Jun 20268 min read
Writing· Essay
A Coding Agent Is a Loop Around a Compiler
The model writes the code; the harness makes it correct by running it.
A coding agent that only writes code is a fancy autocomplete. The thing that turns a plausible diff into a change you can merge is the code harness: the sandbox, the tools, and the loop that runs the code and reads the real errors.
- Code Harness
By Sachin Gupta25 Jun 20265 min read

Explore by tag

AI — Sachin Gupta