Inference

Peer-reviewer for 2nd Workshop on Efficient Reasoning

Peer reviewer for the 2nd Workshop on Efficient Reasoning at COLM 2026, reviewing three submitted papers.

• Inference

In the Machine Room

The last five parts were the recipe. This one is the kitchen at dinner rush: the two engineering tricks that get DSpark running inside a real serving system, and the speedups it delivers there.

Part six of the speculative decoding series, the finale. Parts one to five were the idea. This one is the engineering: two tricks that get DSpark running inside a live serving system. First, do not ship the whole dictionary of scores between machines while training the guesser, ship the compact summary and do the last step locally. Second, a scheduler that cannot wait, because real GPU speed rises in jagged steps and the pipeline cannot pause, so it predicts the batch size ahead of time. Plus the real production speedups. Plain words, an analogy, a diagram, and two auto-playing visualizers.

• Inference

The Expediter

DSpark's last machine, on its own: how it decides exactly how much of the draft to check, across everyone it is serving at once, like loading a delivery van to its sweet spot.

Part five of the speculative decoding series, and the last of three that open up DSpark one machine at a time. This one is the expediter, the paper's greedy scheduler. It decides how much of the draft the big model should check, across everyone it is serving at once, and the rule is neat: pool every guessed word, sort the most likely to pass first, and keep loading the batch while the words-per-second keeps climbing, then stop at the peak. Explained with a delivery-van analogy, a diagram, and an auto-playing visualizer. Short, one idea, and it wraps the three machines.

• Inference

The Honesty Check

DSpark's second machine, on its own: the AI's confidence scores are overconfident, like a forecaster who says 90% but is right 70% of the time, so DSpark corrects them to match reality.

Part four of the speculative decoding series, and the second of three that open up DSpark one machine at a time. This one is the honesty check. DSpark puts a confidence number on every guessed word, but those numbers run high, the same way a weather forecaster who says 90% is often right only 70% of the time. Calibration shaves them down until they match reality, without changing their order, which is what lets the next machine trust them. Plain words, a worked example, a diagram, and an auto-playing visualizer. Short, one idea only.

• Inference

The Cheat Sheet

DSpark's first fix, on its own: how a tiny cheat sheet, the same trick your phone uses, makes each guessed word fit the one before it.

Part three of the speculative decoding series, and the first of three that open up DSpark one machine at a time. This one is the cheat sheet, the paper's Markov head: the same trick your phone's predictive text uses, where the word you just typed nudges the next one. It explains what a score even is, why the fast guesser is blind, and how a small nudge breaks the tie, with a worked example, a diagram, and an auto-playing visualizer. Short, one idea only.

• Inference

Draft Better, Verify Smarter

A fast drafter gets sloppy toward the end, and checking its whole guess wastes the big model's time. DSpark makes two small changes that fix both, and the output never changes.

Part two of the speculative decoding series, in plain language. Part one left a dilemma: a draft model is either accurate but slow or fast but sloppy, and checking a long shaky draft wastes the big model's time. This part walks through DSpark, an open method from DeepSeek, one small change at a time. First, let each drafted word see the word before it, so the fast drafter stops contradicting itself. Second, score each word's chance of surviving, multiply those scores into a running number that only falls, and stop checking the moment it stops being worth it. Two interactive visualizers, one of them auto-playing through the whole cycle, and the production numbers, with no change to the output.

• Inference

Draft, Then Verify

Writing one token at a time makes a large model slow. Speculative decoding speeds it up by letting a small model guess ahead.

Part one of a series on speculative decoding, the trick that lets a large language model generate text faster with no change to the output. This part covers why one-token-at-a-time is slow, the lossless draft-and-verify rule that fixes it, and the real difficulty: the draft model. It walks through the two families of draft models and the flaw in each, autoregressive drafters that are accurate but slow, and parallel drafters that are fast but decay down the block. With two interactive visualizers. Part two covers how DeepSeek's DSpark gets the best of both.

• Inference