Draft Better, Verify Smarter
A fast drafter gets sloppy toward the end, and checking its whole guess wastes the big model's time. DSpark makes two small changes that fix both, and the output never changes.
Part two of the speculative decoding series, in plain language. Part one left a dilemma: a draft model is either accurate but slow or fast but sloppy, and checking a long shaky draft wastes the big model's time. This part walks through DSpark, an open method from DeepSeek, one small change at a time. First, let each drafted word see the word before it, so the fast drafter stops contradicting itself. Second, score each word's chance of surviving, multiply those scores into a running number that only falls, and stop checking the moment it stops being worth it. Two interactive visualizers, one of them auto-playing through the whole cycle, and the production numbers, with no change to the output.
Part one left a dilemma. A draft model can be accurate but slow, writing one word at a time, or fast but sloppy, writing a whole block at once and watching the end of it fall apart. And even a good draft can be wasteful: making the big model check words that are probably wrong spends its time for nothing. DSpark, an open method from DeepSeek, makes two small changes that fix both, and it never changes a single word of the output.
Picture the slow head chef and the fast line cook from part one again. Two things were going wrong. The line cook plated the next several items all at once without looking at what he had just plated, so he would start one dish and finish another. And the head chef, to be safe, inspected each run all the way to its last shaky plate, even when that plate was clearly going to be wrong, while other orders backed up behind her.
DSpark fixes one thing on each side. It makes the line cook glance at the item he just plated before adding the next, so he stops contradicting himself. And it lets the head chef stop inspecting the moment a run stops looking trustworthy, and stop sooner when the kitchen is slammed. Same finished dishes as before. Far less wasted effort.
The two problems, plainly
Part one named both. Here they are in one line each:
- A fast drafter gets sloppy. A parallel drafter writes every word of the block in a single pass, each word blind to the others, so it grows less reliable toward the end. Part one called this suffix decay.
- Checking everything wastes the big model. Verifying a long block that is mostly going to be rejected uses capacity that other users' requests needed. Under heavy traffic, drafting long and checking all of it can actually make the whole system slower.
DSpark (paper) pairs one fix to each. Take them one at a time.
Fix one: let each word see the one before it
First, a fair question: how is a small, fast drafter any good at guessing what a much bigger model will say? The trick is that it does not read the conversation and work it out on its own. As the big model runs, it builds an internal read of the context, the meaning behind the words so far. The drafter is wired to read that directly and continue from it, and it borrows the big model's own input and output layers so the two speak the same language. Like a line cook working straight from the head chef's ticket and prepped ingredients instead of re-reading the order, it skips the expensive understanding step and rides on work the big model already did, which is exactly why it can be small and still guess well.
That still leaves the sloppiness. A parallel drafter is fast because it writes the whole block in one pass, but that speed is exactly the problem: each position is guessed without knowing what the other positions turned out to be. So when two replies both fit, "of course" and "no problem," it can take the first word of one and the second word of the other and hand you "of problem." Every word is fine on its own. Together they clash.
DSpark keeps the fast pass and adds one small thing on top: a lightweight sequential head that runs over the block and lets each word see the word chosen just before it. Now the second word is not guessed blind. Once the first word is "of," the head nudges the next word toward "course" and away from "problem." The crossbreed stops.
In plain terms, the score DSpark uses for the next word is the parallel drafter's own score plus a small bias that depends on the word right before it. That bias is the whole fix. Flip it on and off here, and change what the first word lands on:
This sloppiness, which the paper calls acceptance decay (also called suffix decay), gets measured position by position, on their 4B model, with a simple question: given every earlier word in the block was accepted, how often is this one accepted too? On chat, a plain parallel drafter slides from about 0.72 at the first word down to 0.63 by the seventh, each later word guessed with less to stand on. An autoregressive drafter does the opposite, climbing from about 0.53 to 0.74, because every word genuinely sees the ones before it. DSpark is built to get both at once: it keeps the parallel drafter's strong start, then, thanks to the sequential head, holds a high and steady rate deep into the block instead of letting it slide. That steady line, not any single headline number, is the real proof that fix one works.
The important part is the cost, which is almost nothing. The heavy work, reading the big model's context and proposing the block, still happens in one fast pass; only the tiny head runs word by word. In the paper, growing the block from 4 to 16 words adds between 0.2% and 1.3% to the time per round on their 4B model, while raising the number of words accepted per round by as much as 30%. Across three Qwen3 model sizes, it accepts about 16% to 18% more words per round than the parallel drafter it is built on, and about 27% to 31% more than an autoregressive one.
Fix two: stop checking when the numbers keep dropping
Drafting a longer block does not make anything faster on its own. If the big model still has to check the whole thing, a long block full of shaky words just moves the waste from drafting to checking. So DSpark adds a second small output, a confidence head, that gives each drafted word a score: how likely that word is to survive the big model's check, assuming every word before it already passed.
Now the key move. Remember from part one that acceptance is a prefix: the first wrong word throws away everything after it. So the chance a whole run survives is not any single word's score, it is all of them multiplied together. Say the first four words score 0.93, 0.80, 0.55, and 0.35 on their own. Multiply as you go:
- through word 1: 0.93
- through word 2: 0.93 x 0.80 = 0.74
- through word 3: 0.74 x 0.55 = 0.41
- through word 4: 0.41 x 0.35 = 0.14
Look at what that running number does: it only falls. Every new word multiplies in another score below one, so the survival chance of the block can only get worse the longer the block runs. The fourth word was already a long shot at 0.35, and by the time you chain it on, the whole run is down to 0.14.
That is the whole idea behind checking smarter. Because the number only goes down, once it drops below what a check is worth, you can stop, and you never have to look further, because further is always worse. DSpark keeps the confident front of the block and drops the shaky tail before the big model ever sees it. In the example it keeps "next Monday at" and throws "noon" away unchecked.
Where it draws that line is load-aware, and this is the part that matters in production. When traffic is light, spare capacity is nearly free, so it checks deep and grabs every word it can. When traffic is heavy, it checks only the surest words, so the big model's scarce capacity goes to serving more people instead of checking words that were going to be rejected anyway. (The exact rule it uses to draw the line, across all the requests it is serving at once, is part three.)
Change the server load here and watch where it stops. The bars are the running number falling; the dashed line is what a check is worth. Heavier load raises the line, so it stops sooner:
How deep it pays to check also depends on the text. Code and math are predictable, so their drafts stay right further down the block; open-ended chat is not. In the paper, on the same 4B model, as the cut gets stricter, chat's acceptance rate climbs from about 46% to 96%, because chat has long shaky tails to trim, while code and math (already steadier) rise from the high 60s and 70s to about 92%.
And through all of this it stays lossless. Dropping a word before the check never changes the answer; it only gives up a possible speedup. Whatever DSpark does keep is still checked against the big model exactly as before, so the text that comes out is still the big model's own, word for word.
The whole cycle
Put both fixes together and one round looks like this: the big model commits one word to start (the anchor), the parallel pass plus the sequential head draft a block with a confidence on each word, the running number decides how much of the block is worth checking, and the big model checks that much in a single pass, keeping the correct part and correcting the first miss. The correction becomes the anchor for the next round.
Here is that same cycle running on a real sentence, start to finish, on its own. Watch the confidence scores turn into a running number, watch it fall, and watch DSpark stop early and drop the tail before the check:
The model is writing a reply. So far: "The meeting is scheduled for". It needs the next words.
Does it hold up
Offline, across four target models and the math, code, and chat domains, DSpark accepts more words per round than both baselines from part one, the accurate-but-slow autoregressive drafter and the fast-but-sloppy parallel one. That is the "draft better" half, measured directly.
Online is the real test. Deployed inside DeepSeek's V4 serving system under live user traffic, and compared against the previous production setup, DSpark made each user's text come out 60% to 85% faster on one model tier and 57% to 78% faster on another, at the same total throughput. It also held its speed under strict latency limits where the old setup's capacity fell apart, opening service tiers that were not reachable before. That is the "verify smarter" half: the gain shows up only once you stop wasting the big model's capacity under load.
DeepSeek open-sourced the DSpark checkpoints alongside DeepSpec, a training repository that also ships the two baselines from part one, the autoregressive Eagle3 and the parallel DFlash, so the whole comparison is reproducible.
Where this goes
Two small changes, a glance at the previous word while drafting and a running number that decides when to stop checking, turn "fast but sloppy, and wasteful to check" into a real production speedup with the output untouched.
Part three opens the machinery underneath. The sequential head is a small Markov or RNN module, and the way it turns "see the previous word" into an actual number is a clean piece of math worth seeing. The confidence scores need calibrating before you can trust them enough to stop on, because a raw neural score tends to be overconfident. And "stop when it is no longer worth it" becomes one exact rule, a greedy scheduler that decides, across every request being served at once, how many words are worth the big model's time this step.
