In the Machine Room
The last five parts were the recipe. This one is the kitchen at dinner rush: the two engineering tricks that get DSpark running inside a real serving system, and the speedups it delivers there.
Part six of the speculative decoding series, the finale. Parts one to five were the idea. This one is the engineering: two tricks that get DSpark running inside a live serving system. First, do not ship the whole dictionary of scores between machines while training the guesser, ship the compact summary and do the last step locally. Second, a scheduler that cannot wait, because real GPU speed rises in jagged steps and the pipeline cannot pause, so it predicts the batch size ahead of time. Plus the real production speedups. Plain words, an analogy, a diagram, and two auto-playing visualizers.
Parts one to five were the recipe: draft ahead, draft better, check smarter. On paper it is clean. But a recipe and a working kitchen at 8pm on a Friday are two different things. Getting DSpark to run inside a live serving system, feeding thousands of people at once, takes two more tricks that have nothing to do with the idea and everything to do with the machine. This is the machine room.
A quick recap. A big slow AI is sped up by having a small model guess the next few words while the big one checks them. Across the series, DSpark added three machines: a cheat sheet so the draft fits together, an honesty check so its confidence is trustworthy, and the expediter so the big model only checks what is worth checking. That is the whole idea. Now: how do you run it for real?
Why the recipe is not the whole story
The neat algorithm quietly assumes a perfect machine: data moves instantly between computers, and the hardware speeds up smoothly as you give it more work. Real hardware does neither. Two things bite. One shows up while training the little guesser. The other shows up while running the scheduler live.
Trick one: do not ship the whole dictionary
To train the little guesser, it needs to see what the big model would have said. The obvious way is to send the big model's full answer over to the guesser's computer. But "the full answer" is a score for every possible word, and a model knows a hundred thousand words or more. Sending a hundred thousand numbers for every single word, between computers, is a firehose that clogs everything up.
So DSpark sends the compact version instead. The big model does almost all of its thinking and produces a short internal summary just before the final step, one modest list of numbers instead of a score for every word. DSpark ships only that summary, and does the last step, turning the summary into word scores, locally on the guesser's computer, only for the words it actually needs. Same information, a small fraction of the traffic. It is the difference between mailing someone the whole phone book and mailing them the one page they asked for.
To train the small guesser, the big model has to send its answer over to the guesser's computer, for every word it teaches.
Trick two: a scheduler that cannot wait
The expediter from part five finds the sweet spot by assuming throughput rises smoothly as the batch fills. Real GPUs are not smooth. Their speed goes up in jumps, like a staircase instead of a ramp, so the neat "climb to the peak" is bumpy.
Worse, to stay fast a GPU runs on a fixed plan it cannot pause. It needs to know the size of the next batch before the current one has even finished. So DSpark cannot work out the perfect batch size at the last second, because by then it is too late. Instead it predicts: it uses the confidence numbers from two steps earlier to guess how big the next batch should be, and sets it up in advance. It is a chef who starts prepping the next order before the ticket lands, because he cannot stop the line. A small guess, made early, keeps the whole kitchen moving.
Part five picks the batch size by finding where the GPU's speed peaks. To do that, it assumes speed changes smoothly as the batch grows.
Does it actually work
Yes, and this is the part that matters. DSpark was not just tested on benchmarks. It was deployed inside DeepSeek's own V4 serving system, under real user traffic. Against the previous production setup, at the same total throughput, it made each user's text come out 60% to 85% faster on one model tier and 57% to 78% faster on another. And under strict speed limits, where the old setup's capacity fell apart, DSpark held up, opening service levels that were not reachable before. The recipe was good; the machine-room work is what let it ship.
Where this goes
That closes the series. Five parts of idea and one of engineering: a fast draft that stays coherent, an honest confidence on every word, a scheduler that spends the big model only where it pays, and the plumbing to run all of it at scale, with the output identical to the slow way. The checkpoints, the training code, and the real equations are open in DeepSpec, alongside the paper.
