The Honesty Check
DSpark's second machine, on its own: the AI's confidence scores are overconfident, like a forecaster who says 90% but is right 70% of the time, so DSpark corrects them to match reality.
Part four of the speculative decoding series, and the second of three that open up DSpark one machine at a time. This one is the honesty check. DSpark puts a confidence number on every guessed word, but those numbers run high, the same way a weather forecaster who says 90% is often right only 70% of the time. Calibration shaves them down until they match reality, without changing their order, which is what lets the next machine trust them. Plain words, a worked example, a diagram, and an auto-playing visualizer. Short, one idea only.
A good draft is only half the job. The big model still has to check the draft, and DSpark wants to be smart about how much checking to do. To do that, it first needs to know how likely each guessed word is to pass, so it puts a confidence number on every word. There is just one problem: those numbers lie. This is the honesty check.
A quick recap. In this series, a big slow AI is sped up by letting a small, fast model guess the next few words while the big one checks them in one go. Part three fixed the guesser: on its own it could mix two good replies, "of course" and "no problem," into "of problem," so DSpark gave it a cheat sheet that looks at the word just written and nudges the next one to fit. The draft is good now. This part is about the checking.
You already know this problem from the weather. A forecaster says "90% chance of rain," you grab an umbrella, and it stays dry. Do that enough times and you learn the truth: when he says 90, it really rains about 70 times out of 100. He is not lying on purpose. He is just overconfident. So you quietly shave his numbers down in your head. That shaving is the whole idea here.
The confidence number
After the cheat sheet picks each word, DSpark adds a second small output: a confidence. It is a number between 0 and 1 for how likely that word is to survive the big model's check. Higher means surer. The model learns these from lots of examples, so they are usually close to right.
The numbers run high
Here is the catch, and it is the forecaster all over again. Left alone, these numbers run high. The AI says 0.95 when the word really survives about 0.88 of the time. The paper measured this gap, and it is real: off by a few points, again and again.
Why a few points matter
You might shrug at "a few points." Here it matters a lot, because of what happens next. The last machine, coming in part five, does real math with these numbers. It multiplies them together to decide how much to check. If 0.95 secretly means 0.88, that decision goes wrong, and the big model checks too much or too little. A number that is only roughly right is not good enough once you start doing math with it.
The fix: cool the numbers down
So DSpark calibrates them, which is a big word for a simple thing: it shaves the numbers down until they match reality. Picture one dial. Turn it up and the high numbers come down. DSpark sets the dial using a batch of examples it saves for this, so that a 0.88 really does mean the word survives 88 times out of 100. Two things make this safe. The dial never changes the order, the surest word is still the surest, it only fixes the numbers. And it brings the gap down to about one point. The paper calls this Sequential Temperature Scaling; you can just call it cooling the numbers.
Watch it happen. The bars are the AI's confidence; the ticks are how often the guesses really survive. Watch the bars cool down to meet the ticks, while their order stays put:
The AI puts a confidence on each guessed word: 0.95, 0.90, 0.85, 0.80. It sounds sure.
Where this goes
That is machine two: honest confidence numbers you can actually trust. The code and the real math are open in DeepSpec, alongside the paper.
Now DSpark has a good draft and an honest number on every word. The last machine puts them to work. It decides, across everyone the big model is serving at once, exactly how many words are worth checking. That is the expediter, in part five.
