The Cheat Sheet
DSpark's first fix, on its own: how a tiny cheat sheet, the same trick your phone uses, makes each guessed word fit the one before it.
Part three of the speculative decoding series, and the first of three that open up DSpark one machine at a time. This one is the cheat sheet, the paper's Markov head: the same trick your phone's predictive text uses, where the word you just typed nudges the next one. It explains what a score even is, why the fast guesser is blind, and how a small nudge breaks the tie, with a worked example, a diagram, and an auto-playing visualizer. Short, one idea only.
DSpark's first fix, from part two, was to stop the fast guesser tripping over itself by letting each word see the one just before it. That trick has a tiny piece of machinery behind it, and it is the same thing your phone does when you type. Here it is on its own, start to finish.
You have met this machine already, on your phone. Type "good" and the keyboard offers "morning." Type "thank" and it offers "you." It is using the word you just typed to guess the next one. DSpark adds a tiny version of exactly this to its fast guesser. The paper calls it the Markov head; we will call it the cheat sheet.
To see why it helps, we need one idea first.
What a score is
Before a model picks a word, it gives every possible next word a number. Higher means it likes that word more. These are not percentages. They are just preference numbers, and they can be anything, even negative. A score only means something next to the other scores: what matters is which is higher. Only at the very end does the model turn the scores into probabilities and pick the top one.
The fast guesser is blind
Part two's fast pass writes a whole block of words in one shot, and it is quick because it fills every slot without looking at its neighbors. That is also its weakness. After someone says "Thanks!", two replies fit equally well, "of course" and "no problem." Asked for the second word blind, the fast pass scores "course" and "problem" almost the same, so it is a coin toss, and half the time it mixes them into "of problem."
The cheat sheet breaks the tie
The cheat sheet is simple. Given the word just chosen, it adds a nudge to the score of every possible next word. After "of," it nudges "course" up and "problem" down. After "no," it does the reverse. The final score is simply the blind score plus this nudge, and then you pick the highest.
Here it is with real words. The numbers are made up to show the idea; the real ones come out of the model, and all that matters is the tie and the nudge that breaks it. The blind pass scores "course" at 2.0 and "problem" at 2.1, so "problem" is barely ahead, a coin toss. The cheat sheet's row for "of" adds "course" plus 1.6 and "problem" minus 1.1. Now the totals are "course" at 3.6 and "problem" at 1.0, so "of course" wins clearly. Watch it play out, first for "of" and then for "no", so you can see the same two base scores lead to opposite picks that both make sense:
Someone says "Thanks!" The AI writes its reply one word at a time.
Why it stays cheap
There is one catch. A full cheat sheet would need a row for every word and a column for every word, and a real model knows a hundred thousand words or more, so that table would be huge. DSpark does not store the whole thing. It keeps a much smaller version, made from two little tables, that holds only the patterns that matter and takes up far less room. That is what lets the cheat sheet run without slowing the guesser down, which is the whole point: draft better at almost no extra cost.
(There is also a fancier version. Instead of looking at only the last word, it keeps a small memory of everything written so far, so its nudge can use all of it, not just the previous word. The paper calls this one the RNN head (short for recurrent neural network), which just means a small neural network that carries a memory forward as it reads. It is a little more accurate but a lot more work, so DSpark sticks with the simple cheat sheet.)
Where this goes
That is machine one: each word sees the last, through a nudge, so the fast draft stops tripping over itself. The real equations are open in DeepSpec, alongside the paper.
A good draft is only half the story. The big model still has to check it, and DSpark is careful about how much checking is worth doing. That starts by putting a confidence score on every guessed word, and those scores have a problem: they lie. Part four is the honesty check.
