Caching: Don't Do the Same Work Twice
How caching works, from the ground up: the layers a request passes, the cache-aside read, the three write patterns, and why a full cache has to evict, each with its own visualizer.
A from-the-ground-up look at caching. Why it works (keep the answer close), where caches live (browser, CDN, app, database), the cache-aside read pattern (the hit and the miss), the three write patterns (through, back, around), and eviction (why a full cache must forget, with LRU). Everyday analogies, a hand-drawn diagram, and a visualizer per idea. The hard parts (invalidation, stampedes) and CDNs get their own pieces next.
We have followed a request all the way to a server: it found the address (DNS), got a server (the load balancer), and passed the front desk (the gateway), covered in the request-path pieces. Now the theme changes from getting there to going fast. The first and biggest trick: do not do the same work twice.
You do not walk to the shop every time you want a snack. You keep some in the kitchen. And the snacks you reach for all day, you keep in a drawer at your desk, within arm's reach. Why? Because the trip is slow, and you make it over and over for the same thing. Keep a copy close, and most of the time you skip the trip entirely.
That is caching. Somewhere behind your system is a slow, expensive step: a database query, a call to another service, a big computation. A cache keeps a copy of the answer somewhere fast and close, so the next time the same thing is asked, you hand back the copy in a fraction of the time. Get this right and it is the single biggest speedup in most systems.
Where caches live
Think about finding your keys. You check your pocket first, then your bag, then you walk back into the house. Each step is slower, so you always try the closest one first. A request looks for a cached answer the same way, checking a ladder of caches from closest to farthest:
- Your browser keeps recent files, so a page you just visited loads instantly, with no network at all.
- A CDN (a network of servers near users, its own piece coming up) keeps copies close to you geographically.
- The app server keeps a cache in memory or in a shared store like Redis, right next to the code.
- Even the database caches its own recent results.
At each rung, if the answer is there (a hit), you stop and return it. The closer the hit, the faster the response. Only if every cache misses do you pay the full price of the slow step.
The browser already has it: instant, no network at all.
Reading: the hit and the miss (cache-aside)
Zoom into one cache. The most common way to use it is called cache-aside, and it is just checking the drawer before walking to the storeroom.
A request comes in. First, look in the cache. If the answer is there, that is a hit: return it immediately, done. If it is not there, that is a miss: now you make the slow trip to the database, get the answer, and, before returning it, drop a copy into the cache. So the first person to ask pays full price, but everyone after them, until it is evicted, gets the fast hit.
The number that decides whether caching is worth it is the hit ratio: the fraction of requests that find their answer in the cache. If 95 out of 100 requests are hits, only 5 ever touch the database, and the whole system feels fast and the database barely breaks a sweat.
First request: the cache is empty, so it misses and takes the slow trip to the database, then drops a copy in the cache on the way back.
Writing to a cached system
Reads are the easy half. The moment you change something, the cache and the database can disagree, and you have to decide how to keep them honest. There are three common ways, and they trade safety against speed differently. Here they are, one at a time.
Write-through
Imagine writing every note on carbon paper: one stroke makes two identical copies at once, one for your desk and one for the file room. Write-through is exactly that. Every write lands in the cache and the database together, so the two never disagree and a read right after is always correct. The price is that the write waits for both to finish, so it is a little slower.
One write, two identical copies made at once. The cache and the database update together, so they never disagree. The write just waits a little longer for both.
Write-back
Now imagine jotting each change on a sticky note and filing it properly later, at the end of the day. Write-back (also called write-behind) writes to the fast cache immediately and copies it to the database a bit later, in the background. Writes feel instant, which is why it is used for write-heavy workloads. The risk is the gap: for a moment the cache and the database disagree, and if you lose the sticky notes before filing them, which is the cache crashing before it flushes, those writes are gone for good.
The write hits the fast cache and returns instantly. The database is not touched yet, so for a moment the two disagree.
Write-around
Now picture your desk and a filing cabinet across the room. The desk is small and fast, right in front of you; the cabinet is big and slow. Papers you use often, you keep on the desk. But a paper you have to store yet will almost certainly not open again soon, you skip the desk and file straight into the cabinet, so the desk stays clear for what you actually use. Write-around is that move: the write goes straight to the database (the cabinet) and never touches the cache (the desk). The cache stays full of genuinely hot items instead of one-off writes. So when does the cache get this value? Not on the write, ever. Only later, if and when someone reads the key: that read misses, takes the slow trip to the cabinet, and fills the desk on its way back, exactly the cache-aside read from earlier. From then on it is a normal fast hit. The catch is simply that first read: right after a write-around write, the value is not on the desk yet, so whoever asks for it first pays one slow miss to warm it.
The write goes straight to the database and skips the cache entirely. Nothing clutters the cache with data nobody has asked to read. So the cache does not hold this value yet.
Eviction: a cache has to forget
Here is the catch that makes caches interesting: a cache is small on purpose. Fast memory is expensive, so a cache holds only a slice of everything, maybe the hottest few percent. That means it is almost always full, and to make room for a new item it has to throw an old one out. That is eviction, and the question is which one to drop.
The usual answer is LRU, least recently used: drop whatever has gone untouched the longest, on the bet that what you used recently you will use again soon. Picture a small desk that holds four things: every time you touch one it goes to the front, and when you reach for a fifth, the thing at the back, the one you have ignored longest, goes back on the shelf. LRU is the workhorse, so it gets the picture here. There are other policies, LFU (drop the item used the fewest times) and TTL (expire items after a set time), and each is really its own idea worth its own analogy, so they get their own visualizers in a dedicated piece on eviction rather than a crammed mention here.
A 4-slot cache. Watch it fill, then evict the least recently used to make room.
Where this goes
That is the machinery of a cache: it sits in a ladder from your browser down to the database, you check it first and fill it on a miss (cache-aside), you keep it honest on writes (through, back, or around), and you evict the coldest item when it fills (LRU). A few things are still open, and each is its own piece. The other eviction policies, LFU and TTL, up close, each with its own visualizer. The famous hard part: keeping a cache from serving stale answers (cache invalidation), and the stampede that hits when a hot item expires and everyone rushes the database at once. And the cache that lives closest to your users: the CDN. Those are next.
