Systems Engineering

Writing on systems engineering

Notes and how-tos on distributed systems, billing platforms, reliability, and the Java/backend craft behind them. Written to help other engineers.

Portrait of Sachin Gupta rendered in binary

Articles

26 articles

Writing· System Design
Delivery Guarantees: At-Least-Once, Exactly-Once, and Idempotency
Once messages can be retried, a new problem appears: a payment could be charged twice, or lost entirely. This piece is about the three promises a messaging system can make (at-most-once, at-least-once, exactly-once), why exactly-once is really achieved by making the work idempotent, and why order matters. It closes the messaging arc.
A from-the-ground-up look at delivery guarantees, the last piece of the messaging arc. The three promises a system can make: at-most-once (a message may be lost but never duplicated), at-least-once (never lost but may arrive more than once), and exactly-once (once and only once). Why real systems run at-least-once and reach exactly-once in practice by making each operation idempotent, so a retried request has no extra effect, the way a Stripe idempotency key prevents a double charge. Why ordering matters, and what Amazon SQS standard and FIFO queues, Apache Kafka, and application-level idempotency keys actually guarantee. Everyday analogies, a hand-drawn diagram, and a visualizer per idea.
- System Design
By Sachin Gupta17 Jun 202610 min read
Writing· System Design
The Log: One Record Everyone Can Replay
A queue empties as it is read, so a note is gone once someone handles it. A log does the opposite: it keeps every record, in order, and lets anyone read back from any point, even a service that starts up next year. That is the idea behind Kafka. Here is the append-only log, offsets and replay, partitions and ordering, and consumer groups.
A from-the-ground-up look at the event log, the idea behind Apache Kafka. A log is append-only and keeps every record for a retention window whether or not anyone has read it, the opposite of a queue that empties. Each record has an offset, and every reader keeps its own offset, so a new service can replay from the beginning and a service that was down can catch up from where it left off. Partitions split the log so many producers and consumers work at once while order is kept within a partition, and consumer groups either split the work or each receive a full copy. Everyday analogies, a hand-drawn diagram, a visualizer per idea, and what Kafka really does.
- System Design
By Sachin Gupta16 Jun 202610 min read
Writing· System Design
Pub/Sub: One Event, Many Listeners
A queue hands each note to exactly one worker. Sometimes you need the opposite: one thing happens and many services all need to hear it. That is pub/sub. Here is fan-out, how it differs from a queue, the queue-per-subscriber pattern that makes it durable, and how a notification system is built on it.
A from-the-ground-up look at pub/sub. Fan-out: a publisher posts one event to a topic and every subscriber gets its own copy, decoupled from the publisher. Why that is the opposite of a work queue (distribute vs broadcast), the queue-per-subscriber pattern (SNS to SQS) that gives each subscriber isolation and its own retries, and how a notification system fans an event out to email, SMS, and push. Everyday analogies, a hand-drawn diagram, a visualizer per idea, and what Amazon SNS, Google Cloud Pub/Sub, and Redis pub/sub really do.
- System Design
By Sachin Gupta15 Jun 20268 min read
Writing· System Design
Message Queues: Work That Waits Its Turn
Not every job should happen while the user waits. A queue lets one side drop off work and walk away, and another side pick it up and do it later, at its own pace. Here is how a message queue absorbs a rush, spreads work across many workers, survives a crash, and quarantines the messages nobody can handle.
A from-the-ground-up look at message queues. Why you decouple the caller from the doer, how a queue absorbs a traffic spike (load leveling), how a pool of competing consumers scales the work, why a message is hidden and redelivered (visibility timeout, acknowledgments, at-least-once delivery and idempotency), and how a dead-letter queue quarantines poison messages. Everyday analogies, a hand-drawn diagram, a visualizer per idea, and what Amazon SQS and RabbitMQ actually do.
- System Design
By Sachin Gupta14 Jun 20269 min read
Writing· System Design
CDNs: The Cache Nearest Your Users
Every cache so far lived next to your database. The last one lives next to your users, thousands of miles closer than your servers. A CDN keeps copies of your content in cities all over the world, so the trip is short. Here is how it works, one idea at a time, each with its own analogy and visualizer.
A from-the-ground-up look at CDNs, the cache nearest your users. Why serving from a nearby edge beats a round trip to a far origin, how content reaches the edge (pull vs push), how long the edge keeps it (TTL and Cache-Control, max-age vs s-maxage, stale-while-revalidate), the cache key and how marketing query params fragment the cache, purging and versioned URLs when content changes, and the origin shield that turns a thundering herd of edge misses into one origin fetch. Everyday analogies, a hand-drawn diagram, and a visualizer per idea.
- System Design
By Sachin Gupta13 Jun 202610 min read
Writing· System Design
The Cache Stampede: When a Hot Key Expires
A cache that works beautifully at steady state can still fall over in a single instant. The moment one very popular key expires, thousands of requests miss at the same time and stampede the database at once. Here is the problem, and the three ways to keep that spike from taking the system down, each with its own analogy and its own visualizer.
A dedicated look at the cache stampede, or thundering herd. Why a single hot key expiring turns thousands of harmless cache hits into a synchronized flood of database queries in the same instant, and why that spike can take a system down even when average load is fine. Then the three standard defenses, each analogy-first with a visualizer: locking and single-flight (only the first miss refetches, the rest wait), stale-while-revalidate (keep serving the stale value while one background refresh runs), and probabilistic early expiration (XFetch, where one request refreshes early before the herd forms), plus jittered TTLs for synchronized warm-ups.
- System Design
By Sachin Gupta12 Jun 20268 min read
Writing· System Design
Cache Invalidation: Keeping a Cache Honest
A cache miss is slow but correct. A stale cache hit is fast and wrong, and wrong is worse. Invalidation is the work of making sure the copy in the cache still matches the truth in the database. Here are the ways to do it, each with its own analogy and its own visualizer.
A dedicated look at cache invalidation, the hard half of caching. Why a stale hit is worse than a miss, then the four strategies to keep a cache honest: TTL expiry (bounded staleness), write-through (update on write), invalidate-on-write (delete the key and let the next read refill), and versioned or immutable keys (bump the version, never edit in place). Everyday analogies, a hand-drawn stale-window timeline, and visualizers for the delete-on-write cycle and versioned keys. Why the races make it genuinely hard, and a tee-up to the stampede.
- System Design
By Sachin Gupta11 Jun 20268 min read
Writing· System Design
Cache Eviction: What to Forget When It's Full
A cache is small on purpose, so it is almost always full. To store one more thing it has to throw one thing out, and the whole art is choosing which. Three policies answer that question in three different ways: LRU by recency, LFU by frequency, TTL by age. Here is each one, with its own analogy and its own visualizer.
A dedicated look at cache eviction. Why a cache must forget, then the three policies each with its own everyday analogy and interactive visualizer: LRU (recency, a desk), LFU (frequency, a bestseller shelf), and TTL (age, milk with an expiry date). A worked example for each, a hand-drawn diagram showing the same cache evict three different items, guidance on which to use when, and what Redis and memcached run in production.
- System Design
By Sachin Gupta10 Jun 20269 min read
Writing· System Design
Caching: Don't Do the Same Work Twice
The fastest work is the work you never do. A cache keeps a copy of an answer close by, so the next time someone asks, you hand it over instead of computing it all again. Here is how caches actually work: where they live, how reads and writes flow through them, and why they have to forget things.
A from-the-ground-up look at caching. Why it works (keep the answer close), where caches live (browser, CDN, app, database), the cache-aside read pattern (the hit and the miss), the three write patterns (through, back, around), and eviction (why a full cache must forget, with LRU). Everyday analogies, a hand-drawn diagram, and a visualizer per idea. The hard parts (invalidation, stampedes) and CDNs get their own pieces next.
- System Design
By Sachin Gupta9 Jun 20269 min read
Writing· System Design
How a Load Balancer Sticks
Sometimes you want the opposite of spreading traffic evenly: send the same client, or the same request, to the same server every time. That is stickiness, and it runs from a simple hash all the way to the elegant trick of consistent hashing.
A dedicated look at the sticky load-balancing rules, the ones that send the same key to the same server. Why you want it (sessions and caches), then each rule with its own analogy and visualizer: source-IP hash, URL hash, the reshuffling problem, consistent hashing (the ring), Maglev (a fast table), and cookie-based sticky sessions. Part two of two; part one covered the even-spreading rules.
- System Design
By Sachin Gupta8 Jun 20268 min read

Articles

Delivery Guarantees: At-Least-Once, Exactly-Once, and Idempotency

The Log: One Record Everyone Can Replay

Pub/Sub: One Event, Many Listeners

Message Queues: Work That Waits Its Turn

CDNs: The Cache Nearest Your Users

The Cache Stampede: When a Hot Key Expires

Cache Invalidation: Keeping a Cache Honest

Cache Eviction: What to Forget When It's Full

Caching: Don't Do the Same Work Twice

How a Load Balancer Sticks