All activity

Pub/Sub: One Event, Many Listeners

A queue hands each note to exactly one worker. Sometimes you need the opposite: one thing happens and many services all need to hear it. That is pub/sub. Here is fan-out, how it differs from a queue, the queue-per-subscriber pattern that makes it durable, and how a notification system is built on it.

A from-the-ground-up look at pub/sub. Fan-out: a publisher posts one event to a topic and every subscriber gets its own copy, decoupled from the publisher. Why that is the opposite of a work queue (distribute vs broadcast), the queue-per-subscriber pattern (SNS to SQS) that gives each subscriber isolation and its own retries, and how a notification system fans an event out to email, SMS, and push. Everyday analogies, a hand-drawn diagram, a visualizer per idea, and what Amazon SNS, Google Cloud Pub/Sub, and Redis pub/sub really do.

• System Design

MCP on the Wire

JSON-RPC, the lifecycle handshake, and the two transports, stdio and Streamable HTTP.

How MCP messages actually move. The JSON-RPC base, the initialize handshake and capability negotiation that set up every connection, and the two standard transports, stdio for local servers and Streamable HTTP for remote ones, plus why the old HTTP-plus-SSE transport was replaced.

• MCP

Message Queues: Work That Waits Its Turn

Not every job should happen while the user waits. A queue lets one side drop off work and walk away, and another side pick it up and do it later, at its own pace. Here is how a message queue absorbs a rush, spreads work across many workers, survives a crash, and quarantines the messages nobody can handle.

A from-the-ground-up look at message queues. Why you decouple the caller from the doer, how a queue absorbs a traffic spike (load leveling), how a pool of competing consumers scales the work, why a message is hidden and redelivered (visibility timeout, acknowledgments, at-least-once delivery and idempotency), and how a dead-letter queue quarantines poison messages. Everyday analogies, a hand-drawn diagram, a visualizer per idea, and what Amazon SQS and RabbitMQ actually do.

• System Design

Rerankers

Retrieve fast and roughly, then read the top few closely and reorder. The second pass that fixes the first.

The closing part of the search series. First-stage retrieval is fast but ranks crudely. A reranker takes the top candidates and scores each one by reading the query and document together with a cross-encoder, then reorders. This covers the bi-encoder vs cross-encoder split, why it must be a second stage, and where it lives in production, with an interactive retrieve-then-rerank demo.

• Search

CDNs: The Cache Nearest Your Users

Every cache so far lived next to your database. The last one lives next to your users, thousands of miles closer than your servers. A CDN keeps copies of your content in cities all over the world, so the trip is short. Here is how it works, one idea at a time, each with its own analogy and visualizer.

A from-the-ground-up look at CDNs, the cache nearest your users. Why serving from a nearby edge beats a round trip to a far origin, how content reaches the edge (pull vs push), how long the edge keeps it (TTL and Cache-Control, max-age vs s-maxage, stale-while-revalidate), the cache key and how marketing query params fragment the cache, purging and versioned URLs when content changes, and the origin shield that turns a thundering herd of edge misses into one origin fetch. Everyday analogies, a hand-drawn diagram, and a visualizer per idea.

• System Design

ToolContractGate

Runtime contract gate for agent tools

A runtime gate that evaluates every agent tool call against a versioned contract derived from an approved OpenAPI artifact, blocking or escalating calls when the live tool surface drifts.

• Java 21
• Spring Boot 3
• Kubernetes
• OpenAPI

• AI Agent Harness

The Cache Stampede: When a Hot Key Expires

A cache that works beautifully at steady state can still fall over in a single instant. The moment one very popular key expires, thousands of requests miss at the same time and stampede the database at once. Here is the problem, and the three ways to keep that spike from taking the system down, each with its own analogy and its own visualizer.

A dedicated look at the cache stampede, or thundering herd. Why a single hot key expiring turns thousands of harmless cache hits into a synchronized flood of database queries in the same instant, and why that spike can take a system down even when average load is fine. Then the three standard defenses, each analogy-first with a visualizer: locking and single-flight (only the first miss refetches, the rest wait), stale-while-revalidate (keep serving the stale value while one background refresh runs), and probabilistic early expiration (XFetch, where one request refreshes early before the herd forms), plus jittered TTLs for synchronized warm-ups.

• System Design

Hybrid Search

Keyword search and vector search fail in opposite directions. Run both, merge the results, and you keep the strengths of each.

Part six of the search series. BM25 is strong on exact terms and blind to synonyms; vector search is the reverse. Hybrid search runs both and fuses the two ranked lists. The hard part is combining incompatible scores, which Reciprocal Rank Fusion (RRF) sidesteps by using ranks instead. Worked examples plus an interactive RRF explorer.

• Search

AgentTrustCI

Build-time trust tests for LLM agents

A JUnit 5 framework that catches enterprise AI-agent failures before release by turning allowed-tools, grounded-answers, and golden-trajectory expectations into regression tests that run in CI.

• Java 21
• JUnit 5
• LangChain4j

• AI Agent Harness
• LLM Ops & Eval

Cache Invalidation: Keeping a Cache Honest

A cache miss is slow but correct. A stale cache hit is fast and wrong, and wrong is worse. Invalidation is the work of making sure the copy in the cache still matches the truth in the database. Here are the ways to do it, each with its own analogy and its own visualizer.

A dedicated look at cache invalidation, the hard half of caching. Why a stale hit is worse than a miss, then the four strategies to keep a cache honest: TTL expiry (bounded staleness), write-through (update on write), invalidate-on-write (delete the key and let the next read refill), and versioned or immutable keys (bump the version, never edit in place). Everyday analogies, a hand-drawn stale-window timeline, and visualizers for the delete-on-write cycle and versioned keys. Why the races make it genuinely hard, and a tee-up to the stampede.

• System Design