Message Queues: Work That Waits Its Turn
How a message queue works from the ground up: decoupling the caller from the doer, load leveling a spike, a pool of competing workers, visibility timeouts and at-least-once delivery, and dead-letter queues, each with its own analogy and visualizer.
A from-the-ground-up look at message queues. Why you decouple the caller from the doer, how a queue absorbs a traffic spike (load leveling), how a pool of competing consumers scales the work, why a message is hidden and redelivered (visibility timeout, acknowledgments, at-least-once delivery and idempotency), and how a dead-letter queue quarantines poison messages. Everyday analogies, a hand-drawn diagram, a visualizer per idea, and what Amazon SQS and RabbitMQ actually do.
The caching arc was about going fast by not repeating work, from caching through CDNs. All of that happened inside the request, while the user waited. This piece is about the opposite move: work that should not happen while anyone is waiting at all. The tool for that is a queue.
Think about ordering at a busy coffee shop. You do not stand frozen at the register while the barista pulls your espresso, steams the milk, and pours the leaf. You order, the cashier scribbles a ticket and clips it to a rail, and you step aside. The barista works down the rail of tickets at their own pace, and the line keeps moving, because taking the order and making the drink were pulled apart.
That rail of tickets is a message queue. One side, the producer, drops a small note describing work to be done. The other side, the consumer, picks notes off the rail and does the work later, at its own speed. A message is just that note. Nobody stands and waits on anybody.
The queue absorbs a rush
A sudden rush of ten orders at once does not make the barista grow ten arms. The tickets just pile on the rail, the barista keeps working at a steady pace, and the rail soaks up the burst and drains again as things calm down.
That is the first thing a queue buys you: it decouples the rate of asking from the rate of doing. Producers can spike all they want; the consumer works at whatever steady rate it can manage; the queue sits in between as a buffer that holds the backlog. This is called load leveling. A traffic spike stops being a moment where your server melts or the user waits, and becomes a temporary backlog that quietly drains.
Worked example. A checkout service sends a receipt email after each order. On a normal minute, 50 orders come in. During a flash sale, 5,000 orders land in a single minute, but the email sender can only push about 100 emails a second. Without a queue, each checkout request blocks on the slow email step and times out, and the whole checkout falls over. With a queue, checkout drops 5,000 "send receipt" notes in a blink and returns the confirmation page instantly; the email worker then drains those 5,000 over about 50 seconds. The customer saw "order confirmed" immediately, and the receipt arrived a moment later.
A calm minute: about 50 orders come in, and the email worker keeps up. The queue stays almost empty.
Add hands, not arms
When the rail gets long, you do not beg one barista to move faster forever. You put a second and a third barista on the same rail. Each one grabs the next ticket, each ticket is made by exactly one of them, and three baristas clear the rail about three times as fast.
That is the second thing a queue buys you. Several consumers read from the same queue, a setup called competing consumers. Each message is handed to exactly one worker in the pool, so the work is spread out, never duplicated across the pool, and to handle more load you simply add more workers. A queue is what turns a spike into something you scale sideways, by adding hands.
Worked example. One email worker at 100 a second takes about 50 seconds to drain 5,000 receipts. Add four more workers, five in total, all pulling from the same queue, and the 5,000 drain in about 10 seconds. No code change, just more consumers on the same rail.
One worker pulling the rail: 5,000 receipts at about 100 a second take roughly 50 seconds to clear.
What if a worker drops the ticket?
A barista grabs a ticket and, halfway through the drink, their machine jams and they wander off to fix it. If that ticket is simply gone, the customer never gets their coffee. So the rule is: a grabbed ticket is not thrown away, it is only flipped face-down for a while. If the barista finishes and marks it done, the ticket is discarded. If they do not mark it done within that while, the ticket flips face-up again and another barista makes the drink.
That is exactly how a queue handles a crash. When a consumer receives a message, the queue does not delete it; it hides the message for a set time called the visibility timeout. The consumer does the work and then acknowledges the message, which deletes it. If the consumer crashes before acknowledging, the timeout expires, the message becomes visible again, and another consumer picks it up. This is why a queue gives at-least-once delivery: a message is never lost, but it can be delivered more than once, either because a worker was slow, or because it crashed after doing the work but before acknowledging. So consumers should be idempotent, meaning doing the same work twice is safe. Amazon SQS sets the visibility timeout to 30 seconds by default and allows up to 12 hours.
Worked example. Worker A receives "send receipt for order 812," and the message hides for 30 seconds. A sends the email but crashes before deleting the message. At the 30-second mark the message reappears, worker B receives it, and the receipt goes out a second time. The customer gets two receipts: annoying, not a disaster, and avoidable if the sender first checks "have I already sent order 812?" before sending.
A message waits on the queue, visible for any worker to take.
The ticket nobody can make
Someone orders "a large glass of nothing." No barista can make it. If that ticket keeps flipping face-up and getting retried forever, it clogs the rail and starves the real orders behind it. So after a few honest attempts, you take it off the rail and drop it in a separate "problems" tray for a person to look at later.
Some messages are like that: a malformed payload, a reference to a record that was deleted, a plain bug. Left alone, they redeliver forever and burn your workers on the same doomed task. This is called a poison message. A dead-letter queue (DLQ) is a separate queue where a message is moved once it has been received too many times. In SQS you set a redrive policy with a maxReceiveCount, say 5; after the fifth failed attempt, the message is moved to the DLQ instead of back onto the main queue. The main queue keeps flowing, and the bad messages wait in the DLQ for someone to inspect.
Worked example. Order 999 points at a product that was deleted, so the receipt worker throws an error every single time. With a maxReceiveCount of 5, it is tried five times over a few minutes, then moved to the DLQ. An engineer later reads the DLQ, spots the deleted-product bug, fixes it, and replays the message. Without the DLQ, that one order would have kept a worker busy failing, forever.
Order 999 points at a deleted product, so the worker throws every time. Attempt 1 of 5: it fails and reappears on the queue.
What actually runs
A few names cover most of what you will meet, one from each big cloud plus one you run yourself.
Google Cloud Tasks is Google Cloud's managed queue. You add a task, often an HTTP request, and the service reliably dispatches it to a worker at any HTTP endpoint (Cloud Run, GKE, Compute Engine, even an on-premises server), with configurable rate limits and automatic retries that back off between attempts. It is at-least-once, and Google's own docs are blunt that "in some rare circumstances, multiple task execution is possible, so your code must ensure that there are no harmful side-effects of repeated execution," which is the idempotency point a later piece is built on.
Amazon SQS (Amazon Web Services) is a fully managed queue. Standard queues give at-least-once delivery, best-effort ordering, and nearly unlimited throughput. FIFO queues give exactly-once processing and strict ordering, at lower throughput, for when duplicates or out-of-order handling would be wrong (say, a sequence of account balance changes). The visibility timeout defaults to 30 seconds and can be set up to 12 hours, and dead-lettering is a redrive policy with a maxReceiveCount.
RabbitMQ is a broker you run yourself, with explicit acknowledgments, dead-letter exchanges, and richer routing. (Kafka is often used like a queue too, but it is really a durable log, which is its own piece.)
One honest caveat. A queue absorbs a spike; it does not fix a permanent imbalance. If producers outrun consumers forever, the queue grows without bound and the backlog never clears. The queue buys you time to catch up, not a license to ignore the math. Watch the queue depth: if it only ever climbs, add consumers or slow the producers down.
A queue hands each message to exactly one worker, which is the whole point: the work gets done once. But sometimes you want the opposite, one event that many different services all need to hear, like an order being placed while billing, inventory, email, and analytics all care at once. That is pub/sub, the fan-out cousin of the queue, and it is the next piece.
Sources: Amazon SQS visibility timeout; Amazon SQS FAQs; Understand Cloud Tasks (Google Cloud).
