Distributed systems

Queues turn spikes into lines

A queue does not make work disappear. It makes work wait in a place you can observe.

~5 minTech
scroll
01 · the shape

Synchronous work breaks at the edge

When every request tries to do every job immediately, spikes become timeouts. A queue lets the user-facing path accept work quickly and lets workers drain it at a controlled pace.

worker · backlog

The useful signal is usually already there.

system steady

Systems stories get more interesting when you can feel the pressure move through the stack.

02 · the move

Design for waiting

Queued work needs idempotency, retries, dead-letter handling, progress visibility, and a clear answer to how old is too old.

jobs/send-receipt.js

Idempotency keeps retries from becoming duplicate side effects.

03 · the check

Watch age, not just depth

Queue depth tells you how much work exists. Oldest job age tells you what users are experiencing.

reliability control room
brittle
resilient0
brittle: the system works only while the weather is perfect

Flip the switches and watch the story turn into a tiny operating model.

0core alerts: age and failure rate
0dead-letter path for poison jobs
0unbounded retries
04 · the takeaway

A queue is a promise to finish later

Use queues when later is acceptable, then make later observable enough that nobody has to guess.

source trail