Build fluency in the vocabulary of preventing many concurrent requests from overwhelming an origin at once.
0 / 5 completed
1 / 5
At standup, a dev mentions a popular cache entry expiring and dozens of concurrent requests all missing the cache at once, each one hitting the slow origin database simultaneously to regenerate the exact same value. What is this problem called?
A cache stampede, or dogpile effect, happens when a popular cache entry expires and dozens of concurrent requests all miss the cache at once, each hitting the slow origin database simultaneously to regenerate the exact same value. Every request finding a valid cached value describes the opposite, normal case with no stampede at all. This simultaneous regeneration is what can suddenly overwhelm an origin database the moment a heavily requested cache entry expires.
2 / 5
During a design review, the team wants only the first request that misses the cache to actually regenerate the value, while every other concurrent request waits for that result instead of independently hitting the origin. Which capability supports this?
Request coalescing, sometimes called single-flight locking, lets only the first request that misses the cache actually regenerate the value, while every other concurrent request waits for that same result instead of independently querying the origin. Letting every concurrent request hit the origin independently multiplies the load on it by however many requests happened to arrive during that same cache-miss window. This coalescing is what keeps a cache miss from turning into a multiplied hit on the origin.
3 / 5
In a code review, a dev notices a cache entry is refreshed slightly before its expiration by a background process, so a request never actually experiences the moment the entry is fully expired and missing. What does this represent?
Early or background refresh updates a cache entry slightly before its expiration through a background process, so a request never actually experiences the moment the entry is fully expired and missing. Letting an entry sit expired until the next request arrives to regenerate it reintroduces exactly the window where a stampede of concurrent requests can pile onto the origin simultaneously. This proactive refresh is another common defense against a stampede, complementary to request coalescing.
4 / 5
An incident report shows the origin database was overwhelmed for several minutes right after a heavily requested cache entry expired, because every one of dozens of concurrent requests independently queried the database to regenerate the exact same value. What practice would prevent this?
Coalescing concurrent requests so only the first one regenerates the value, or refreshing the entry proactively before it expires, prevents exactly the kind of overwhelming, simultaneous origin load this incident describes. Letting every concurrent request independently query the origin multiplies that load by however many requests happen to arrive during the cache-miss window. This coalescing or proactive-refresh approach is a standard defense for any cache entry that's popular enough for its expiration to matter.
5 / 5
During a PR review, a teammate asks why the team adds request coalescing on top of caching instead of just trusting that a cache miss is rare enough not to need any special handling. What is the reasoning?
A popular cache entry's expiration can cause many concurrent requests to miss at exactly the same moment, especially for a heavily requested value, and without coalescing, each one independently overwhelms the origin regenerating the same result. Trusting that a miss is rare enough to ignore misses exactly this predictable failure mode for a popular entry. The tradeoff is the added implementation complexity of coordinating concurrent requests around a single in-flight regeneration rather than letting each one proceed independently.