DevOpsInterviews logo
← DevOps System Design & Architecture / 18
mediumSwiggyUberLinkedIn

Design notifications for a food delivery app. Order updates cannot be late; coupon blasts cannot spam.

Two contracts in one system: an OTP with a 30-second promise and a sale announcement reaching fifty million phones without burning providers. Priority lanes and double-layer dedup decide the grade.

Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Two contracts, two lanes: transactional notifications get dedicated capacity and a tight latency SLA, promotional sends get rate-limited bulk delivery shaped by preferences and frequency caps. Dedup runs twice, on event identity and per-user volume, and provider throughput is the wall every burst plan hits.

How to approach it

Split the requirement by class before designing anything: which notifications are contractual (OTP, order status) and which are discretionary (offers). Every architectural choice, queue separation, dedup placement, burst behaviour, follows from that classification.

A strong answer

Classes and their contracts:

TransactionalPromotional
ExamplesOTP, rider assigned, order delayedSale started, win-back nudges
Latency targetp99 under 30 secondsMinutes to hours, tolerable
Under overloadProtected, pre-allocated capacityDelayed or dropped first
Volume governed byEvents as they happenPreferences, caps, campaigns

Dedup, twice. Event-level: consumers key on a stable event ID or(order_id, transition_version), since status text can recur so a replayed Kafka partition cannot send "rider assigned" twice. User-level: a suppression service enforcing per-user promo budgets (say four per day), quiet hours in the user's local timezone, and cross-channel suppression so an in-app message suppresses the redundant push. Most duplicate-notification incidents trace to a missing idempotency key on the consumer, not to exotic races.

Fan-out and channels. One event fans out to channel jobs: push first (cheapest, richest), SMS according to an explicit deadline/fallback policy. Provider push acceptance does not prove device delivery; missing receipts are ambiguous and cross-channel fallback can produce duplicate user notifications, email for content-heavy formats. Per-user channel preferences and device registries resolve here; templates render centrally so localization and deep links stay consistent. Validate deep links at send time; notifying users about an expired offer converts a campaign into support tickets.

Burst arithmetic. A flash-sale push to 50 million opted-in users cannot happen at once: over a 30-minute ramp that is roughly 28,000 sends per second sustained, and provider limits (FCM and APNs batching, connection counts, SMS aggregator pipes) dictate the curve. Pre-warm connections, spread the blast, watch per-provider success rates, and keep a circuit breaker ready to reroute or pause. Promotional volume always yields before transactional; that priority is enforced by separate queues and worker pools, not by hope.

Compliance and feedback. SMS in India means scrubbing against DND/TRAI preferences; unsubscribes apply globally, not per campaign. Delivery receipts, opens, and uninstall events flow back into suppression logic: rising app-disable rates after a campaign are the signal that marketing oversent, and they predict tomorrow's unreachable users.

What interviewers probe next

"The SMS aggregator is down during dinner rush." Circuit breaker per provider, brief queue hold for contractual sends, failover to a secondary aggregator; the promotional lane pauses entirely. Order-critical messaging degrades last.

"A user received the same push twice. Walk me through it." Check event-level idempotency first, then fan-out job uniqueness, then provider-level retries colliding with client-side display; each layer has a distinct fix.

"How do you validate burst readiness before the sale?" Shadow launch to one percent of the audience, measure provider throughput and error mixes, then ramp; plus load tests proving the transactional lane holds its SLA while the blast runs.

Common mistakes

One shared FIFO queue where a 50-million-user campaign parks in front of OTPs, the canonical version of this outage.

Treating push as free volume until uninstall curves prove otherwise; reach decays permanently.

Rendering quiet hours per server timezone instead of per user, blasting Mumbai at 3am local time.

That one was free, and so are 18 answers per topic without an account. Signing in doubles that to 28, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

Nothing here yet. Say how you would answer it.