Vol 1 · Ch. 1
Scale From Zero to Millions of Users
The canonical scaling progression — introduce complexity only when the current tier is the bottleneck.
Core Thesis
Scale each tier independently. Every new layer solves a specific bottleneck. Don't over-engineer before you need it.
Scaling Stages
| Stage | Key Addition | Bottleneck Solved |
|---|---|---|
| Single Server | Everything on one box | Starting point — no redundancy |
| Separate DB | Web + DB split | Independent tier scaling |
| Load Balancer | LB → web server pool | Web tier SPOF; horizontal scaling |
| DB Replication | Master (write) + Slaves (read) | Read throughput; DB high availability |
| Cache (Redis) | Cache tier in front of DB | Repeated expensive DB reads |
| CDN | Edge servers for static assets | Global latency for JS/CSS/images |
| Stateless Web | Session → shared store (Redis) | Sticky sessions; enables autoscaling |
| Multi-DC | GeoDNS routing | Regional failover; global latency |
| Message Queue | Async producer/consumer | Tight coupling; burst absorption |
| DB Sharding | Horizontal DB partitioning | Write throughput ceiling |
| Microservices | Decompose monolith | Monolith scalability limits |
Vertical vs Horizontal Scaling
| Dimension | Vertical (Scale Up) | Horizontal (Scale Out) |
|---|---|---|
| Mechanism | Add CPU/RAM to server | Add more servers to pool |
| Ceiling | Hard limit — can't add unlimited resources | Virtually unlimited |
| Failover | Single point of failure | Redundant — others absorb traffic |
| Cost | Exponential at high specs | Commodity hardware, linear cost |
| Best for | Low traffic, simple ops | Production-grade, HA systems |
8 Laws of Scale — Memorise These
1) Stateless web tier · 2) Redundancy at every tier · 3) Cache aggressively · 4) Multi-DC · 5) CDN for statics · 6) Shard the DB · 7) Split tiers into services · 8) Monitor & automate
Cache Considerations
| Topic | Detail |
|---|---|
| When to cache | Read frequently, modified infrequently. Never use as primary data store — volatile memory. |
| TTL / Expiry | Too short → DB hammering (cache stampede). Too long → stale data. Balance based on freshness needs. |
| SPOF risk | Single cache server = SPOF. Multiple cache nodes across AZs. Overprovision memory by ~20%. |
| Eviction policy | LRU (most common), LFU (frequency), FIFO. Choose based on access pattern. |
| Cache stampede | Many requests hit DB simultaneously on miss. Mitigate with mutex lock, probabilistic early expiration, or cache warming. |
Vol 1 · Ch. 2
Back-of-the-Envelope Estimation
Numbers every Staff engineer must know cold. Do these estimates before proposing architecture.
Latency Numbers
| Operation | Latency | Relative |
|---|---|---|
| L1 cache reference | ~0.5 ns | 1× |
| L2 cache reference | ~7 ns | 14× |
| RAM access | ~100 ns | 200× |
| Redis GET | ~1 ms | 2M× |
| SSD random read | ~100 µs | 200K× |
| DB query (indexed) | ~1–10 ms | 2–20M× |
| Network same DC | ~0.5 ms | 1M× |
| Network cross-continent | ~150 ms | 300M× |
| HDD disk seek | ~10 ms | 20M× |
Estimation Rules
Time Constants
Seconds per day: 86,400 ≈ 100K
Seconds per month: ~2.5M
Seconds per year: ~31.5M
Seconds per month: ~2.5M
Seconds per year: ~31.5M
QPS Estimation
1M DAU → ~12 QPS avg
Peak = 2–3× avg
1B DAU → ~12K QPS avg
Peak = 2–3× avg
1B DAU → ~12K QPS avg
Storage Sizes
1 char = 1 B · 1 int = 4 B
1 photo ≈ 300 KB
1 min HD video ≈ 150 MB
1 TB = 10¹² B · 1 PB = 10¹⁵ B
1 photo ≈ 300 KB
1 min HD video ≈ 150 MB
1 TB = 10¹² B · 1 PB = 10¹⁵ B
Twitter Example
300M MAU, 100M DAU
50% mobile · 10% post daily
Write QPS ≈ 1,150 · Peak ≈ 3.5K
Storage: 10% × 100M × 250B ≈ 2.5 GB/day
50% mobile · 10% post daily
Write QPS ≈ 1,150 · Peak ≈ 3.5K
Storage: 10% × 100M × 250B ≈ 2.5 GB/day
Vol 1 · Ch. 3
System Design Interview Framework
A 4-step process to structure any system design answer. Follow this every single time.
Golden Rule
Never jump to solution. Always clarify scale first. "Design Twitter for 10 users" vs "1B users" are completely different systems.
4-Step Process
| Step | Time | What to Do |
|---|---|---|
| 1. Understand the problem | 3–10 min | Clarify features, scale, constraints. Ask: DAU? Read/write ratio? Latency SLA? Consistency requirements? |
| 2. High-level design | 10–15 min | Draw rough architecture. Get buy-in before diving deep. Cover client → LB → API → DB → cache → queue. |
| 3. Design deep dive | 10–25 min | Focus on hardest parts: data model, critical APIs, scale bottlenecks, failure modes. Follow interviewer's lead. |
| 4. Wrap up | 3–5 min | Summarize, discuss bottlenecks, operational concerns (monitoring, deployment, failure scenarios), future improvements. |
Clarifying Questions to Always Ask
Scale
How many users? DAU? Reads vs writes? Peak QPS? SLA latency?
Features
What features are in scope? Mobile vs web? APIs only or full system?
Data
How much data? Retention period? Consistency requirements? ACID needed?
Constraints
Existing tech stack? Budget? Latency SLA? Multi-region requirements?
Staff-Level Signal
Frame answers as an evolution: "At 1K users, X is fine. At 100K I'd add Y. At 1M I'd add Z." This shows you think incrementally, not in one giant upfront design.
Vol 1 · Ch. 4
Design a Rate Limiter
Control traffic flow. Prevent abuse. Protect downstream services. Know all 5 algorithms cold.
The 5 Algorithms
Token Bucket
Bucket holds N tokens; refills at rate R/s. Each request consumes 1 token. Reject if empty. Used by Amazon, Stripe.
PROS
Burst-tolerant. Memory efficient.
CONS
Two params hard to tune.
Leaky Bucket
FIFO queue. Requests processed at fixed output rate. Drop if queue full. Used by Shopify.
PROS
Stable outflow rate.
CONS
Bursts fill queue; recent requests dropped.
Fixed Window Counter
Divide time into fixed windows. Count requests per window. Reject if over limit.
PROS
Simple. Memory efficient.
CONS
Burst at window boundary doubles effective rate.
Sliding Window Log
Store timestamp of each request. Count timestamps in last N seconds. Reject if over limit.
PROS
Accurate. No boundary problem.
CONS
High memory — stores every timestamp.
Sliding Window Counter
Hybrid: fixed window + weighted previous window count. Approximates sliding log with much less memory.
PROS
Accurate and memory efficient. Best of both worlds.
CONS
Slight approximation vs true sliding log.
Design Considerations
| Topic | Detail |
|---|---|
| Storage | Use Redis for shared counter across distributed nodes. Key = user_id or IP; value = count + TTL expiry. |
| Placement | API gateway (edge) or middleware per service. Gateway preferred for centralized policy management. |
| Response headers | Return X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Retry-After in 429 responses. |
| Multi-DC | Deploy at edge. Use eventual consistency for cross-DC counter sync — slight over-allowing is acceptable. |
| Soft vs hard limits | Soft: allow small burst over limit. Hard: strictly enforce. Most production systems use soft limits. |
Vol 1 · Ch. 5
Consistent Hashing
Minimize key redistribution when nodes are added or removed. Foundation of distributed data stores.
The Problem
Naive modulo hashing:
hash(key) % N. Adding/removing a server reshuffles nearly ALL keys → massive cache misses, full data redistribution.Core Concepts
| Concept | Detail |
|---|---|
| Hash ring | Map both server IPs and keys onto a circular ring (SHA-1 space 0 to 2¹⁶⁰). Key stored on first server clockwise from its position. |
| Adding a server | Only keys between predecessor and new server need remapping — roughly k/n keys total. |
| Removing a server | Keys from removed server reassigned to next clockwise server. Only ~k/n keys affected. |
| Virtual nodes | Each server gets v positions on the ring. More positions = more even distribution. Tune v per server capacity. Trade-off: more v = more even, more memory. |
| Data replication | Walk clockwise from key position; replicate to first N unique physical servers (skip same-DC nodes for disaster tolerance). |
| Hotspot limitation | Consistent hashing distributes keys evenly — NOT traffic. Celebrity keys still create hot nodes. Mitigate with read replication or key salting. |
Real-World Use
Amazon DynamoDB, Apache Cassandra, Discord (scaled Elixir to 5M concurrent users), Akamai CDN all use consistent hashing for data partitioning.
Vol 1 · Ch. 6
Design a Key-Value Store
Distributed hash table design covering CAP theorem, quorum, gossip protocol, and Bloom filters.
CAP Theorem
Definition
A distributed system can only guarantee 2 of 3: Consistency (all nodes see same data), Availability (always responds), Partition Tolerance (works during network splits). During partition, choose CP or AP.
CP Systems
Block reads/writes during partition. Prioritize consistency.
Examples: HBase, ZooKeeper
Examples: HBase, ZooKeeper
AP Systems
Serve potentially stale data during partition. Prioritize availability.
Examples: Cassandra, DynamoDB, Couchbase
Examples: Cassandra, DynamoDB, Couchbase
Key Design Components
| Component | Design Decision |
|---|---|
| Partitioning | Consistent hashing distributes keys across nodes. Virtual nodes for even distribution. |
| Replication | Async replication to N nodes walking clockwise. Replicas in distinct DCs for disaster tolerance. |
| Quorum (W+R > N) | N=3, W=2, R=2 guarantees strong consistency. Lower W/R = higher availability, lower consistency. |
| Conflict resolution | Vector clocks: each write increments [server, version] pair. Client resolves conflicts on read. |
| Failure detection | Gossip protocol: each node maintains heartbeat table, propagates to random neighbors. Node marked down if heartbeat stale. |
| Sloppy quorum | Write to W healthy nodes even if some replicas are offline. Hinted handoff: temp node stores data, pushes back when original recovers. |
| Write path | Request → commit log (durability) → memory table → SSTable (disk) when memory threshold hit. |
| Read path | Check memory table → Bloom filter determines SSTable → read SSTable. Bloom filter reduces unnecessary disk reads. |
Vol 1 · Ch. 7
Unique ID Generator (Snowflake)
Generate IDs that are globally unique, sortable by time, and work across distributed nodes without coordination.
Approaches Compared
| Approach | How It Works | Pros | Cons |
|---|---|---|---|
| UUID (128-bit) | Generate locally using random bits. No coordination. | Simple, no SPOF, no network call | Not sortable, 128-bit, rare collisions possible |
| DB Auto-Increment | DB identity column. Multi-DB with step (server 1: 1,3,5… server 2: 2,4,6…) | Simple, sortable | DB bottleneck, exposes business data |
| Snowflake ★ | 64-bit: sign + timestamp + datacenter + machine + sequence | Sortable by time, 64-bit, ~4096 IDs/ms/machine | Clock skew risk, machine IDs must be managed |
| Ticket Server | Centralized auto-increment server | Simple, numeric IDs | SPOF without HA setup |
Snowflake 64-bit Layout
Bit Allocation
1 bit
sign=0
41 bits
timestamp (ms)
~69 years
~69 years
5 bits
datacenter
(32 DCs)
(32 DCs)
5 bits
machine
(32/DC)
(32/DC)
12 bits
sequence
(4096/ms)
(4096/ms)
Throughput
Snowflake generates up to 4,096 IDs per millisecond per machine without any coordination or network calls. Total capacity: 1024 machines × 4096 IDs/ms = ~4M IDs/ms globally.
Vol 1 · Ch. 8
URL Shortener (TinyURL)
Hash long URLs to short codes. Handle redirects, collisions, and scale to billions of URLs.
| Component | Design Detail |
|---|---|
| APIs | POST /api/v1/shorten → return shortURL. GET /{shortURL} → redirect to longURL. |
| Redirect type | 301 Permanent: browser caches, reduces server load. 302 Temporary: every request hits server, better for analytics. Choose based on needs. |
| Hash length | Base62 (a-z, A-Z, 0-9): 62⁷ = 3.5 trillion unique URLs. For 365B URLs → 7-char hash is sufficient. |
| Hash approach A | MD5/SHA-1 of longURL, take first 7 chars. Collision: append random salt and retry. Simple but requires DB lookup on each. |
| Hash approach B | Base62 encode unique auto-increment ID. No collision, but exposes sequence and requires distributed ID gen. |
| Data model | shortURL (PK), longURL, createdAt, expiredAt, userId |
| Cache layer | Cache top 20% most-accessed short URLs in Redis. 80% of traffic from cache (Pareto principle). 80/20 rule. |
| Scale | Stateless web tier behind LB. DB replication + sharding on shortURL key. CDN for redirect latency reduction. |
Vol 1 · Ch. 9
Web Crawler
Systematically browse the web. BFS traversal with politeness constraints, deduplication, and distributed worker fleet.
| Component | Design Detail |
|---|---|
| Seed URLs | Start with well-connected URLs. Prioritize by PageRank, traffic, domain authority. |
| URL frontier | Priority queue + politeness queue. Priority = importance score. Politeness = rate-limit per domain (1 req/s per host). Use BFS, not DFS (depth too deep). |
| HTML downloader | Distributed workers using consistent hashing. Robots.txt respected. Per-request timeout. Parallel downloads. |
| DNS cache | Cache DNS lookups to avoid N lookups per download. Cache TTL ~1 min. Big performance win at scale. |
| Content parser | Extract URLs from HTML. Validate and filter. Bloom filter for fast URL deduplication check. |
| Content dedup | Hash page content. Store hash → skip if already seen. Handles mirrors and duplicate content across domains. |
| Fault tolerance | Checkpoint crawl state to storage periodically. Resume from last checkpoint on failure. |
| Politeness | Honor robots.txt. Rate-limit per domain. Identify crawler via User-Agent. Avoid crawl traps (infinite loops). |
Vol 1 · Ch. 10
Notification System
Multi-channel delivery (push, SMS, email) with deduplication, retry logic, and user preference management.
Notification Channels
| Channel | Protocol / Provider | Notes |
|---|---|---|
| iOS Push | APNs (Apple Push Notification Service) | Requires device token. Provider → APNs → device. |
| Android Push | FCM (Firebase Cloud Messaging) | Replaced deprecated GCM. Google-managed infrastructure. |
| SMS | Twilio, Nexmo/Vonage | Third-party gateway. Best deliverability. Cost per SMS. |
| SendGrid, Mailchimp | Better deliverability than self-hosted SMTP. |
System Design
| Component | Detail |
|---|---|
| Notification service | Single entry point. Builds payload. Queues to channel-specific workers via message queue. |
| Message queues | One queue per channel type. Decouples producers from senders. Absorbs traffic spikes. |
| Deduplication | Redis cache with event_id → prevents duplicate sends on retry. Critical for at-least-once delivery. |
| Rate limiting | Don't spam users. Limit push notifications per user per time window. Per-channel limits. |
| User preferences | Users can opt out per channel. Check preference table before enqueuing. Cache preferences. |
| Retry logic | Exponential backoff with jitter. Dead-letter queue for permanently failed deliveries. Alert on DLQ depth. |
Vol 1 · Ch. 11
News Feed System (Facebook/Twitter)
Fan-out architecture for publishing and retrieving personalized feeds. The push vs pull tradeoff.
Two Critical Flows
Feed Publishing
User posts → API server → Post service → fanout service → message queue → workers write post_id to each follower's feed cache (Redis sorted set by timestamp)
Feed Retrieval
User requests → API server → news feed service → read from Redis feed cache → hydrate post + user data → return ranked feed
Fanout Strategy
| Approach | How | Best For | Problem |
|---|---|---|---|
| Push (fanout on write) | Pre-compute feed for each follower on post | Regular users — fast reads | Expensive writes for celebrities (10M followers) |
| Pull (fanout on read) | Compute feed on read from followees | Celebrity accounts | Slow reads — must aggregate many sources |
| Hybrid ★ | Push for regular users; pull for celebrities | Production systems | More complex but solves both extremes |
Key Data Structure
Redis Sorted Set per user:
{score: timestamp, member: post_id}. Store only post IDs in cache — fetch full data from post/user cache separately. This minimizes cache storage.Vol 1 · Ch. 12
Chat System (WhatsApp/Slack)
Real-time bidirectional messaging. WebSocket connections, presence, group chat, and multi-device sync.
Protocol Comparison
| Protocol | Pattern | Pros | Cons |
|---|---|---|---|
| WebSocket ★ | Full-duplex bidirectional | True real-time; server can push without polling | Persistent connections; harder to scale; requires pub/sub |
| HTTP Long Polling | Simulated push | Works everywhere; no special protocol | Higher latency; server holds many open connections |
| SSE | Server → client push only | Simple; built on HTTP | One-way only; can't send client messages |
Architecture Components
| Component | Design Detail |
|---|---|
| Chat servers | Hold WebSocket connections. ~100K connections per server. Consistent hashing routes user to assigned chat server. |
| Message storage | Key-value store (HBase/Cassandra). Row key = channel_id + message_id (time-ordered). Not relational — write-heavy, no complex queries. |
| Message ordering | Snowflake-style sequence ID per channel. Must be unique + ordered within a conversation. |
| Online presence | Heartbeat from client every 5s. Presence server stores last_active_at. Fan-out status changes to friends via Redis pub/sub. |
| Group chat | Message sent to channel. Fanout service writes to each member's inbox. Limit group size for fanout scalability (e.g. 500–10K). |
| Multi-device sync | Each device polls from its last_message_id. Message queue per user for cross-device delivery. |
Vol 1 · Ch. 13
Search Autocomplete System
Trie-based prefix matching with top-K suggestions. Batch aggregation + hot cache for sub-millisecond responses.
| Component | Design Detail |
|---|---|
| Data collection | Log every search query with frequency. Aggregate daily in batch jobs (MapReduce). Build trie from aggregated frequency data weekly. |
| Trie structure | Prefix tree. Each node stores top-k most frequent completions for that prefix cached at the node — avoids full tree traversal on lookup. |
| Query service | Read from trie. Cache results in Redis/Memcached. Serve from cache for sub-ms response. Rebuild trie periodically, not per query. |
| Trie storage | Serialize trie to document store or Redis. Shard by first character of prefix for horizontal scaling. |
| Real-time updates | Don't update trie on every query (too slow). Batch aggregate. Use probabilistic structures for real-time trending detection. |
| Browser optimizations | Debounce: only query after pause. Prefetch first few characters. CDN cache for common top-level prefixes. |
| Content filtering | Remove hateful/inappropriate completions using content filters before storing in trie. |
Vol 1 · Ch. 14
Design YouTube
Video upload pipeline with DAG transcoding, CDN delivery, adaptive bitrate streaming, and cost optimization.
Two Critical Flows
Video Upload
Client → pre-signed S3 URL (bypass API) → raw video S3 → message queue → transcoding workers (DAG pipeline) → CDN origin → serve
Video Streaming
Client → nearest CDN PoP → stream directly. NOT origin. Adaptive bitrate (DASH/HLS). Auto-quality based on bandwidth.
Architecture Deep Dive
| Component | Design Detail |
|---|---|
| Transcoding pipeline | DAG scheduler. Workers: split video into GOPs → encode at multiple bitrates (360p/720p/1080p/4K) → thumbnail → watermark → merge → upload CDN. |
| Pre-signed upload URL | Client gets temp S3 URL. Uploads large file directly to S3. API server never handles video bytes — huge bandwidth saving. |
| CDN strategy | Popular videos pre-warmed at all edges. Long-tail served from regional CDN or origin on demand. CDN reduces bandwidth cost 90%+. |
| Metadata DB | Video metadata (title, uploader, likes) in MySQL (sharded/replicated). Completely separate from video binary storage. |
| Streaming protocols | MPEG-DASH, HLS, Smooth Streaming. Protocol depends on client support. Adaptive bitrate: client switches quality based on current bandwidth. |
| Cost optimization | Store only popular videos at all CDN edges. Move infrequently accessed videos to cheaper cold storage. Compress thumbnails. |
Vol 1 · Ch. 15
Google Drive
Block-level storage with delta sync, deduplication, conflict resolution, and offline support.
| Component | Design Detail |
|---|---|
| File upload | Resumable chunked upload (~5MB chunks). Client retries failed chunks without restarting entire file. Chunks stored in object storage (S3). |
| Block storage | Split file into fixed blocks. Store each block by hash(block). Delta sync: only changed blocks re-uploaded on edit. Efficient for large files with small edits. |
| Metadata DB | File metadata: file_id, name, path, size, owner, created_at, blocks[]. MySQL. Separate from block storage. |
| Sync service | Notification service pushes file change events to all user's devices. Long polling or WebSocket for real-time sync. |
| Conflict resolution | Last-write-wins OR present both versions to user (Dropbox approach). Vector clocks for advanced conflict detection. |
| Deduplication | Block-level dedup: if hash(block) already exists, don't re-upload — just reference existing block. Major storage cost reduction. |
| Cold storage | Files not accessed in N months moved to cheaper cold storage (S3 Glacier). Transparent to user — retrieved on demand. |
Vol 2 · Ch. 1
Proximity Service (Yelp)
Find N nearest businesses within radius R. Geospatial indexing with Geohash or Quadtree.
Geospatial Indexing Approaches
Geohash
Encode lat/lng as base32 string. Nearby locations share prefix. Level 6 (~1.2km grid) suits most proximity searches. Search: target cell + 8 neighbors to prevent boundary edge cases.
Quadtree
Recursively subdivide 2D space into 4 quadrants until ≤100 businesses per cell. In-memory tree (~1GB for 200M businesses). Fast reads; complex real-time updates.
| Design Decision | Detail |
|---|---|
| Read vs write ratio | Read-heavy (searches) vs write-light (business updates). Separate read replicas for geospatial queries. |
| Boundary problem | Geohash boundary: a business just across the cell boundary won't appear in exact cell search. Always search target cell + 8 surrounding cells. |
| Precision levels | Geohash length 4 = ~39km, 5 = ~4.9km, 6 = ~1.2km, 7 = ~152m. Choose based on search radius. |
| Cache strategy | Cache popular search results by (city, category) with TTL ~10min. Precompute for major metro areas. |
| Business service | CRUD for business metadata separate from location/search service. Cache business info in Redis. |
Vol 2 · Ch. 2
Nearby Friends
Real-time location sharing. Unlike static businesses — friends move. Continuous GPS updates via WebSocket + Redis pub/sub.
Key Difference from Proximity Service
Businesses are static. Friends move in real-time. Location must be continuously pushed to friends, not just queried on demand.
| Component | Design Detail |
|---|---|
| Location updates | Client sends GPS every 30s via WebSocket. Location history service stores to Cassandra (append-only, time-series). |
| Redis pub/sub | Location update → publish to Redis channel per user. Friends subscribed to that channel receive updates instantly. |
| Fan-out | For each user who updates: find friend list → for each online friend → push update via their WebSocket connection. |
| Distance computation | Haversine formula on incoming friend location update. Only surface friend if within threshold (e.g. 5km). |
| Scale | Redis Cluster. Pub/sub channels sharded by user_id. Location servers route pub/sub messages to correct WebSocket connection server. |
Vol 2 · Ch. 3
Google Maps
Map tiles, routing graph, live traffic aggregation, and ETA prediction at global scale.
| Component | Design Detail |
|---|---|
| Map tiles | Pre-rendered image tiles at zoom levels 0–21. Served from CDN keyed by tile ID (zoom/x/y). Client loads tiles on demand as user pans/zooms. |
| Road graph | Graph: nodes = intersections, edges = segments with weight = travel time. Compressed adjacency list. Partitioned geographically. |
| Routing algorithm | A* or Bidirectional Dijkstra. Preprocessed contraction hierarchies for fast long-distance routing. Live traffic adjusts edge weights dynamically. |
| Live traffic | GPS data from millions of phones aggregated in real-time. Streaming MapReduce → segment travel times → update graph edge weights. |
| ETA prediction | Historical speed data + real-time traffic → ML model for ETA per segment. Sum along route. Recalculate on traffic changes. |
| Data encoding | Geohash or S2 cells for location queries. Protocol Buffers for compact tile data transfer. ~50% size reduction vs JSON. |
Vol 2 · Ch. 4
Distributed Message Queue (Kafka)
High-throughput, durable, replayable event streaming. Partitioned log model with consumer groups.
| Concept | Detail |
|---|---|
| Core model | Producer → Topic (partitioned) → Consumer Group. Each partition is an ordered, append-only log. |
| Partitioning | Topic split into N partitions distributed across brokers. Partition key determines which partition (hash(user_id) % N). Enables parallel consumption. |
| Consumer groups | Each consumer in a group reads from disjoint partitions. Multiple groups read same topic independently without interference. |
| Offset | Consumer tracks its position (offset) per partition. Stored in __consumer_offsets topic. Enables replay and at-least-once delivery. |
| Delivery semantics | At-most-once (fire/forget). At-least-once (ack + retry). Exactly-once (idempotent producer + transactional consumer). Most systems use at-least-once. |
| Replication | Each partition: 1 leader + N-1 follower replicas. Leader handles reads/writes. Leader election via ZooKeeper/Raft on broker failure. |
| Retention | Messages retained for N days regardless of consumption. Consumers can replay from any offset. Enables event sourcing, multiple consumer groups. |
Vol 2 · Ch. 5
Metrics Monitoring & Alerting
Time-series data collection, storage, query, visualization, and alerting at infrastructure scale.
| Component | Design Detail |
|---|---|
| Data collection | Agents on each host collect metrics (CPU, mem, disk, custom app metrics) at configurable interval (every 10s). Push to metrics collector. |
| Metrics collector | Receives data points. Validates. Writes to time-series DB. Kafka as buffer to handle collection spikes. |
| Time-series DB | Optimized for sequential writes of (timestamp, value, tags). Examples: InfluxDB, Prometheus TSDB, OpenTSDB. NOT relational DB. |
| Query service | Query language (PromQL, Flux) to aggregate: sum, avg, p99 over time windows. Powers dashboards and alert evaluation. |
| Alerting | Rules define threshold conditions. Alerting service evaluates rules against metric stream. Routes to PagerDuty, Slack, email. |
| Downsampling | Recent data: 10s resolution. Older data: downsample to 1min, 1hr. Reduces storage cost without losing trend visibility. |
| Metric types | Host-level (CPU/mem), aggregated tier metrics (DB/cache perf), business metrics (DAU, revenue, retention, conversion). |
Vol 2 · Ch. 6
Ad Click Event Aggregation
Billion-scale click ingestion with streaming aggregation, late event handling, and billing-grade accuracy.
Core Challenges
High write volume (billions of clicks/day). Aggregation must be accurate for billing. Late-arriving events (network delay) must be handled. Results queried in real-time.
| Component | Design Detail |
|---|---|
| Data flow | Ad click → Kafka → aggregation service (Flink/Spark Streaming) → aggregated results DB → query API |
| Aggregation service | Streaming aggregation: count clicks per ad per N-minute window. Emit counts to results store. Handles out-of-order events with watermarks. |
| Storage | Raw events: Kafka (7-day retention). Aggregated results: ClickHouse or Cassandra (fast range queries on time + ad_id). |
| Late event handling | Watermark: allow events up to X minutes late. Re-aggregate window on late arrival. Mark and reprocess affected windows. |
| Deduplication | event_id (Snowflake ID) deduplicates at-least-once delivery. Redis Set per time window for fast dedup lookup. |
| Reconciliation | End-to-end: reprocess raw Kafka events daily in batch (MapReduce) to validate streaming counts. Fix discrepancies. Critical for billing accuracy. |
| Kafka partitioning | Partition by ad_id → same ad's events go to same partition → consistent aggregation without distributed join. |
Vol 2 · Ch. 7
Hotel Reservation System
Inventory management with race condition prevention, idempotency, and overbooking buffer logic.
Core Challenge
Race condition: two users trying to book the last room simultaneously. Must prevent double booking without sacrificing performance.
| Component | Design Detail |
|---|---|
| Data model | hotel, room_type, room_inventory(hotel_id, room_type_id, date, total_rooms, reserved_rooms), reservation(id, user_id, hotel_id, check_in, check_out, status) |
| Availability check | SELECT total_rooms - reserved_rooms WHERE date BETWEEN check_in AND check_out. Must be transactional with the booking write. |
| Optimistic locking | Add version field: UPDATE room_inventory SET reserved=reserved+1, version=version+1 WHERE id=? AND reserved < total AND version=?. Retry on conflict. |
| DB transaction | Wrap availability check + update in single DB transaction with SELECT FOR UPDATE. Prevents race conditions at DB level. |
| Overbooking buffer | Hotel industry practice: allow ~5% over capacity to account for cancellations. Configurable per hotel and room type. |
| Idempotency | Reservation request carries idempotency_key. Server deduplicates on this key. Safe to retry failed network requests without double booking. |
Vol 2 · Ch. 8
Distributed Email Service
SMTP send/receive pipeline, S3 storage for bodies, full-text search, and deliverability infrastructure.
| Component | Design Detail |
|---|---|
| Protocols | SMTP (send), IMAP (retrieve, sync across devices), POP3 (download and delete). Modern webmail uses proprietary HTTP APIs. |
| Send flow | User → send API → metadata DB (store email) → message queue → SMTP outbound servers → DNS MX lookup → recipient mail server → inbox |
| Receive flow | Inbound SMTP server → content filter (spam/virus) → S3 (body) + metadata DB (headers/attachments list) → IMAP server → client |
| Storage | Email body: S3/object storage. Metadata + headers: Cassandra (append-only, query by user + time). Attachments: S3 with CDN for download. |
| Search | Full-text search on subject/body via Elasticsearch. Index updated asynchronously from write path. Not relational DB search. |
| Spam/virus filtering | Multiple layers: IP reputation, content scoring, ML classifier, virus scanner. All queued before inbox delivery. |
| Deliverability | SPF, DKIM, DMARC records prevent spoofing and impersonation. Dedicated IP warming for new sending IPs. Bounce handling. |
Vol 2 · Ch. 9
S3-like Object Storage
Immutable blob storage with erasure coding, multipart upload, versioning, and placement service.
| Component | Design Detail |
|---|---|
| Core abstraction | Immutable objects stored by bucket/key. Flat namespace per bucket (not a real file system). Objects up to terabytes. |
| Upload path | Client → API service (generate UUID, write metadata) → data store routing → data nodes (replicate to N) → return object_id |
| Data nodes | Store objects as files on disk. Heartbeat to placement service. N=3 replicas standard. Cross-AZ replication for durability. |
| Metadata DB | Object metadata: object_id, bucket, key, size, hash, created_at, storage_node_ids. MySQL/Postgres. Separate from actual data. |
| Erasure coding | Split object into k data + m parity shards. Any k shards can reconstruct. 1.5× storage overhead vs 3× replication. Slower recovery but much more efficient. |
| Multipart upload | Large objects split into parts (5MB–5GB each). Upload parts in parallel. Assemble on server. Retry only failed parts. |
| Versioning | Each PUT creates new version. Delete creates delete marker. GET returns latest; specify version_id for older version. Critical for accidental delete recovery. |
Vol 2 · Ch. 10
Real-time Gaming Leaderboard
Redis Sorted Set as the core data structure. Sub-millisecond rank queries for millions of players.
Core Data Structure
Redis Sorted Set:
ZADD leaderboard {score} {user_id} — O(log N) insert/update. ZREVRANK for rank in O(log N). ZREVRANGE leaderboard 0 K-1 WITHSCORES for top-K in O(log N + K).| Operation | Redis Command | Complexity |
|---|---|---|
| Update score | ZADD leaderboard score user_id | O(log N) |
| Get user rank | ZREVRANK leaderboard user_id | O(log N) |
| Get top-K | ZREVRANGE leaderboard 0 K-1 WITHSCORES | O(log N + K) |
| Get score | ZSCORE leaderboard user_id | O(1) |
| Design Topic | Detail |
|---|---|
| Persistence | Redis is primary. Async write to MySQL for durability. Rebuild Redis from DB on restart. Eventual consistency acceptable. |
| Scale | Redis Cluster: shard by game_id or region. For global top-K across shards: each shard returns top-K, merge and sort at API layer. |
| Historical leaderboards | Separate sorted sets per time window (daily/weekly/all-time). TTL on older sets to auto-expire. Key: leaderboard:{game_id}:{period} |
| Anti-cheat | Server-side score validation. Score anomaly detection. Rate limit score update API per user. |
Vol 2 · Ch. 11
Payment System
Exactly-once processing, idempotency, distributed transactions, double-entry ledger, and reconciliation.
Core Challenges
Exactly-once processing. Idempotency across retries. Distributed transactions across services. Reconciliation for discrepancies. Regulatory compliance (PCI-DSS).
| Component | Design Detail |
|---|---|
| Payment flow | User → payment service → PSP (Stripe/Adyen) → bank/card network → settlement → ledger update → notify user |
| Idempotency | Every request carries unique idempotency_key (UUID). Payment service deduplicates on this key. Safe to retry on timeout without double charge. |
| Payment states | Pending → Processing → Success / Failed / Cancelled. State machine with explicit transitions stored in payment_orders table. |
| Double-entry ledger | Every payment = debit one account + credit another. Immutable append-only entries. Ensures books always balance. Enables full audit trail. |
| PSP idempotency | PSP (Stripe, etc.) also guarantees idempotency with their own key. Our key + their key → two levels of protection against duplicate charges. |
| Reconciliation | Daily batch job compares internal ledger vs PSP settlement statement. Flag discrepancies. Human review for unmatched records. |
| Retry strategy | Exponential backoff with jitter. Dead-letter queue for permanently failed payments. Alert on DLQ depth. Never retry indefinitely. |
Appendix
Quick Reference Tables
Database selection guide, CAP theorem, protocol comparison, and interview problem → answer lookup.
Database Selection Guide
| Use Case | Best Choice | Why | Key Commands |
|---|---|---|---|
| User profiles, orders, billing | MySQL / PostgreSQL | ACID transactions, complex JOINs, relational schema, strong consistency e.g. Uber rider/driver profiles · Airbnb listings · Stripe payment records |
|
| Session / cache / rate limiting | Redis | In-memory, sub-ms latency, sorted sets, pub/sub, TTL-based expiry e.g. Twitter trending cache · Discord leaderboard · Slack session tokens |
|
| Chat messages / time-series writes | Cassandra | Append-only, high write throughput, wide columns, tunable consistency e.g. WhatsApp message store · Discord history · Instagram activity log |
|
| Full-text search / autocomplete | Elasticsearch | Inverted index, fuzzy match, faceted filters, geo queries, aggregations e.g. Airbnb listing search · GitHub code search · Spotify track search |
|
| Product catalog / content (flexible) | MongoDB | Schema-flexible documents, rich queries, horizontal sharding e.g. Netflix show catalog · Shopify product docs · Medium article metadata |
|
| Social graph / fraud detection | Neo4j | Native graph traversal, friend-of-friend queries, relationship-heavy queries e.g. LinkedIn connections · Twitter follow graph · Pinterest interest graph |
|
| Video / images / blobs | S3 / Object Storage | Cheap durable blob store, CDN integration, 11-nines durability, unlimited scale e.g. YouTube raw uploads · Dropbox file blocks · Instagram photos |
|
| Click events / analytics (OLAP) | ClickHouse / Druid | Columnar storage, fast aggregation on billions of rows, petabyte scale e.g. Cloudflare 50M events/sec · Uber ad-click pipeline · Datadog metrics |
|
| Event streaming / message bus | Kafka | High throughput, replayable log, consumer groups, partitioned ordering e.g. LinkedIn activity feed · Airbnb booking pipeline · Uber trip events |
|
| Geospatial / nearby queries | Redis GEO / PostGIS | GEORADIUS for point-radius, PostGIS for complex polygons, spatial index e.g. Yelp radius search · Uber surge zones · Google Maps POI lookup |
|
CAP Theorem Quick Reference
| System | CAP | Behavior | Key Commands |
|---|---|---|---|
| HBase | CP | Blocks during partition. Strong consistency. Used in Hadoop ecosystem. e.g. Facebook Messenger history · Apache HBase analytics at Yahoo |
|
| ZooKeeper | CP | Distributed coordination, leader election. Blocks writes on partition. e.g. Kafka broker coordination · Hadoop NameNode HA · Cassandra gossip seed |
|
| MongoDB | CP | Primary-only writes. Secondary reads may be stale. Strong by default. e.g. Robinhood trade records · Stripe config service · Coinbase account store |
|
| Cassandra | AP (tunable) | Eventual consistency default. QUORUM reads/writes give CP-like behavior. e.g. Instagram timeline · Netflix viewing history · Discord messages (500M/day) |
|
| DynamoDB | AP (default) | Eventual consistency default. ConsistentRead=true for strong (2× RCU cost). e.g. Amazon shopping cart · Snapchat stories · Lyft real-time driver location |
|
| Redis Cluster | AP | Eventual consistency during network partition. Single-node Redis = no partition. e.g. Twitter rate limiter · Airbnb session store · Uber surge pricing cache |
|
| Google Spanner | CP + external consistency | Global strong consistency via TrueTime. Only CP system that scales globally. e.g. Google Ads billing · YouTube content metadata · Stripe global ledger |
|
Communication Protocols
| Protocol | Pattern | Use Case + Example | Key Commands |
|---|---|---|---|
| REST / HTTP | Request-Response | Public APIs, CRUD operations, microservices, browser clients e.g. GitHub API · Stripe Payments API · Twitter REST API |
|
| WebSocket | Full-duplex persistent | Chat, live gaming, collaborative editing, real-time dashboards e.g. Slack messaging · Robinhood live quotes · Figma multiplayer |
|
| SSE | Server push (one-way) | Live feeds, deploy logs, notification streams — client can't send back e.g. GitHub Actions logs · Vercel deploy output · Twitter live timeline |
|
| gRPC | Binary RPC (HTTP/2) | Internal microservice comms, streaming, low latency, type-safe contracts e.g. Google internal services · Netflix gRPC mesh · Cloudflare edge workers |
|
| GraphQL | Flexible query | Mobile clients with diverse data needs, reduces over-fetching/under-fetching e.g. GitHub GraphQL API v4 · Shopify Storefront API · Facebook news feed |
|
| Long Polling | Simulated push | Legacy real-time where WebSocket unavailable; simple infrastructure e.g. Basecamp classic · older Jira realtime · Dropbox early notification system |
|
| MQTT | Pub/sub IoT | IoT sensors and devices, low-bandwidth, unreliable networks, QoS levels e.g. AWS IoT Core · Tesla vehicle telemetry · Philips Hue hub protocol |
|
| WebRTC | Peer-to-peer media | Video/audio calls, screen sharing, P2P file transfer e.g. Google Meet · Discord voice · WhatsApp video calls |
|
Interview Quick-Fire: Problem → Answer
| Problem | Staff-Level Answer + Concrete Example | Key Commands |
|---|---|---|
| Reduce DB read load | Add read replicas + Redis cache. Cache-aside pattern. Target >90% cache hit rate. Expire on write. Real world: Netflix: 95% of API responses served from Redis · Twitter: feed from Redis sorted set, never DB |
|
| Handle traffic spikes | Stateless web tier + autoscaling. Message queue to buffer async work. Circuit breaker for downstream. Real world: Shopify: queues checkout writes during Black Friday · Slack: SQS absorbs notification bursts |
|
| Eliminate single point of failure | Redundancy at every tier: dual LBs, multiple web servers, DB replication, multi-AZ cache. Real world: GitHub: Anycast IP across multiple LBs · Netflix: active-active across 3 AWS regions |
|
| Reduce global latency | CDN for statics. GeoDNS routes to nearest DC. Edge caching. Regional service replicas. Real world: Cloudflare: 300+ PoPs serve statics · Discord: regional voice servers with GeoDNS routing |
|
| Prevent double booking | SELECT FOR UPDATE (pessimistic) or version-field CAS (optimistic). Idempotency key on every request. Real world: Booking.com: SELECT FOR UPDATE on room_inventory · Stripe: idempotency key deduplicates retries |
|
| Scale to 1B chat users | WebSocket servers + Redis pub/sub for routing + Cassandra for durable message store + consistent hash. Real world: WhatsApp: 1B users on Erlang + Cassandra · Discord: Elixir WebSocket + Cassandra + Redis pub/sub |
|
| Exactly-once payments | Idempotency key stored in Redis (24h TTL) + PSP-level idempotency. Daily reconciliation vs PSP CSV. Real world: Stripe: idempotency_key on every API call · Uber: Flink exactly-once for fare calculation |
|
| Find nearby places | Geohash level 6 in Redis GEO. Query target cell + 8 neighbors to fix boundary miss. Haversine filter. Real world: Yelp: Redis GEORADIUS for restaurant search · Uber: geohash for driver matching grid |
|
| Store + serve 1B videos | S3 raw upload → parallel DAG transcoding → HLS segments → CDN. Metadata in MySQL. ABR streaming. Real world: YouTube: TUS upload + GPU transcode farm + 500+ CDN POPs · Netflix: 1200 quality variants |
|
| Generate IDs at 10K/ms | Snowflake: 41-bit ts + 10-bit machine + 12-bit seq = 4096/ms/machine. ZooKeeper assigns machine ID. Real world: Twitter Snowflake: all tweet IDs · Instagram: encodes shard ID + table in ID · Uber: UUID v5 |
|
| Aggregate billions of events | Kafka ingest → Flink tumbling windows → ClickHouse. Dedup by event_id in RocksDB state (24h TTL). Real world: Uber ad-click pipeline: 50K clicks/sec · Cloudflare analytics: 50M events/sec into ClickHouse |
|
| Fix DB write bottleneck | Shard by consistent hashing on write key. CQRS to split read/write paths. Async writes via Kafka. Real world: Discord: shards by guild_id · Instagram: shards by user_id · Cassandra: partition key = conv_id |
|
| Design a rate limiter | Token bucket in Redis with Lua script for atomicity. Key = userId:windowId. SETNX + EXPIRE for fixed window; sliding window = sorted set of timestamps. Return 429 + Retry-After header on breach. Fail-open if Redis down. Real world: Stripe: per-API-key token bucket via Redis Lua — same atomic check+decrement on every request · GitHub: 5000 req/hr per token, X-RateLimit-* headers |
|
| Build a type-ahead / autocomplete | Trie with top-K cached at each node. Redis sorted set per prefix: ZADD prefix:{str} score term. Query: ZREVRANGE prefix:{q} 0 4 for top-5. Build trie offline from search logs; push to Redis. Edge CDN caches responses per prefix. Real world: Google Search: trie + ML ranking + personalization · Twitter: Redis sorted sets per prefix for @mention autocomplete · Elasticsearch prefix queries for Airbnb search |
|
| Serve a social media feed | Fan-out on write for ≤5K followers: push post_id to each follower's Redis sorted set (score = timestamp). Fan-out on read for celebrities. Feed read = ZREVRANGE uid:feed 0 19. Hybrid: push to active followers only. Cap feed at 1000 items. Real world: Instagram: fan-out on write to follower feeds via async Kafka workers · Twitter: push for normal users, pull for celebrities (>100K followers), merge at read time |
|
| Send notifications at scale | Decouple trigger from delivery via Kafka. Notification service reads events, looks up user preferences + device tokens. Route to channel workers (APNs, FCM, SMS, email). Retry with exponential backoff. Deduplicate with Redis SET (notif_id, TTL 24h). Batch APNs/FCM pushes. Real world: Uber: Kafka event → notification fanout → FCM/APNs/SMS per channel worker · Airbnb: booking event triggers email via SendGrid + push via FCM, deduped by event_id |
|
| Store and query time-series data | Append-only writes: TimescaleDB (Postgres extension) or InfluxDB. Partition by time (daily/monthly chunks). Compress old chunks automatically. Downsample: 1s raw → 1min avg → 1hr avg via continuous aggregates. Retain raw 7 days, aggregates 1 year. Index on (metric_name, time DESC). Real world: Datadog: InfluxDB-style storage, downsampled to 1min after 7 days · Tesla: TimescaleDB for vehicle telemetry, 10M datapoints/sec · Prometheus: local TSDB with 15-day retention |
|
| Build a distributed cache | Consistent hashing across N Redis nodes — same key always routes to same node. Virtual nodes (150/server) for even distribution. Cache-aside: app checks cache, miss → DB fetch → cache write. Write-through for strong consistency. Eviction: LRU. Monitor hit rate (target >90%). Warm cache on deploy. Real world: Facebook Memcached: consistent hashing across 1000s of nodes, 99%+ hit rate · Netflix EVCache: multi-region Redis with write-through for session and metadata caching |
|
| Track millions of concurrent users online | Heartbeat every 30s from client → presence service → Redis SETEX uid:online 1 35 (TTL slightly > heartbeat interval). WebSocket ping/pong as alternative. For friend presence: SMEMBERS friends:{uid} then batch MGET. Update on disconnect event. Approximate with Bloom filter for massive scale. Real world: WhatsApp: heartbeat every 30s, last_seen stored in Redis · Slack: WebSocket ping/pong, server marks idle after 5min no activity · Discord: gateway heartbeat_interval per shard |
|
| Implement distributed locking | Redis Redlock: SET key uuid NX PX 30000 (SET if not exists, expire 30s). Lock owner = UUID (prevents other process unlocking). Release: Lua script checks UUID then DEL (atomic). For stronger guarantees: ZooKeeper ephemeral node or etcd lease. Never use lock for >10s operations. Real world: Redisson (Java): Redlock across 5 Redis nodes for quorum · Airbnb: Redis NX lock for inventory reservation · Kubernetes: etcd-based leader election for controller manager |
|
| Search across billions of documents | Elasticsearch: inverted index on tokenized fields. Shards = horizontal partitions (each shard is a Lucene index). Replicas for HA + read scale. CDC pipeline from DB → Kafka → ES indexer keeps in sync (1-5s lag). Use filter context for structured fields (fast, cached). Query context for full-text (scored). Real world: GitHub code search: custom Elasticsearch on 200M+ repos · Stack Overflow: Elasticsearch for question search · Airbnb: ES for listing search with geo_distance + text |
|
| Handle large file uploads reliably | Multipart upload: client splits file into 5–100MB chunks, uploads each chunk independently to S3. Server issues pre-signed URL per part. Client uploads directly to S3 (bypasses your servers). After all parts complete, call CompleteMultipartUpload. Resumable: track uploaded parts in Redis. Show progress from part ETags. Real world: YouTube: TUS protocol for resumable upload, splits to S3 parts · Dropbox: block-level chunking with SHA-256 dedup, direct-to-S3 upload · Google Drive: resumable upload session URI |
|
| Sync data across devices | Event sourcing: every change is an immutable event with a Lamport timestamp (or vector clock for multi-master). Sync = exchange events since last_sync_ts. Conflict resolution: last-write-wins by timestamp, or operational transform for collaborative editing. Store delta log in Cassandra. Client applies events on reconnect. Real world: Dropbox: block-level delta sync, server is source of truth · Apple iCloud: CRDTs for conflict-free sync · Notion: operational transform for real-time collaboration · Figma: CRDT for multiplayer |
|
| Build a payment processing system | Idempotency key (UUID) per request stored in Redis (NX, 24h TTL) prevents double-charge. Double-entry ledger: every transfer = debit + credit in one atomic transaction. Saga pattern for multi-step: charge card → update balance → notify. Async PSP webhook for confirmation. Nightly reconciliation vs PSP CSV. Real world: Stripe: idempotency_key on every API call, double-entry ledger · Venmo: Saga for peer transfers, Redis idempotency gate · Coinbase: immutable ledger with daily reconciliation |
|
| Design a job scheduling system | Priority queue (Redis sorted set: score = next_run_at epoch). Scheduler polls with ZRANGEBYSCORE ts 0 now LIMIT 10, atomically moves to processing set. Worker claims job via SETNX job:{id} worker_id. Heartbeat while processing. Timeout detection by separate monitor. Dead-letter queue for failed jobs after N retries. Real world: Sidekiq (Ruby): Redis sorted set for scheduled jobs, workers BRPOP from queues · Airflow: DAG scheduler with Postgres for job state · AWS EventBridge: cron-style scheduling at cloud scale |
|
| Implement a content delivery network | GeoDNS routes users to nearest PoP. Edge server checks local cache → origin fetch on miss. Cache key = URL + Accept-Encoding + Vary headers. TTL from Cache-Control. Purge via API (tag-based or URL). Origin shield: one intermediate layer absorbs stampede. Push model for known large assets (video segments). 99%+ cache hit ratio target. Real world: Cloudflare: 300+ PoPs, Anycast routing, cache at edge for static · Akamai: origin shield to protect origin from stampede · Netflix Open Connect: ISP-embedded appliances with pre-populated content |
|
| Build a recommendation engine | Collaborative filtering: user-item matrix factorization (ALS). Two-stage: (1) Candidate retrieval — find 100-1000 candidates via approximate nearest neighbor (ANN) on user embedding. (2) Ranking — DNN re-ranks candidates with user context. Pre-compute embeddings offline (Spark), serve via FAISS or Pinecone. Update embeddings nightly. Real world: Netflix: two-stage retrieval+ranking, embeddings updated daily · Spotify: ALS matrix factorization for Discover Weekly · Amazon: item-to-item collaborative filtering for 'Customers also bought' |
|
| Design a URL shortener | Base62 encode a 7-digit counter (62^7 = 3.5T URLs) from a distributed ID generator (Snowflake or Redis INCR). Store short→long in Redis (hot) + Cassandra (durable). Redirect: 301 (cached by browser, saves future requests) vs 302 (tracked every time). Analytics: async Kafka click event. Custom aliases: check uniqueness before insert. Real world: bit.ly: base62 encoding, Redis for hot redirects, MySQL for persistence · TinyURL: sequential counter in DB with Redis cache layer · Twitter t.co: 302 redirect to track all clicks |
|
| Handle webhook delivery reliably | Store webhook payload in DB with status=PENDING. Async worker delivers via HTTP POST to customer endpoint. On 2xx: mark DELIVERED. On failure: exponential backoff (1s, 2s, 4s, ... up to 24h). After N retries: mark FAILED, alert customer. Signature HMAC-SHA256 in header for verification. Idempotency key in payload. Real world: Stripe: webhook retry for 72h with exponential backoff · GitHub: webhook delivery log with redeliver button · Shopify: HMAC-SHA256 signature in X-Shopify-Hmac-SHA256 header |
|
| Design a comments / activity feed system | Comments stored in Cassandra: partition by (entity_id, comment_id timeuuid). Fetch latest 20 per entity in O(1). For nested threads: adjacency list (parent_id column) or closure table for deep nesting. Real-time updates: SSE or WebSocket push via Redis pub/sub on entity channel. Reaction counts: Redis INCR per (entity_id, reaction_type). Real world: Reddit: Postgres adjacency list for comment trees, Redis for vote counts · YouTube: Cassandra for top-level comments, Postgres for replies · Facebook: HBase for activity feed |
|
| Build a real-time collaborative editor | Operational Transform (OT) for convergence: every edit is an operation (insert/delete at position). Server serializes concurrent ops via a sequencer. Each op transformed against concurrent ops before applying. Alternative: CRDT (no central sequencer, conflict-free by design). WebSocket for real-time sync. Snapshot every 100 ops. ShareDB or Yjs for production use. Real world: Google Docs: OT with central sequencer · Figma: CRDT for multiplayer design · Notion: CRDT for offline-first sync · VS Code Live Share: OT-based co-editing |
|
| Design a fraud detection system | Real-time scoring: Flink stream on transaction events, compute features in 1-min windows (velocity, geo-distance, device fingerprint, merchant category). Lightweight ML model (gradient boosting) scores in <50ms. Score >threshold: block + flag for review. Feedback loop: confirmed fraud retrains model weekly. Rules engine for hard blocks (known bad IPs, sanctioned countries). Real world: Stripe Radar: ML + rules engine on every payment, 99.9% accuracy · PayPal: Flink real-time feature computation + GBM model · Uber: real-time geolocation anomaly detection for account takeover |
|
| Store user sessions at scale | Stateless tokens (JWT): session data encoded in signed token, no server state needed. Verify signature on every request. Refresh token (longer TTL) rotates access token. Revocation: maintain Redis blocklist of invalidated JTIs (token IDs). Alternatively: opaque session token → Redis HSET session:{id} with user data + TTL. Scale: Redis cluster, sticky sessions for WebSocket. Real world: Auth0: JWT access tokens (15min TTL) + refresh tokens (30d) · Google: opaque session tokens in distributed Memcached · GitHub: personal access tokens hashed in DB, Redis session cache |
|
| Design an image/video processing pipeline | Upload: client → S3 pre-signed URL (bypasses your servers). S3 event → SQS queue → transcoding workers (EC2 Spot or Lambda for cost). DAG of processing steps: thumbnail generation → format transcoding → CDN upload → DB metadata update. Store job state in DynamoDB. Idempotent workers: re-process if job state ≠ DONE. Serve via CloudFront. Real world: YouTube: TUS upload → parallel GPU transcode farm → 1200+ quality variants → CDN · Instagram: Lambda for thumbnail resize on S3 event · Netflix: Cosmos microservices DAG for encoding |
|
| Build a multi-tenant SaaS platform | Three isolation models: (1) Silo — separate DB per tenant (max isolation, high cost). (2) Bridge — shared DB, separate schema per tenant. (3) Pool — shared everything, tenant_id column with row-level security. Use Pool for most tenants, Silo for enterprise. Route at API gateway: extract tenant from subdomain/JWT claim → connection pool per tenant. Real world: Salesforce: shared Oracle DB + row-level security (Force.com) · Shopify: shared MySQL sharded by shop_id · Stripe: separate Postgres schemas per isolation tier |
|
| Handle cascading failures / circuit breaker | Circuit breaker states: CLOSED (normal), OPEN (fast-fail, no calls to downstream), HALF-OPEN (probe 1 request to test recovery). Trip to OPEN when error rate >50% in a 10s window. Auto-recover after 30s timeout. Bulkhead: separate thread pools per downstream service so one failure can't exhaust all threads. Fallback: return cached stale data or default response. Real world: Netflix Hystrix: circuit breaker + bulkhead for all inter-service calls · Resilience4j (Java): circuit breaker in Spring Boot microservices · AWS SDK: built-in retry + exponential backoff |
|
| Design an audit log / event sourcing system | Immutable append-only log: every state change = event with (event_id, aggregate_id, event_type, payload, actor, timestamp). Store in Kafka (durable, replayable) + Cassandra (queryable). Current state = replay of all events or snapshot + delta. Never update/delete events. Query: by aggregate_id (who/what changed) or by actor (who did it). Retain 7 years for compliance. Real world: Stripe: every API call appended to immutable event log · AWS CloudTrail: every API action logged · GitHub: all git operations append-only · Axon Framework: CQRS + event sourcing in Java |
|
| Implement cursor-based pagination | Never use OFFSET (scans all preceding rows, O(N)). Use a cursor = encoded last-seen value (e.g. base64 of {id, created_at}). Query: WHERE (created_at, id) < (cursor_ts, cursor_id) ORDER BY created_at DESC, id DESC LIMIT N+1. Return N items + has_more flag. Stable under inserts (unlike OFFSET). Works with any sort order. Real world: Stripe API: 'starting_after' object ID cursor on all list endpoints · Twitter: since_id/max_id on timeline · GitHub: Link header with next page cursor · Instagram Graph API: after cursor param |
|
| Design a config management system | Central config store (etcd or Consul) with watch API — services subscribe to their config namespace. On change: push to watchers within 1s. Config versioned (immutable: config/{service}/{version}/{key}). Rollback = point to previous version. Local cache with TTL as fallback if config service is down. Feature flags: A/B testing by user cohort via config values. Real world: LaunchDarkly: feature flag service, SDK caches flags locally · etcd: Kubernetes cluster config and leader election · Netflix Archaius: dynamic properties refreshed from config service every 60s |
|
| Build a data pipeline / ETL system | Source → Kafka ingest → stream processor (Flink for real-time, Spark for batch) → sink (data warehouse, S3, Elasticsearch). Idempotent sinks: UPSERT or dedup by event_id. Schema registry (Avro + Confluent) for backward-compatible evolution. Dead-letter queue for malformed records. Monitor: lag per Kafka consumer group. Reprocess by resetting consumer offset. Real world: Airbnb: Kafka → Spark → Hive data warehouse · LinkedIn: Kafka → Samza → Elasticsearch + HDFS · Uber: Flink for real-time trip analytics → ClickHouse |
|
AWSAmazon Web Services — Key Services
Core AWS services mapped to system design patterns, with pros/cons, cost tiers, and CLI commands.
| Service | Category | Use In Systems | Pros / Cons | Cost | Key Commands | Java Single-Server Equiv. |
|---|---|---|---|---|---|---|
| EC2 | Compute | Web servers, app servers, any stateless compute tier e.g. Scale From Zero (web tier), YouTube (transcode workers), Stock Exchange (matching engine) | ✓ Full OS control, widest instance variety, spot instances for 90% cost savings on batch ✗ Must manage OS patching, slower to scale than Lambda/ECS | On-demand ~$0.10/hr (t3.medium), Spot ~$0.03/hr, Reserved 1yr saves 40% |
| JVM Process on bare metal // EC2 = your JVM process on a physical/virtual machine
Runtime rt = Runtime.getRuntime();
int cpus = rt.availableProcessors(); // instance vCPUs
long ram = rt.maxMemory(); // -Xmx heap
// Scale: add more threads, not more machinesEC2 gives you the machine. Your JVM is the tenant. Vertical scale = bigger instance (-Xmx, more vCPUs). |
| S3 | Object Storage | Blob storage for any large files, static assets, data lake e.g. YouTube (raw video + HLS segments), Google Drive (file blocks), Notification (attachments) | ✓ 11-nines durability, unlimited scale, CDN-ready, lifecycle tiering to Glacier ✗ Not a filesystem, eventual consistency for overwrite-then-read (now strong by default) | Standard $0.023/GB/mo · Glacier $0.004/GB/mo · GET $0.0004/1K requests |
| java.io.File / NIO Path // S3 = local filesystem
Path file = Path.of("/data/uploads/video.mp4");
Files.write(file, bytes); // PUT object
byte[] data = Files.readAllBytes(file); // GET object
String url = file.toUri().toString(); // no presign neededS3 is a networked filesystem. Locally: Files.write/readAllBytes. Durability = RAID. CDN = Nginx serving /static. |
| RDS / Aurora | Relational DB | ACID transactions, user data, financial records, any relational schema e.g. URL Shortener (short→long), Hotel Reservation (inventory + bookings), Payment (ledger) | ✓ Managed MySQL/Postgres, Multi-AZ HA, Aurora 5× faster than MySQL, auto-scaling storage ✗ Max ~32K IOPS, vertical scaling limited, expensive at large scale vs self-managed | db.t3.medium ~$0.068/hr · Aurora Serverless v2 ~$0.12/ACU-hr · storage $0.10/GB/mo |
| JDBC + HikariCP connection pool HikariConfig cfg = new HikariConfig();
cfg.setJdbcUrl("jdbc:mysql://localhost/db");
cfg.setMaximumPoolSize(20); // RDS max_connections
HikariDataSource ds = new HikariDataSource(cfg);
try (Connection c = ds.getConnection();
PreparedStatement s = c.prepareStatement(
"SELECT * FROM users WHERE id=?")) {
s.setLong(1, userId);
ResultSet rs = s.executeQuery();
}RDS = managed MySQL/Postgres. Locally: same JDBC driver, HikariCP for pooling. Multi-AZ HA = JDBC failover URL. |
| ElastiCache (Redis) | In-memory Cache | Session store, rate limiter, leaderboard, pub/sub, feed cache e.g. Rate Limiter (token bucket), News Feed (sorted sets), Nearby Friends (GEO), Chat (pub/sub) | ✓ Sub-ms latency, Redis data structures (sorted sets, streams), Cluster mode for horizontal scale ✗ Data must fit in RAM, not durable by default (needs RDB+AOF), cluster adds complexity | cache.t3.micro ~$0.017/hr · cache.r6g.large ~$0.169/hr · data transfer extra |
| ConcurrentHashMap + Caffeine cache // ElastiCache Redis = in-process cache
Cache<String, Object> cache = Caffeine.newBuilder()
.maximumSize(10_000) // memory bound
.expireAfterWrite(5, MINUTES) // TTL
.build();
cache.put("session:123", sessionData);
Object val = cache.getIfPresent("session:123");
// No ZADD/pub-sub — use PriorityQueue + AtomicLongRedis sorted sets → TreeMap. Pub/sub → Disruptor or LinkedBlockingQueue. Caffeine for TTL-based caching. |
| DynamoDB | NoSQL Key-Value | High-throughput key lookups, session store, shopping cart, serverless backends e.g. Notification (device tokens), Ad Click (dedup state), Nearby Friends (location store) | ✓ Fully managed, single-digit ms at any scale, DAX cache layer, DynamoDB Streams for CDC ✗ No joins, limited query flexibility, hot partitions, expensive at high RCU/WCU | On-demand: $1.25/million WRU · $0.25/million RRU · $0.25/GB/mo storage |
| ConcurrentHashMap<String, Object> // DynamoDB = distributed HashMap
ConcurrentHashMap<String, Map<String,Object>> table =
new ConcurrentHashMap<>();
// PUT item
table.put(pk, Map.of("sk",sk, "data",val));
// GET item
Map<String,Object> item = table.get(pk);
// Conditional write (IF NOT EXISTS)
table.putIfAbsent(pk, newItem);
// No secondary index → maintain separate MapDynamoDB partition key = HashMap key. Conditional writes = CAS via putIfAbsent / replace. GSI = second HashMap. |
| SQS / SNS | Messaging | Async task queue, fan-out notifications, decoupling microservices e.g. Notification System (channel fan-out), Payment (async PSP retry), Web Crawler (URL queue) | ✓ SQS: at-least-once delivery, unlimited throughput, FIFO option. SNS: fan-out to millions ✗ SQS not replayable (message deleted on consume), max 14-day retention, no consumer groups | SQS: first 1M free, then $0.40/million · SNS: $0.50/million notifications |
| LinkedBlockingQueue + ExecutorService // SQS = BlockingQueue, SNS = pub/sub via listeners
BlockingQueue<Task> queue =
new LinkedBlockingQueue<>(1000); // max depth
ExecutorService workers =
Executors.newFixedThreadPool(10);
// Producer
queue.put(new Task(event)); // blocks if full
// Consumer
workers.submit(() -> {
while(true) process(queue.take());
});
// SNS fan-out → CopyOnWriteArrayList<Consumer<Event>>SQS backpressure = BlockingQueue.put() blocking. DLQ = catch block → failedQueue. SNS = Observer pattern. |
| Lambda | Serverless Compute | Event-driven processing, API backends, image resizing, scheduled jobs e.g. Ad Click (event processor), URL Shortener (redirect function), Metrics (alert evaluator) | ✓ Zero server management, auto-scale to zero, pay per invocation, 15-min max duration ✗ Cold starts (100ms–3s), 15-min timeout, no persistent connections, limited memory (10GB) | First 1M invocations free, then $0.20/million · $0.00001667/GB-second compute |
| Runnable / CompletableFuture // Lambda = stateless function, invoked on demand
ExecutorService pool = Executors.newCachedThreadPool();
// Async invocation
CompletableFuture.runAsync(() -> {
processEvent(event); // your handler logic
}, pool);
// Sync invocation (RequestResponse)
Future<Result> f = pool.submit(() -> handle(req));
Result result = f.get(15, MINUTES); // Lambda timeoutLambda cold start = new thread spin-up. 15-min timeout = Future.get(15, MINUTES). Scale to zero = cached thread pool. |
| CloudFront | CDN | Static asset serving, video streaming, API acceleration, DDoS shield e.g. YouTube (HLS segments), URL Shortener (redirect caching), Google Maps (tile serving) | ✓ 450+ PoPs globally, integrates with S3/ALB, signed URLs for private content, WAF built-in ✗ Cache invalidation takes ~5min and costs money, custom SSL extra, limited routing logic | First 10TB/mo $0.0085/GB (US) · HTTPS requests $0.0100/10K · invalidation $0.005/path |
| Caffeine + static file servlet // CloudFront = HTTP cache in front of origin
Cache<String, byte[]> httpCache =
Caffeine.newBuilder()
.maximumSize(500) // cache N responses
.expireAfterWrite(1, HOURS) // Cache-Control TTL
.build();
// Reverse proxy pattern
byte[] response = httpCache.get(requestPath,
path -> fetchFromOrigin(path)); // miss → origin
// Cache invalidation
httpCache.invalidate("/videos/*");CloudFront = Nginx/Varnish locally. Caffeine as reverse-proxy cache. TTL = Cache-Control max-age. Invalidation = cache.invalidate(). |
| Kinesis / MSK | Event Streaming | Real-time event ingestion, stream processing, log aggregation, CDC e.g. Ad Click (click stream), Metrics Monitoring (metric ingest), Google Maps (GPS pipeline) | ✓ MSK = managed Kafka (full API compatibility). Kinesis: simpler, serverless, auto-scaling ✗ Kinesis: 7-day retention max, 1MB/record limit. MSK: more ops than Kinesis | Kinesis: $0.015/shard-hr + $0.014/million PUT · MSK: $0.21/broker-hr (kafka.m5.large) |
| Disruptor / LinkedBlockingQueue (replayable) // Kafka/Kinesis = append-only replayable log
// LMAX Disruptor for high-throughput single-JVM
RingBuffer<EventSlot> ring =
Disruptor.createSingleProducer(
EventSlot::new, 1024, SINGLE);
// Or simple: BlockingQueue + persist to file
Queue<Event> log = new LinkedBlockingQueue<>();
Files.writeString(logFile, json + "\n",
APPEND); // durable replay
// Consumer group = multiple threads, shared offsetKafka partition = ordered queue per key. Consumer group offset = shared AtomicLong. Replay = re-read from file offset. |
| EKS / ECS | Container Orchestration | Microservices, long-running services, batch workloads at scale e.g. Scale From Zero (web tier), KV Store (Cassandra cluster), Chat (WebSocket servers) | ✓ EKS: full Kubernetes, huge ecosystem. ECS: simpler, tighter AWS integration, Fargate serverless ✗ EKS: complex ops, expensive control plane ($0.10/hr). ECS: proprietary, less portable | EKS: $0.10/cluster-hr + EC2/Fargate nodes · ECS: free control plane, pay for EC2/Fargate only |
| Thread pool + service registry // ECS/EKS = process lifecycle manager
// Locally: threads are your 'containers'
ExecutorService svc = Executors.newFixedThreadPool(4);
// Health check loop (liveness probe)
ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(
() -> healthCheck(), 0, 30, SECONDS);
// Service discovery = local Map<String, InetAddress>
Map<String,String> registry = new ConcurrentHashMap<>();
registry.put("user-service", "localhost:8081");ECS task = JVM thread. Kubernetes Pod = OS process. Health probe = scheduled health check. Service mesh = localhost routing. |
GCPGoogle Cloud Platform — Key Services
Core GCP services mapped to system design patterns, with pros/cons, cost tiers, and CLI commands.
| Service | Category | Use In Systems | Pros / Cons | Cost | Key Commands | Java Single-Server Equiv. |
|---|---|---|---|---|---|---|
| Compute Engine | Compute | General-purpose VMs, GPU workloads, stateful services e.g. Scale From Zero (web tier), YouTube (GPU transcode), Stock Exchange (low-latency) | ✓ Per-second billing, live migration (no downtime for host maintenance), strong GPU/TPU lineup ✗ Smaller ecosystem than AWS EC2, fewer instance types, fewer regions | n2-standard-2 ~$0.097/hr · Preemptible ~70% discount · Committed use 1yr saves ~37% |
| JVM Process (same as EC2) // Compute Engine = VM, same as EC2
// Your JVM is the only tenant
Thread.currentThread().setPriority(Thread.MAX_PRIORITY);
int cores = Runtime.getRuntime().availableProcessors();
// Preemptible VM = daemon thread (can be killed)
Thread t = new Thread(task);
t.setDaemon(true); // killed when JVM exits
t.start();Preemptible VM = daemon thread (terminated without notice). Live migration = JVM continues on new host, no restart. |
| Cloud Storage (GCS) | Object Storage | Data lake, ML training data, video/image storage, CDN origin e.g. YouTube (raw video), Google Drive (file blocks), Google Maps (tile storage) | ✓ Consistent performance, strong consistency (since 2021), excellent BigQuery integration ✗ Egress costs add up, fewer storage classes than S3, less mature lifecycle management | Standard $0.020/GB/mo · Nearline $0.010/GB/mo · Coldline $0.004/GB/mo · ops $0.004/10K |
| java.nio.file.Files (same as S3) // GCS = durable networked filesystem
Path uploads = Path.of("/data/uploads");
Files.createDirectories(uploads);
Path obj = uploads.resolve("video.mp4");
Files.copy(inputStream, obj,
StandardCopyOption.REPLACE_EXISTING);
// List objects (like gsutil ls)
try (Stream<Path> s = Files.list(uploads)) {
s.forEach(System.out::println);
}GCS strong consistency since 2021 = Files.write then immediate Files.read is always safe. Lifecycle = scheduled Files.delete. |
| Cloud SQL / Spanner | Relational DB | SQL: regional ACID apps. Spanner: globally distributed strong consistency e.g. Hotel Reservation (Cloud SQL), Payment (Spanner for global ledger), Ad Click (audit log) | ✓ Spanner: global strong consistency + 5 nines SLA, unique in industry. Cloud SQL: managed PG/MySQL ✗ Spanner: expensive ($0.90/node-hr), complex schema design. Cloud SQL: max 96 vCPUs | Cloud SQL db-n1-standard-2 ~$0.095/hr · Spanner $0.90/node-hr + $0.30/million ops |
| JDBC / JDBC + synchronized (Spanner) // Cloud SQL = standard JDBC (same as RDS)
// Spanner global consistency = synchronized writes
public synchronized void transfer(
long from, long to, BigDecimal amount) {
// Spanner transaction = synchronized block
// guaranteeing serializable execution
debit(from, amount);
credit(to, amount);
}
// Spanner TrueTime = System.currentTimeMillis()
// + clock uncertainty window for orderingSpanner serializable isolation = synchronized method. TrueTime = monotonic clock with bounded skew. No local equivalent truly matches. |
| Memorystore (Redis) | In-memory Cache | Session store, leaderboards, pub/sub, feed cache, rate limiting e.g. Rate Limiter, News Feed (sorted sets), Nearby Friends (GEO), Chat (pub/sub routing) | ✓ Fully managed Redis, automatic failover, VPC-native, Redis Cluster support ✗ No cross-region replication, limited to 300GB per instance, no Redis modules | Basic M1 (1GB) ~$0.049/hr · Standard (HA) doubles price · $0.049/GB/hr roughly |
| Caffeine cache (same as ElastiCache) // Memorystore = managed Redis, same JVM pattern
LoadingCache<String, String> cache =
Caffeine.newBuilder()
.maximumSize(50_000)
.expireAfterWrite(Duration.ofMinutes(10))
.build(key -> loadFromDB(key)); // auto-load
cache.get("user:42"); // hit or load
cache.invalidate("user:42"); // evict on writeMemorystore = Redis, same as ElastiCache. LoadingCache auto-populates on miss = read-through cache pattern. |
| Firestore / Bigtable | NoSQL | Firestore: mobile/web app data. Bigtable: wide-column, HBase-compatible, time-series e.g. Nearby Friends (location history in Bigtable), Gmail (email index in Bigtable), Chat (messages) | ✓ Bigtable: petabyte scale, HBase API, ideal for time-series/analytics. Firestore: real-time sync ✗ Bigtable: no SQL, schema design critical. Firestore: limited query, 1MB doc size, costly at scale | Bigtable: $0.65/node-hr + $0.026/GB/mo · Firestore: $0.06/100K reads, $0.18/100K writes |
| TreeMap (Bigtable) / HashMap (Firestore) // Bigtable = sorted key-value store
TreeMap<String, Map<String,byte[]>> bigtable =
new TreeMap<>(); // rows sorted by rowkey
bigtable.put("user#001#2026",
Map.of("cf:name", "Alice".getBytes()));
// Range scan (Bigtable row range)
bigtable.subMap("user#001", "user#002")
.forEach((k,v) -> process(k,v));
// Firestore = nested ConcurrentHashMap
Map<String,Object> doc = new HashMap<>();
doc.put("name", "Alice"); doc.put("age", 30);Bigtable rowkey sort = TreeMap. Range scan = subMap(start, end). Column family = nested Map. Firestore = document store = nested HashMap. |
| Pub/Sub | Messaging | Event-driven architecture, async fan-out, stream ingestion, dead-letter queues e.g. Notification System (channel fan-out), Metrics (ingest), Web Crawler (URL queue) | ✓ At-least-once delivery, 7-day retention, push+pull modes, exactly-once option, global ✗ Messages not replayable after ack (unlike Kafka), no consumer groups, ordering needs keys | First 10GB/mo free · then $0.04/GB · snapshots $0.014/GB/mo |
| LinkedBlockingQueue (same as SQS) // GCP Pub/Sub = managed message queue
// At-least-once = BlockingQueue + ack tracking
BlockingQueue<Message> sub = new LinkedBlockingQueue<>();
Set<String> acked = ConcurrentHashMap.newKeySet();
// Process with manual ack
Message msg = sub.take();
try {
process(msg);
acked.add(msg.id()); // acknowledge
} catch (Exception e) {
// not acked = redelivered (at-least-once)
}Pub/Sub ack deadline = visibility timeout. Unacked message redelivered = retry from queue.take(). Dead-letter = failedQueue. |
| Cloud Run | Serverless Containers | Stateless APIs, event processors, ML inference, background jobs e.g. URL Shortener (redirect service), Ad Click (event processor), Metrics (alert evaluator) | ✓ Deploy any container, scale to zero, concurrency model (1000 req/instance), HTTPS auto ✗ Cold starts (up to 4s), no persistent local disk, max 60-min timeout, stateless only | First 2M requests/mo free · $0.40/million requests · $0.00002400/vCPU-second |
| CompletableFuture / Virtual Threads (Java 21) // Cloud Run = stateless HTTP handler, scales to 0
// Java 21 virtual threads = lightweight scale-to-zero
try (var executor =
Executors.newVirtualThreadPerTaskExecutor()) {
// Each HTTP request = one virtual thread
// 1000 concurrent = 1000 virtual threads
executor.submit(() -> handleRequest(req));
}
// Cold start = new VirtualThread spin-up (~1ms)
// vs Cloud Run cold start (100ms–4s)Cloud Run concurrency model (1000 req/instance) = virtual thread per request. Scale to zero = no threads running between requests. |
| GKE | Kubernetes | Microservices, stateful workloads, ML training pipelines e.g. Scale From Zero (web tier), KV Store (Cassandra on GKE), Chat (WebSocket fleet) | ✓ Google invented Kubernetes, Autopilot mode removes node management, Workload Identity ✗ Autopilot less flexible than Standard, premium channel costs extra, complex networking | Zonal cluster free · Regional cluster $0.10/hr · plus node (Compute Engine) costs |
| Thread pool + health check (same as EKS) // GKE = Kubernetes, same JVM pattern as EKS/ECS
// Pod = JVM process, Container = thread group
ExecutorService podWorkers =
Executors.newFixedThreadPool(
Runtime.getRuntime().availableProcessors());
// Readiness probe = flag checked by load balancer
volatile boolean ready = false;
// Liveness probe = heartbeat
scheduler.scheduleAtFixedRate(
() -> { if(!isAlive()) restart(); },
0, 10, SECONDS);GKE Autopilot removes node management = virtual threads remove thread management. Workload Identity = SecurityManager (conceptually). |
| BigQuery | Data Warehouse | Analytics on petabyte datasets, ad-hoc SQL, BI dashboards, ML training data e.g. Ad Click (reporting), Metrics (long-term storage), Email (analytics), YouTube (view counts) | ✓ Serverless, petabyte-scale SQL, 0-node management, GIS functions, ML via BQML ✗ Not for OLTP, query cost unpredictable (scans all data), streaming inserts expensive | Storage $0.020/GB/mo · Queries $5/TB scanned (first 1TB/mo free) · Streaming $0.010/200MB |
| Java Stream + parallel collectors // BigQuery = distributed SQL over large datasets
// Locally: parallel stream over in-memory collection
List<ClickEvent> events = loadAllEvents(); // fits in RAM?
Map<String, Long> byAd = events
.parallelStream() // multi-core
.collect(Collectors.groupingBy(
ClickEvent::adId,
Collectors.counting())); // GROUP BY ad_id
// For data > RAM: use Apache Flink/Spark locallyBigQuery = parallel columnar scan. Java parallel stream = same idea bounded by one machine RAM and cores. For > RAM: Flink/Spark. |
| Cloud CDN | CDN | Static asset caching, video streaming, API response caching, DDoS mitigation e.g. YouTube (HLS segment delivery), Google Maps (tile caching), URL Shortener | ✓ Tight integration with GCS/Cloud Run, anycast IP, signed URLs, cache invalidation API ✗ Less mature than CloudFront/Fastly, fewer edge PoPs (~100 vs CloudFront's 450+) | Cache egress $0.008/GB (Americas) · Cache fill $0.010/GB · HTTPS requests $0.0075/10K |
| Caffeine HTTP cache (same as CloudFront) // Cloud CDN = HTTP reverse-proxy cache
// Same local pattern as CloudFront
Cache<String, ResponseEntity<byte[]>> cdnCache =
Caffeine.newBuilder()
.maximumSize(1000)
.expireAfterWrite(1, HOURS)
.build();
ResponseEntity<byte[]> tile =
cdnCache.get("/tiles/z6/x12/y34.mvt",
key -> tileService.render(key));
// Invalidate on map data update
cdnCache.invalidate("/tiles/z6/*");CDN cache miss = origin fetch. Cloud CDN 100 PoPs = 100 Caffeine instances worldwide. Invalidation API = cache.invalidate(). |
AzureMicrosoft Azure — Key Services
Core Azure services mapped to system design patterns, with pros/cons, cost tiers, and CLI commands.
| Service | Category | Use In Systems | Pros / Cons | Cost | Key Commands | Java Single-Server Equiv. |
|---|---|---|---|---|---|---|
| Virtual Machines | Compute | General-purpose VMs, Windows workloads, SQL Server, SAP, HPC e.g. Scale From Zero (web tier), Stock Exchange (.NET matching engine), Email (Exchange compat) | ✓ Best Windows/SQL Server integration, hybrid cloud (Arc), Reserved Instances save 72% ✗ Complex pricing, slower provisioning than GCP, less Linux-native feel | B2s ~$0.046/hr · D2s v5 ~$0.096/hr · Spot VMs up to 90% discount |
| JVM Process (same as EC2/GCE) // Azure VM = your JVM process on a machine
// Windows VM = JVM on Windows OS
String os = System.getProperty("os.name");
long maxHeap = Runtime.getRuntime().maxMemory();
// Reserved Instance = long-running daemon
Thread daemon = new Thread(serverLoop);
daemon.setDaemon(false); // non-daemon = keeps JVM alive
daemon.start();
// Spot VM = preemptible (handle SIGTERM)
Runtime.getRuntime().addShutdownHook(
new Thread(this::gracefulShutdown));Azure Spot VM = preemptible process; register shutdown hook for graceful drain. Windows affinity = best for .NET/SQL Server workloads. |
| Blob Storage | Object Storage | Unstructured data, backups, media files, data lake (ADLS Gen2) e.g. YouTube (video blobs), Google Drive (file blocks), Email (MIME attachments) | ✓ ADLS Gen2 for big data analytics, lifecycle management, immutable storage for compliance ✗ Complex access tiers, slower than S3/GCS in some benchmarks, ACL model more complex | Hot LRS $0.018/GB/mo · Cool $0.01/GB/mo · Archive $0.00099/GB/mo · ops $0.0004/10K |
| java.nio.file.Files (same as S3/GCS) // Azure Blob = durable object store
Path blobRoot = Path.of("/mnt/blob-storage");
// Hot tier = SSD-backed path
Path hotBlob = blobRoot.resolve("hot/video.mp4");
Files.write(hotBlob, bytes);
// Cool tier = HDD path (cheaper, slower)
// Archive = tape (hours to retrieve)
// ADLS Gen2 = hierarchical namespace
Path dir = blobRoot.resolve("data/2026/01/");
Files.createDirectories(dir);Blob tiers (Hot/Cool/Archive) = SSD vs HDD vs tape analogy. ADLS Gen2 = true directory hierarchy (unlike flat S3 key namespacing). |
| Azure SQL / Cosmos DB | Database | Azure SQL: managed SQL Server. Cosmos DB: globally distributed multi-model NoSQL e.g. Hotel Reservation (Azure SQL), Nearby Friends (Cosmos DB), Ad Click (Cosmos change feed) | ✓ Cosmos DB: 5 consistency models, multi-region writes, <10ms p99. Azure SQL: full T-SQL ✗ Cosmos DB: expensive at scale, complex RU capacity planning. Azure SQL: Windows-centric | Azure SQL GP 2vCores ~$0.362/hr · Cosmos DB $0.016/RU/s-hr + $0.25/GB/mo |
| JDBC (SQL) / ConcurrentHashMap (Cosmos) // Azure SQL = JDBC, same as RDS/Cloud SQL
// Cosmos DB 5 consistency levels:
// Strong → synchronized read
// Bounded Staleness → volatile read
// Session → ThreadLocal<Version>
// Consistent Prefix → AtomicReference
// Eventual → plain field read
volatile long version = 0; // session consistency
AtomicReference<State> state = // consistent prefix
new AtomicReference<>();Cosmos 5 consistency levels map to Java memory model guarantees: synchronized > volatile > AtomicReference > plain read. |
| Azure Cache for Redis | In-memory Cache | Session store, distributed cache, leaderboard, pub/sub, rate limiting e.g. Rate Limiter (token bucket), News Feed (sorted sets), Chat (pub/sub routing) | ✓ Managed Redis, geo-replication, Enterprise tier with Redis modules (Search, JSON, TimeSeries) ✗ Enterprise tier very expensive, no Redis Cluster in Basic/Standard, limited to 730GB | C1 Basic (1GB) ~$0.055/hr · C2 Standard (6GB) ~$0.201/hr · P1 Premium ~$0.494/hr |
| Caffeine cache (same as ElastiCache) // Azure Cache for Redis = managed Redis
// Enterprise tier adds Redis modules:
// RediSearch → Lucene in-process
// RedisJSON → Jackson ObjectMapper cache
// RedisTimeSeries → RRD4J / Micrometer
Cache<String, SearchIndex> searchCache =
Caffeine.newBuilder()
.maximumSize(5_000)
.build();
// Redis ZADD for rate limiting
AtomicInteger reqCount = new AtomicInteger();
if (reqCount.incrementAndGet() > LIMIT) reject();Enterprise tier Redis modules: RediSearch = embedded Lucene. RedisTimeSeries = RRD4J or Micrometer. Rate limit = AtomicInteger + reset timer. |
| Service Bus / Event Hubs | Messaging | Service Bus: enterprise messaging, queues, topics. Event Hubs: Kafka-compatible streaming e.g. Notification System (Service Bus fan-out), Ad Click (Event Hubs stream), Metrics (ingest) | ✓ Event Hubs: Kafka protocol support (zero code change), dead-letter queues, AMQP support ✗ Event Hubs: 90-day max retention (vs Kafka unlimited), no consumer group offset management | Service Bus Standard $0.10/million ops · Event Hubs Basic $0.028/million events |
| LinkedBlockingQueue (SB) / Disruptor (EH) // Service Bus = durable queue with sessions
BlockingQueue<Message> serviceBus =
new PriorityBlockingQueue<>(); // ordered delivery
// Sessions = per-key ordering guarantee
Map<String, BlockingQueue<Message>> sessions =
new ConcurrentHashMap<>();
// Event Hubs = Kafka-compatible, use Disruptor
RingBuffer<Event> eventHub =
Disruptor.createMultiProducer(
Event::new, 4096, MULTI);
// Consumer group = independent ring buffer cursorService Bus sessions = per-key FIFO queue. Event Hubs partition = Disruptor RingBuffer with independent consumer cursors. |
| Azure Functions | Serverless | Event-driven code, HTTP APIs, timer jobs, blob triggers, queue processing e.g. Ad Click (event processor), Notification (channel worker), URL Shortener (redirect) | ✓ Durable Functions for stateful workflows (fan-out/fan-in), tight Azure integration, C#/.NET native ✗ Cold starts in Consumption plan, 5-min default timeout (extend to 10min), .NET bias | First 1M executions free · $0.20/million executions · $0.000016/GB-second |
| Virtual Threads / CompletableFuture // Azure Functions = event-driven, scales to zero
// Durable Functions = stateful workflow
// Locally: CompletableFuture chaining
CompletableFuture
.supplyAsync(() -> readInput(event)) // Activity 1
.thenApplyAsync(data -> process(data)) // Activity 2
.thenAcceptAsync(result -> store(result)) // Activity 3
.exceptionally(ex -> {
compensate(event); return null; // saga rollback
});Durable Functions fan-out/fan-in = CompletableFuture.allOf(). Saga compensation = exceptionally() rollback. Timer trigger = ScheduledExecutorService. |
| AKS | Kubernetes | Containerized microservices, stateful apps, ML workloads at scale e.g. Scale From Zero (web tier), Chat (WebSocket fleet), KV Store (Cassandra on AKS) | ✓ Free control plane, Windows node pool support, Azure AD integration, Dapr support ✗ Slower cluster provisioning, complex networking (CNI choices), AAD setup friction | Control plane free · Pay for VM nodes · System node pool min ~$0.096/hr (D2s v3) |
| Thread pool + health check (same as EKS/GKE) // AKS = managed Kubernetes on Azure
// Windows node pool = JVM on Windows
ExecutorService windowsPool =
Executors.newFixedThreadPool(4);
// Dapr sidecar = in-process interceptor
var interceptor = new DaprInterceptor();
interceptor.onInvoke((req, next) -> {
log(req);
return next.apply(req); // pass-through
});
// AAD Workload Identity = SecurityManagerAKS Windows node = JVM on Windows (best for .NET). Dapr sidecar = AOP interceptor for state/pub-sub/service invocation abstraction. |
| Azure CDN / Front Door | CDN & Edge | CDN: static assets. Front Door: global load balancer + WAF + CDN in one e.g. YouTube (HLS delivery), Google Maps (tiles), URL Shortener (redirect caching) | ✓ Front Door: global Anycast, intelligent routing, WAF, automatic failover between regions ✗ Front Door Premium expensive, CDN PoP count lower than Cloudflare/Akamai, complex pricing | CDN Standard $0.0075/GB (Zone 1) · Front Door $0.01/GB data transfer + $35/mo base |
| Caffeine + load balancer map // Front Door = global LB + WAF + CDN
// Locally: cache + routing map
Cache<String, Response> cdnCache =
Caffeine.newBuilder()
.expireAfterWrite(1, HOURS).build();
// Intelligent routing = latency-weighted map
Map<String, Integer> backends = Map.of(
"us-east", 12, // latency ms
"eu-west", 89,
"ap-south", 156);
String best = backends.entrySet().stream()
.min(Map.Entry.comparingByValue())
.map(Map.Entry::getKey).orElseThrow();Front Door Anycast = route to lowest-latency backend. Locally: min-latency Map lookup. WAF = Servlet Filter for request validation. |
| Azure Monitor / Log Analytics | Observability | Metrics, logs, traces, alerts, dashboards for all Azure services and custom apps e.g. Metrics Monitoring (Azure-native), Scale From Zero (health checks), Payment (audit logs) | ✓ Unified platform for metrics+logs+traces, Application Insights for APM, KQL query language ✗ KQL has learning curve, data ingestion costs escalate quickly, retention costs extra beyond 90d | Basic metrics free · Log Analytics $0.30/GB ingest (first 5GB/mo free) · Alerts $0.10/rule/mo |
| Micrometer + SLF4J + OpenTelemetry // Azure Monitor = metrics + logs + traces
// Micrometer (metrics)
MeterRegistry registry = new PrometheusMeterRegistry();
Counter reqs = registry.counter("http.requests");
reqs.increment();
// SLF4J (logs → Log Analytics)
log.info("payment.processed",
kv("amount", 100), kv("userId", id));
// OpenTelemetry (traces → App Insights)
Span span = tracer.spanBuilder("processOrder").startSpan();
try { ... } finally { span.end(); }Application Insights = OpenTelemetry SDK. Log Analytics KQL query = structured log search. Alert = Micrometer gauge threshold watch. |
| Azure OpenAI / Cognitive | AI / ML | LLM inference, embeddings, speech-to-text, vision, document intelligence e.g. Autocomplete (semantic suggestions), Email (spam classification), Ad Click (fraud ML) | ✓ Enterprise SLA for OpenAI models (GPT-4, DALL-E), private endpoints, compliance certifications ✗ Regional availability limited, quota constraints, model lag behind OpenAI.com releases | GPT-4o $5/million input tokens · $15/million output tokens · embeddings $0.13/million tokens |
| DJL / OpenNLP / local model inference // Azure OpenAI = hosted LLM inference
// Locally: Deep Java Library (DJL) or ONNX
Model model = Model.newInstance("gpt2");
Predictor<String,String> predictor =
model.newPredictor(translator);
String result = predictor.predict(prompt);
// Or: call local Ollama via HTTP
HttpClient client = HttpClient.newHttpClient();
HttpRequest req = HttpRequest.newBuilder()
.uri(URI.create("http://localhost:11434/api/generate"))
.POST(BodyPublishers.ofString(
"{\"model\":\"llama3\",\"prompt\":\"" + prompt + "\"}"))
.build();Azure OpenAI = hosted GPU inference. Locally: DJL for Java-native ML, or Ollama for LLM via HTTP. No local equivalent matches GPT-4 quality. |
No results found. Try a different search term.