System Design Interview Questions and Answers

I do not start drawing boxes. First I clarify requirements: which features are in scope, and the non-functional needs such as expected users, latency, availability and consistency. Then I estimate scale roughly, define the core APIs and data model, and sketch a high-level design that works at all.

After that I pick the one or two hardest parts, for example matching nearby drivers, and go deep, naming alternatives and trade-offs. I finish with failure modes, bottlenecks and what I would monitor. The interviewer is judging structure and reasoning, so I think aloud, state assumptions, and invite correction rather than hunting for one "right" architecture.

What the interviewer is checking: Whether you bring a repeatable structure, ask clarifying questions, and treat the answer as a trade-off discussion.

Likely follow-up: The interviewer says "assume 10 million users" and nothing else. What do you ask next?

Vertical scaling means giving one machine more CPU, memory or disk. It is simple, since nothing in the application changes, but it hits a hardware ceiling, costs more per unit at the top end, and leaves a single point of failure. Horizontal scaling means adding more machines and spreading the load across them.

Horizontal scaling gives more headroom and fault tolerance, but your design must allow it: services should be stateless (session data in a shared store, not in process memory), and the data layer needs replication or sharding. In practice I scale up first while it is cheap, then scale out the stateless tier, and treat the database as the hard part.

What the interviewer is checking: That you know scaling out requires design changes, especially statelessness, and that the data tier is the hard part.

Likely follow-up: Your web tier keeps login sessions in memory. What breaks when you add a second server, and how do you fix it?

A load balancer spreads incoming requests over a pool of servers, runs health checks to stop sending traffic to unhealthy ones, and gives clients one stable address. Layer 4 balancers route on IP and port and are fast; layer 7 balancers understand HTTP, so they can route by path or header, terminate TLS and do sticky sessions.

Algorithms: round robin (simple, assumes equal requests), weighted round robin for unequal servers, least connections for long or uneven requests, and IP or key hashing when the same client should hit the same server. The balancer itself must not be a single point of failure, so it is run as a redundant pair or a managed service.

What the interviewer is checking: Knowledge of L4 versus L7, health checks, and picking an algorithm to fit the traffic.

Likely follow-up: When would sticky sessions be a bad idea?

Caching stores the result of expensive work closer to where it is needed, so repeated requests are faster and the backend does less. Layers include the browser, a CDN at the edge, a reverse proxy, an application-level cache such as Redis or in-process memory, and the database's own buffer cache.

The costs are staleness and complexity: you must decide how long data may be out of date and how entries get invalidated. Caches work best for read-heavy data that changes rarely and is requested repeatedly, and you measure success by hit ratio. Data that must always be exact, such as an account balance used for a payment, is a poor candidate. See the Redis course for hands-on patterns.

What the interviewer is checking: That you see caching as a layered trade-off between speed and freshness, not just "put Redis in front".

Likely follow-up: How would you decide what the TTL should be for a product-details cache?

A content delivery network is a set of edge servers spread around the world that cache copies of your content near users. A request goes to the nearest edge; on a hit it is served immediately, and on a miss the edge fetches from your origin and keeps a copy. That cuts latency and shields your origin from load.

Static assets such as images, scripts, stylesheets and video segments are the obvious fit, and most CDNs can also cache API responses that are public and cacheable. Control freshness with cache headers, and use versioned file names (such as app.3f9a.js) so a new release is picked up immediately without waiting for expiry. Personalised or private responses should not be cached publicly.

What the interviewer is checking: Practical knowledge of edge caching, cache headers and safe invalidation by versioning.

Likely follow-up: A wrong image was published and is cached at the edge worldwide. How do you fix it quickly?

I start from the access patterns. If the data is relational, needs joins and ad hoc queries, or needs multi-row transactions (orders, payments, bookings), a relational database is the safe default and scales further than people assume with replicas, caching and later sharding.

I reach for NoSQL when a specific need justifies it: a key-value store for sessions and counters, a document store for flexible nested records, a wide-column store for very high write volume across many nodes, a graph database for relationship traversal. The trade-off is usually weaker joins and transactions in exchange for easier horizontal scaling. I name the access pattern out loud when I choose.

What the interviewer is checking: Reasoning from queries and consistency needs rather than from trends.

Likely follow-up: Could you build the same feature on both? What would you give up on each?

Latency is how long one request takes, best measured as percentiles (p50, p95, p99) because averages hide slow outliers. Throughput is how much work is done per unit of time, such as requests per second. Availability is the fraction of time the system is working.

"Three nines" means 99.9% availability, which allows about 8.76 hours of downtime a year (0.1% of 8,760 hours). Each extra nine cuts that by ten, so four nines is under an hour. Higher availability needs redundancy and automated failover, and gets sharply more expensive, so the target should come from what the business actually needs.

SECONDS_PER_YEAR = 365 * 24 * 3600
for nines in ["99", "99.9", "99.99", "99.999"]:
    down = SECONDS_PER_YEAR * (1 - float(nines) / 100)
    print(f"{nines:>7}% -> {down / 3600:8.2f} hours/year  ({down / 60:9.1f} min)")

     99% ->    87.60 hours/year  (   5256.0 min)
   99.9% ->     8.76 hours/year  (    525.6 min)
  99.99% ->     0.88 hours/year  (     52.6 min)
 99.999% ->     0.09 hours/year  (      5.3 min)

What the interviewer is checking: Fluency with basic metrics, including percentiles over averages and the arithmetic behind availability targets.

Likely follow-up: If service A (99.9%) calls service B (99.9%) on every request, what is the best availability of the combination?

A queue decouples producer and consumer in time and speed. The producer hands off work and returns quickly, the consumer processes at its own pace, and a traffic spike is absorbed as a backlog instead of an outage. If the consumer is down, messages wait and are processed on recovery.

It also lets you scale consumers independently and retry failures without involving the user. The costs: the work is now asynchronous and eventually consistent, ordering and duplicates need thought, and you have a new component to run and monitor. Good fits are emails, image processing and payment follow-ups; a request that needs an immediate answer should stay synchronous.

What the interviewer is checking: Whether you can justify asynchronous messaging and name what it costs in return.

Likely follow-up: The queue keeps growing and the consumers cannot keep up. What are your options?

In cache-aside the application checks the cache, loads from the database on a miss and fills the cache. It is the most common pattern and tolerates cache failure. Write-through writes to cache and database together, so reads are fresh but writes are slower. Write-back writes to the cache first and flushes to the database later, which is fast but can lose data if the cache dies.

For correctness, the usual approach is a TTL plus deleting the key when the source data changes. Watch for a stampede: when a hot key expires, thousands of requests hit the database at once. Mitigate with request coalescing, jittered TTLs, or refreshing in the background. Remember the hardest part is invalidation, so keep the number of cached views small.

What the interviewer is checking: Knowing the named patterns, their failure modes, and a concrete invalidation story.

Likely follow-up: A user updates their profile and still sees the old one for a minute. What went wrong and how do you fix it?

In leader-follower replication all writes go to one leader, which streams changes to followers. Reads can be served by followers, so read capacity scales and a follower can be promoted if the leader fails. Replication is synchronous (the leader waits for acknowledgement, safer but slower) or asynchronous (faster, but recent writes can be lost on failover).

With asynchronous followers there is lag, so a user may save a change and then read an old value from a follower. Common fixes: read your own recent writes from the leader, route by a session timestamp, or only send reads to followers that have caught up. Multi-leader and leaderless setups remove the single write point but introduce conflict resolution.

What the interviewer is checking: Understanding of the consistency consequences of replication, not just that it improves read throughput.

Likely follow-up: How do you decide when to promote a follower, and what could go wrong with a split-brain situation?

Sharding splits a dataset across multiple databases so no single node holds or serves everything. A shard key decides where each row lives: hash partitioning spreads load evenly but makes range queries hard, range partitioning keeps ranges together but can create hot shards, and a lookup directory is flexible but is another thing to maintain.

A good key has high cardinality, spreads writes evenly, and matches the most common query so one request touches one shard. Pain points: cross-shard joins and transactions, rebalancing when you add shards, and hot keys such as one very popular user. Sharding comes after simpler fixes (indexes, caching, read replicas), because it permanently complicates the application.

What the interviewer is checking: Whether you pick a key from query patterns, anticipate hotspots, and know sharding is a last resort.

Likely follow-up: Your tenant ID is the shard key and one customer is 40% of all traffic. What now?

CAP says that when a network partition happens, a distributed data store must choose between consistency (every read sees the latest write) and availability (every request to a live node gets a response). Partition tolerance is not optional in a real network, so the real choice is C or A during a partition.

It is often misquoted as "pick any two". When there is no partition you can have both, and the practical trade-off becomes latency versus consistency. A system favouring consistency may reject or delay requests on the minority side of a partition, while one favouring availability serves possibly stale data and reconciles later. Many databases make this tunable per operation, so describe the behaviour you need rather than labelling a product CP or AP.

What the interviewer is checking: Precision: that you state the partition condition and avoid the slogan version.

Likely follow-up: For a shopping cart versus a bank transfer, which side would you favour and why?

Strong (linearizable) consistency makes the system behave as if there were one copy: once a write returns, every later read sees it. Causal consistency preserves cause and effect, so a reply never appears before the post it answers. Read-your-writes guarantees you see your own updates. Eventual consistency promises only that replicas converge if writes stop.

Stronger models are easier to reason about but cost latency and availability, since nodes must coordinate. With quorum replication, reading from R replicas and writing to W of N replicas gives strong reads when R + W > N. Pick the weakest model your feature tolerates: a like counter can be eventually consistent, a seat reservation cannot.

What the interviewer is checking: Whether you can map product requirements to a consistency level and justify the cost.

Likely follow-up: With N=3, what do you pick for R and W if reads are ten times as frequent as writes?

In a classic queue each message is delivered to exactly one consumer among those competing for it, which suits distributing tasks across workers. In publish/subscribe a message is delivered to every subscriber, which suits broadcasting an event such as "order placed" to billing, email and analytics independently.

Log-based systems like Kafka combine both: a topic is split into partitions that preserve order within a partition, each consumer group receives every message once (partitions are divided among its consumers), and different groups each get their own full copy. Delivery is typically at-least-once, so consumers must tolerate duplicates. See the Kafka course for more.

What the interviewer is checking: Clear distinction between competing consumers and fan-out, plus knowledge of partitions and consumer groups.

Likely follow-up: How do you keep all events for one order in sequence when a topic has many partitions?

An operation is idempotent if doing it several times has the same effect as doing it once. It matters because networks fail ambiguously: a client that times out does not know whether the server processed the request, so it retries, and without protection you charge the customer twice.

The standard design is an idempotency key sent by the client with each logical request. The server stores the key with the result in a table with a UNIQUE constraint, and on a repeat returns the stored result instead of acting again. Store the key and the effect in the same transaction, and expire old keys. Naturally idempotent operations, like setting a value, are safe without a key; "add 30" is not.

processed = {}      # in production: a table with a UNIQUE key on the idempotency key
balance = {"acct": 100}

def charge(idem_key, amount):
    if idem_key in processed:
        return processed[idem_key]          # replay: return the stored result
    balance["acct"] -= amount
    result = {"status": "charged", "balance": balance["acct"]}
    processed[idem_key] = result
    return result

print(charge("order-77", 30))
print(charge("order-77", 30))   # client retried after a timeout
print(charge("order-78", 30))

{'status': 'charged', 'balance': 70}
{'status': 'charged', 'balance': 70}
{'status': 'charged', 'balance': 40}

What the interviewer is checking: That you tie idempotency to retries and can describe a concrete, race-safe implementation.

Likely follow-up: Two identical requests arrive at the same instant on different servers. How does your design stay correct?

First decide what to limit (user, API key, IP), the limits, and what happens on rejection (usually HTTP 429 with a retry hint). A fixed window counter is simplest but allows a burst of double the limit across a window boundary. A sliding window is more accurate but uses more memory. A token bucket adds tokens at a steady rate up to a capacity and spends one per request, allowing short bursts while enforcing a long-run average.

Token bucket is a common default because it is cheap and burst-friendly. The sketch below uses a fake clock so it is deterministic. In a real service the state sits in a shared store so all servers agree. The rate limiting course goes deeper.

class TokenBucket:
    def __init__(self, capacity, refill_per_sec):
        self.capacity = capacity
        self.refill = refill_per_sec
        self.tokens = capacity
        self.last = 0.0

    def allow(self, now):
        self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.refill)
        self.last = now
        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

b = TokenBucket(capacity=3, refill_per_sec=1)
for t in [0.0, 0.1, 0.2, 0.3, 1.5, 1.6]:
    print(f"t={t:<4} allowed={b.allow(t)}")

t=0.0  allowed=True
t=0.1  allowed=True
t=0.2  allowed=True
t=0.3  allowed=False
t=1.5  allowed=True
t=1.6  allowed=False

What the interviewer is checking: Knowledge of the algorithms and their trade-offs, plus the product decisions around limits and responses.

Likely follow-up: Where would you place the limiter: in the API gateway, in each service, or both?

I state assumptions, then do simple arithmetic. Suppose 100 million new URLs a day and ten reads for every write. A day has about 86,400 seconds, so writes are roughly 1,200 a second and reads about 12,000 a second on average, with peaks higher. At about 500 bytes a record, five years of data is around 90 TB.

The point is not precision but deciding what matters: 12,000 reads a second is easy with a cache in front, 90 TB means we need partitioned storage, and a 7-character base-62 code gives 62^7, about 3.5 trillion combinations, which is plenty. I round numbers, keep units visible, and say which assumption I would validate first.

new_urls_per_day = 100_000_000
read_write_ratio = 10
bytes_per_record = 500
years = 5

writes_per_sec = new_urls_per_day / 86_400
reads_per_sec = writes_per_sec * read_write_ratio
storage_tb = new_urls_per_day * 365 * years * bytes_per_record / 1e12

print(f"writes/sec  ~ {writes_per_sec:,.0f}")
print(f"reads/sec   ~ {reads_per_sec:,.0f}")
print(f"storage     ~ {storage_tb:,.0f} TB over {years} years")
print(f"62^7 codes  = {62**7:,}")

writes/sec  ~ 1,157
reads/sec   ~ 11,574
storage     ~ 91 TB over 5 years
62^7 codes  = 3,521,614,606,208

What the interviewer is checking: Comfort with estimation and, more importantly, using the numbers to drive design decisions.

Likely follow-up: Peak traffic is five times the average. What changes in your design?

Two operations: create a short code for a long URL, and redirect a code to its URL. Data is a simple key-value mapping, so a key-value or sharded relational store works. To generate codes, either encode a unique ID in base 62 (an ID service avoids collisions) or hash the URL and truncate, which needs collision checks. Custom aliases need a uniqueness check.

Reads dominate, so put a cache in front of the store and serve hot codes from memory. Redirect with HTTP 302 if you want every click to reach you for analytics, or 301 if you accept browsers caching it permanently. Add expiry, abuse and malware checks, and rate limiting on creation. Analytics should go through a queue so the redirect path stays fast.

What the interviewer is checking: Whether you can turn requirements into a small, coherent design and justify choices like code generation and redirect type.

Likely follow-up: How would you stop two servers from generating the same short code?

Retry only transient failures (timeouts, 503s), only for idempotent operations or with an idempotency key, and with a limit. Exponential backoff waits longer after each failure, for example doubling up to a cap, so a struggling service gets room to recover.

Add jitter, meaning a random wait up to the current ceiling. Without it, thousands of clients that failed together retry together and hammer the service in synchronised waves. Also bound the total time, respect a retry-after header, and be aware that retries multiply load across layers: three layers each retrying three times can turn one request into twenty-seven. A retry budget or retrying only at one layer helps.

def backoff_ceiling(attempt, base=0.2, cap=10.0):
    return min(cap, base * 2 ** attempt)     # full jitter picks random(0, ceiling)

for attempt in range(8):
    print(f"retry {attempt}: wait a random time up to {backoff_ceiling(attempt):.1f}s")

retry 0: wait a random time up to 0.2s
retry 1: wait a random time up to 0.4s
retry 2: wait a random time up to 0.8s
retry 3: wait a random time up to 1.6s
retry 4: wait a random time up to 3.2s
retry 5: wait a random time up to 6.4s
retry 6: wait a random time up to 10.0s
retry 7: wait a random time up to 10.0s

What the interviewer is checking: Whether you know retries can cause outages (storms, amplification) and how to bound them.

Likely follow-up: Why is blindly retrying a non-idempotent POST dangerous?

A timeout limits how long you wait for a dependency. Without one, a slow downstream service ties up threads or connections until your own service stalls, turning one failure into a cascade. Every network call needs a timeout, chosen from the dependency's real latency percentiles.

A circuit breaker watches recent calls. While healthy it is closed; after too many failures it opens and fails fast for a cooling-off period instead of piling up more doomed requests; then it goes half-open to let a few trial calls through and closes again if they succeed. Pair it with a fallback (cached data, a default, a degraded feature) and with bulkheads that keep separate thread or connection pools per dependency.

What the interviewer is checking: Understanding of cascading failure and the standard protective patterns, with the states named correctly.

Likely follow-up: What would you return to the user when the recommendations service is down?

With hash(key) % N, changing N from 4 to 5 changes the result for most keys, so most cached or stored data must move, and a cache cluster would lose nearly all its entries at once. Consistent hashing places nodes and keys on a ring and assigns each key to the next node clockwise, so adding or removing a node moves only the keys in its neighbouring range, roughly 1/N of them.

Virtual nodes (each server appears at many points on the ring) smooth out the distribution and let stronger machines take more. The experiment below measured 80% of keys moving with modulo versus 22% with the ring. It is used in distributed caches, partitioned stores and load balancers that need affinity.

import bisect, hashlib

def h(s):
    return int(hashlib.md5(s.encode()).hexdigest(), 16)

class Ring:
    def __init__(self, nodes, vnodes=100):
        self.points = sorted((h(f"{n}#{i}"), n) for n in nodes for i in range(vnodes))
        self.keys = [p for p, _ in self.points]
    def node_for(self, key):
        i = bisect.bisect(self.keys, h(key)) % len(self.points)
        return self.points[i][1]

keys = [f"user:{i}" for i in range(10_000)]

# naive: hash(key) % number_of_nodes
before = {k: h(k) % 4 for k in keys}
after = {k: h(k) % 5 for k in keys}
moved = sum(before[k] != after[k] for k in keys)
print(f"modulo 4 -> 5 nodes: {moved / len(keys):.0%} of keys moved")

r4 = Ring(["n1", "n2", "n3", "n4"])
r5 = Ring(["n1", "n2", "n3", "n4", "n5"])
moved = sum(r4.node_for(k) != r5.node_for(k) for k in keys)
print(f"ring 4 -> 5 nodes:   {moved / len(keys):.0%} of keys moved")

modulo 4 -> 5 nodes: 80% of keys moved
ring 4 -> 5 nodes:   22% of keys moved

What the interviewer is checking: Whether you understand the rebalancing problem and the role of virtual nodes, not just the name.

Likely follow-up: What happens to the cache hit ratio and to the database when one cache node dies?

Services call a notification API with an event type and recipient. The API checks user preferences and opt-outs, renders a template, applies per-user rate limits, and places a message on a queue, with a separate queue (or topic) per channel so a slow SMS provider cannot block email. Workers consume each queue and call the external providers.

Reliability matters most. Retry with backoff, send repeatedly failing messages to a dead-letter queue, and use an idempotency key per notification so a retry never sends a duplicate. Add priorities (a password reset beats a promotion), scheduling, and delivery tracking for status and analytics. Remember that providers are third parties that fail and throttle, so design for that.

What the interviewer is checking: Ability to produce a sensible asynchronous pipeline and think through duplicates, priorities and third-party failure.

Likely follow-up: How do you make sure a user does not receive the same alert twice if a worker crashes mid-send?

With fan-out on write (push), when someone posts, the system immediately inserts the post ID into each follower's precomputed feed list in a cache. Reading a feed is then very fast, but a user with millions of followers makes one post trigger millions of writes. With fan-out on read (pull), nothing is precomputed; at read time you fetch recent posts from everyone the user follows and merge them. Writes are cheap, reads are slow and expensive.

A hybrid is common: push for ordinary accounts, pull for accounts with huge audiences, merging the two at read time. Store feeds as post IDs, fetch the post bodies separately, paginate with a cursor, and keep ranking as a separate step. The choice depends on read-to-write ratio and follower-count skew.

What the interviewer is checking: Seeing the central trade-off, handling the celebrity skew, and describing a workable hybrid.

Likely follow-up: A user unfollows someone. What happens to the posts already in their precomputed feed?

Per-server counters let a client exceed the limit by the number of servers, so the state must be shared, typically in Redis keyed by user and window. The main hazard is a race: two servers read the counter, both see room, and both allow the request. The check and the increment must be one atomic step, using INCR (which is atomic) with an expiry set on first use, or a Lua script for token bucket or sliding window logic.

Consider latency and failure too. A shared store adds a network hop per request, so some systems allow a small local allowance and sync periodically, accepting slight overshoot. Decide what happens if the store is down: fail open (allow traffic, risking abuse) or fail closed (block, risking an outage), and make that choice explicit.

What the interviewer is checking: Spotting the read-modify-write race and thinking about the failure behaviour of the limiter itself.

Likely follow-up: How would you give each customer plan a different limit without a redeploy?

Over an unreliable network, a sender that does not get an acknowledgement cannot know if the message arrived, so it either retries (at-least-once, possible duplicates) or does not (at-most-once, possible loss). Genuine exactly-once delivery is not achievable in general; what you can build is effectively-once processing.

That means at-least-once delivery plus an idempotent consumer: deduplicate on a message ID, or make the operation naturally repeatable. Some platforms offer transactional features inside their own boundary, but a side effect outside, like sending an email or charging a card, still needs its own idempotency. For publishing events reliably, use the outbox pattern: write the event to an outbox table in the same database transaction as the state change, and a separate relay publishes it.

What the interviewer is checking: Honesty about the limits of distributed systems and knowledge of the outbox and deduplication techniques.

Likely follow-up: Why is writing to the database and then publishing to a queue as two separate steps risky?

Observability rests on three signals. Metrics are cheap aggregates over time (rates, error counts, latency percentiles) for dashboards and alerts. Logs are structured event records for detail. Traces follow one request across services with a shared correlation ID, showing where the time went. For services, watch the four golden signals: latency, traffic, errors and saturation.

Define service level indicators and objectives, for example "99.9% of requests succeed under 300 ms", and alert on symptoms users feel, such as burn of the error budget, rather than on every CPU spike. Keep metric labels low-cardinality, log with request IDs, and make sure alerts link to runbooks. Observability is how you find unknown problems; monitoring only catches ones you predicted.

What the interviewer is checking: A systematic approach to detecting and diagnosing issues, not a list of tool names.

Likely follow-up: Latency p99 doubled but the average did not move. Where do you look first?

Split when the pain is real: many teams blocking each other in one codebase and deploy, parts of the system with very different scaling or availability needs, or a clear domain boundary that changes at its own pace. Microservices give independent deployment and scaling, at the price of network calls, distributed data, partial failure, versioned contracts, and much heavier operations and observability needs.

For a small team or an unsettled product, a well-structured modular monolith is usually faster and safer. When I do split, I follow business boundaries, give each service its own data, avoid sharing a database, and extract one piece at a time. Starting with microservices "for scale" before there is any scale is a common and expensive mistake. See the microservices course.

What the interviewer is checking: Balanced judgement about organisational and technical costs rather than enthusiasm for a trend.

Likely follow-up: Two services must update their own databases in one business operation. How do you keep them consistent?