Design Bitly: A URL Shortener for Billions of Redirects

Scope the problem before drawing anything. The two core features are shortening and redirecting. A user submits a long URL and receives a short one; later, anybody holding the short link, logged in or not, should land on the original page. Everything else in the system exists to make those two flows correct and quick.

Two optional features are worth naming because they change the data model. A custom alias such as my-sale means codes are not always machine-generated, so the system must reject an alias that is already taken. An expiration date means a stored mapping can become invalid, which affects caching and cleanup later. Click analytics is deliberately parked: it is a separate pipeline and would distract from the core path during the first half of the interview.

The non-functional list is where the design gets its shape. Redirects should complete in under 100 ms, because a short link is just a detour on the way to the page the user actually wants, and any delay feels like the destination site is slow. Availability matters just as much: assume a 99.99% target, since a shortener outage breaks links embedded in emails, posters and social posts everywhere at once.

Uniqueness is a correctness requirement, not a nice-to-have. If two long URLs ever share a code, someone gets sent to the wrong page, which is both a bug and a potential security problem. Finally, the workload is lopsided: a link is created a single time and then clicked over and over. That imbalance tells us to spend most of our engineering effort on the lookup path rather than the creation path.

For the estimate, assume roughly 1 billion stored short URLs and around 100 million daily active users who together trigger about 1 billion redirects per day. Spread across 86,400 seconds, that comes to roughly 12K redirects per second on average. Traffic is never flat, so plan for a peak of around 50K per second when a popular link is shared widely.

Storage turns out to be modest. If each row holds the code, the long URL, an owner, timestamps and an optional expiry in about 500 bytes, a billion rows is roughly 500 GB. That fits comfortably on a handful of machines, so storage volume is not the bottleneck. The number that drives the architecture is the read-to-write ratio of about 1000 to 1: optimise the redirect path first, and keep the creation path simple and correct.

The data model needs only three entities: the original URL, the short URL identified by its code, and the user who created it. Keeping the model this small makes the lookup a pure key-value access, which pays off later when we shard and cache.

Creation is a POST /urls call whose body carries the long_url plus optional custom_alias and expires_at fields, and the response returns the finished short URL. Following a link is a plain GET /{short_code} that answers with an HTTP redirect pointing at the long URL. Notice the redirect endpoint sits at the root path so the short link stays as short as possible, and it needs no authentication, since anyone holding the link should be able to use it.

Requests from browsers and apps first reach a load balancer, which distributes them across a fleet of application servers. Because creation and redirection have such different traffic profiles, the design splits them into a write service and a read service. Each can then be scaled, deployed and tuned on its own terms; a burst of clicks on a viral link never slows down link creation.

On the write side, the service asks a counter for the next unique integer, encodes it into a short code, and stores the code-to-URL mapping in the database. On the read side, the service checks a Redis cache before touching the database, because a small fraction of links receives most of the clicks. A miss falls through to the database, and the result is written back into the cache for the next visitor.

Neither service keeps any session or per-user state in memory. That is the property that lets us add instances freely behind the load balancer as traffic climbs, and it means a crashed instance can be replaced without any data recovery step.

There are three common ways to mint a code. Hashing the long URL (for example MD5) and keeping the first seven characters is easy and even deduplicates identical URLs, but truncation creates collisions, so each write must check the database and retry with a tweak. Generating a random string has similar issues: it is unpredictable, which is nice, yet every write still needs an existence check, and the chance of a clash rises as the table fills.

The chosen approach is a global counter encoded in base62, the alphabet of digits plus lowercase and uppercase letters. Each new link receives the next integer, and since integers never repeat, the codes never repeat either, so no collision check is needed. Seven base62 characters cover 62 to the power of 7, roughly 3.5 trillion values, which is thousands of times the billion links we assumed.

A single counter that every write hits would become a hot spot, so each write server reserves a block of, say, 1,000 IDs from Redis and hands them out locally. The shared counter is touched only once per thousand links. If predictable sequential codes worry you, apply a reversible scrambling function to the integer before encoding; uniqueness is preserved and neighbouring links no longer look alike.

Meeting the 100 ms target is about stopping each request as early as possible. The outermost layer is a CDN or edge cache that can answer redirects for the hottest links from a location near the user, so those requests never cross the ocean to our servers. Anything the edge cannot serve reaches the read service, which looks in Redis. A Redis lookup takes well under a millisecond, and with a heavily skewed click distribution most requests end there.

Only on a cache miss does the read service query the database, and even that is cheap because the short code is the primary key. The fetched mapping is written back into Redis with LRU eviction, which naturally keeps popular links resident and lets stale ones age out.

The final step is choosing the status code. A 301 tells the browser the move is permanent, so it remembers the destination and skips us on later clicks, which saves load. A 302 marks the move as temporary, so every click comes back through our servers. Pick 302 if you need click counts, expiring links or the ability to edit a destination; pick 301 if raw traffic reduction matters more.

Scaling follows from the earlier choices. The read service is stateless, so handling more clicks means adding instances behind the load balancer. The database stores simple key-value pairs, which makes sharding by short code straightforward: hash the code to pick a shard, and attach read replicas when lookups outgrow a single primary. The cache becomes a Redis cluster, spreading hot keys across nodes and promoting a replica if a node dies.

The counter needs special care because it is the one component that must never repeat itself. Persist its value so a restart cannot hand out an already-used range. The batching from the code-generation deep dive also buys resilience: servers holding unused IDs can keep creating links through a brief counter outage. Finally, a scheduled background job deletes expired mappings so the table does not grow without bound and stale links stop resolving.

Every decision here had a cost. Choosing between 301 and 302 trades lower traffic for control and visibility over each click. A counter guarantees unique codes, but you now operate a small piece of critical infrastructure that must be durable. Caching makes redirects fast, yet a deleted or expired link has to be evicted from the cache as well, or users will keep landing on a destination that should be gone.

The storage engine is less contentious than candidates often fear. A key-value store shards effortlessly for this access pattern, while a relational database offers constraints such as a unique index on custom aliases. Either is defensible if you explain the reasoning. The thread running through the whole design is simple: favour the read path, make uniqueness structural rather than checked, and keep services free of local state so they can scale out.