Design Instagram: Photo Sharing with Hybrid Feed Fan-out

Three features matter. Users create a post containing a photo or short video plus a caption. Users follow other accounts, and the relationship is one-directional: a fan can follow a famous athlete without the athlete following back. That asymmetry is what produces the extreme follower counts that complicate the feed later.

Users also view a home feed of recent posts from the accounts they follow. To keep the scope tight, the feed is chronological, newest first, rather than ranked by a model. Stories, direct messages, comments and machine-learned ranking are all real features, but each could fill its own interview, so they are set aside here.

The home feed should load in under 500 ms, because it is the very first screen after opening the app and any delay is felt by every user, every session. Media must also load fast globally, which in practice means serving it from edge locations near viewers rather than from one central region.

Availability takes priority over strict consistency. If a friend's new photo reaches your feed a few seconds after they post it, you will never notice, but an app that refuses to load the feed during a partial outage is immediately noticed. Finally, assume massive scale, around 500 million daily active users, which rules out any approach that does heavy per-request work on a single database.

Assume 500 million daily active users creating about 100 million posts per day, roughly 1,150 posts per second. Assuming the typical person checks their feed around ten times daily, that yields about 5 billion feed reads per day, near 58K per second on average.

With an average of about 2 MB of media per post, new uploads total roughly 200 TB per day, which belongs in object storage rather than any database. The most important ratio is between feed reads and posts: reads outnumber posts about fifty to one. That pushes the design toward doing preparation work when a post is created so that each of those far more frequent feed loads becomes a cheap lookup.

The data model has four entities: users, posts, follow relationships, and the media objects attached to posts. Keeping media separate from the post record lets the heavy bytes live in object storage while the post row stays small.

Publishing is POST /posts with a caption and the media type. The response returns a post_id and an upload URL, so the client sends the photo or video directly to storage rather than through our API. Following is POST /users/{id}/follow, returning 200 OK. The feed is GET /feed?cursor={c}, which returns a page of posts and a next_cursor. A cursor is better than page numbers here because new posts keep arriving at the top, and offset-based pages would shift and show duplicates.

Clients reach an API gateway for authentication and routing. Publishing starts with a request to the post service for an upload URL, after which the app pushes the bytes directly into object storage such as S3. The post record itself, holding the caption and a pointer to the media object, goes into a posts database; Cassandra suits this well because the workload is write-heavy, append-only and naturally partitioned by author. Viewers load media from a CDN that caches objects near them, so our servers never stream image bytes.

Each new post emits an event to Kafka. A fan-out service consumes those events and has workers push the new post ID onto the cached timeline of each follower in Redis. When a user opens the app, the feed service fetches the cached ID list for that person and hydrates them with post details. The guiding principle is that the expensive assembly happens once at write time, which turns the frequent feed read into a fast key lookup.

The client begins by requesting a presigned upload URL from the post service, so photo and video bytes never pass through application servers. Large videos use multipart upload, which splits the file into parts sent independently; if the network drops, only the failed part is retried instead of the whole file.

When an upload completes, S3 emits an event that triggers background workers. For images, they generate thumbnails and several resized versions so a phone on a small screen downloads a compact image, not a 12-megapixel original. For videos, workers transcode into multiple bitrates packaged for adaptive streaming such as HLS, letting the player step down to a lower quality on a weak connection instead of stalling.

Every processed file is immutable, which is ideal for caching. The CDN can store these objects with very long expiry times, and the overwhelming majority of media requests are answered at the edge without touching S3 at all. That keeps image loads fast worldwide and keeps origin bandwidth costs under control.

Fan-out on read assembles a feed at request time by querying recent posts from every followed account and merging them. Posting is cheap, but a user following 500 accounts triggers a large multi-partition query on every feed open, and at 58K reads per second that becomes the dominant cost.

Fan-out on write does the opposite: when someone posts, the system inserts the post ID into each follower's precomputed timeline, so reading is a single lookup. The weakness is extreme accounts. One post from an account with 200 million followers would require 200 million timeline writes, delaying delivery for everyone and wasting effort on followers who rarely log in.

The hybrid keeps fan-out on write for ordinary accounts and skips it for accounts above a threshold, for example around 1 million followers. When a user opens the feed, the service reads their precomputed timeline and also pulls recent posts from the handful of large accounts they follow, merging the two lists. Two refinements help further: users inactive for weeks are skipped during fan-out and have their timeline rebuilt on return, and each timeline is capped at a few hundred IDs.

Each user's Redis timeline is a capped list of roughly the latest 500 post IDs. Most sessions never scroll that far, and anyone who does falls back to querying the database. Post details live in a separate shared cache, so a viral post is stored once and reused by millions of timelines rather than copied into each.

Post IDs are time-sortable, in the style of Snowflake IDs, meaning a larger ID is always a newer post. That makes merging celebrity posts into a timeline a simple merge by ID, and it powers cursor pagination: the next page begins after the last ID the client saw, so no posts repeat or go missing even as new ones arrive. Finally, posts are partitioned in Cassandra by author and clustered by post ID, so fetching an account's latest posts, which the celebrity pull path relies on, is a single partition read.

The hybrid fan-out gives fast reads without write storms, at the price of maintaining two code paths and a threshold that needs tuning. Eventual consistency keeps the system available, though a new post may reach some followers a few seconds after others. Precomputed timelines make reads instant but consume Redis memory for every active user, which is why timelines are capped and inactive users are skipped.

Choosing a chronological feed keeps behaviour simple and predictable for now, and a ranking stage can later be inserted between fetching candidate post IDs and returning the page, without changing the storage or fan-out design. In short: push posts to most followers, pull from celebrities when reading, and let a CDN deliver every byte of media.