Design YouTube: Video Upload, Transcoding and Global Streaming

Creators need to upload videos that may be many gigabytes, frequently over unreliable connections, so the upload path has to tolerate interruptions. Viewers need to watch any published video smoothly on any device, which means the same content must work on a small phone screen and a 4K television.

That leads directly to the third requirement: every video should exist in several qualities, from 240p up to 4K, so a player can pick what the screen and network can handle. Each video also gets a page with its title, description, view count and processing status. Recommendations, comments, live streaming and advertising are each large systems in their own right and are deliberately left for another discussion.

Playback quality drives most decisions. A target of under two seconds to first frame, with rare rebuffering, is what keeps viewers from leaving. Durability matters just as much on the creator side: once an upload is confirmed, losing it is unacceptable, so originals belong in highly durable object storage with replication.

Performance must be global. Someone in a small town far from any data centre should get the same smooth experience as someone next to one, which hints strongly at edge caching. In contrast, processing is allowed to be eventual. A creator can wait a few minutes after upload before the video goes live, and that relaxation is what lets the transcoding pipeline run as asynchronous background work rather than on the request path.

Assume about 5 billion views per day, which works out to roughly 58,000 new views starting every second. Assume creators upload about 500 hours of footage per minute, or around 720,000 hours a day. If one hour stored in every quality takes roughly 2 GB, new storage grows by about 1.4 PB per day. That is large but manageable with object storage and tiering.

Bandwidth is the figure that dwarfs the rest. If the average view lasts about five minutes, roughly 17 million people are watching at any instant. At around 3 Mbps each, outbound traffic reaches roughly 50 Tbps. No handful of origin data centres can serve that, so the estimate makes the central decision for us: video bytes must flow from a CDN close to viewers, with our own servers handling only metadata and misses.

The main entities are the video, an upload session that tracks a transfer in progress, renditions (each one a single quality of a video), and the channel that owns it. Separating the upload session from the video lets an interrupted transfer be resumed without creating duplicate videos.

Uploading is a three-step conversation. POST /uploads with the title, size and content type returns an upload ID and a chunk size. The client then sends pieces with PUT /uploads/{id}/chunks/{n}, and each response reports which chunks the server holds, so the client always knows its progress. POST /uploads/{id}/complete finalises the transfer and returns a video ID in the processing state. For playback, GET /videos/{id} returns metadata plus a manifest URL, and the player takes over from there by fetching segments directly from the CDN.

On the upload side, a creator sends chunks to an upload service, which records a new video in the metadata database and writes the original into raw object storage such as S3. Often the service only hands out pre-signed URLs and the bytes flow straight to storage, keeping heavy traffic off application servers.

Once the upload completes, an event lands on a queue and a pool of transcoding workers converts the original into many qualities. Their output, short video segments plus playlists, goes to a separate video storage bucket, and the database flips the status to ready. This asynchronous hand-off is why a busy encoding backlog never blocks new uploads.

On the viewing side, the player calls the video service for page details and the manifest URL, then streams segments from the nearest CDN edge. The edge only reaches back to our storage when it does not already hold a segment. The result is two independent paths that scale separately: a write-heavy pipeline and a read-heavy delivery network.

Picture a 5 GB file going up over mobile data and the connection dying at 90 percent. Restarting from zero would waste time and data and drive creators away. The fix is to treat an upload as a session made of independent pieces.

The client opens a session and receives an ID and a chunk size, then slices the file into parts of roughly 8 MB and sends a few at a time in parallel to use the available bandwidth. The server acknowledges each part and stores its number and checksum. After a drop, the client simply asks which parts already exist and sends only the gaps. When every part is present, the server checks the checksums, stitches the file together and kicks off processing.

A common practical implementation is S3 multipart upload with pre-signed URLs. The client uploads parts directly to object storage, our servers only coordinate the session, and the expensive byte traffic never passes through the application tier.

Transcoding a long film on one machine would take hours, so the original is first cut into pieces a few seconds long. Hundreds of workers can then encode those pieces at once, turning a two-hour feature around in minutes. Each piece is encoded at every rung of a quality ladder from 240p to 4K. H.264 offers near-universal device support, while newer codecs such as VP9 or AV1 compress better and are worth the extra encoding cost for videos that will be watched heavily.

The encoded output is packaged as HLS and DASH: each quality becomes a series of small segment files, and a manifest lists every quality and every segment. That packaging enables adaptive bitrate streaming. The player measures throughput as it downloads and selects the quality of the next segment accordingly. When a commuter's train enters a tunnel, playback steps down a rung instead of freezing, then climbs back up when the signal returns.

The whole process runs as a queue-driven pipeline of jobs with retries, so a crashed worker only costs one piece of work. Side tasks such as thumbnails, captions and content safety checks run alongside the encoding rather than after it.

Serving all traffic from origin data centres is simple but fails on both distance and capacity; roughly 50 Tbps cannot leave a few buildings, and far-away viewers would see slow starts. Pushing every video to every edge fixes distance but wastes enormous storage, since most uploads are watched rarely.

The winning approach is a pull-based CDN, plus pre-warming for big launches. Edge servers keep whatever local viewers actually request, and when a large creator publishes, the new video is pushed to edges in advance to absorb the expected rush. This works because popularity is extremely skewed: a small fraction of videos accounts for most views, so edge hit rates stay very high while the long tail lives at origin. Cache layers stack from edge to regional cache to origin, and large platforms may place cache servers inside ISP networks. Private videos use signed manifest URLs that expire, so links cannot be shared indefinitely.

Storing every rendition costs space, but it means each video is transcoded once instead of on every view, which is far cheaper at billions of plays. Codec choice is a similar bargain: AV1 trims bandwidth noticeably but is slow to encode, so it makes sense only where view counts justify the compute.

Segment length balances agility against efficiency. Short segments let the player switch qualities quickly, while longer ones reduce request overhead and compress slightly better. Edge coverage trades cost for start time; more edge capacity speeds playback but the rarely watched long tail still stays at origin. Taken together, the design uploads in resumable chunks, encodes pieces in parallel, and lets the edge carry the streaming load.