Design Dropbox: Cloud File Storage and Multi-Device Sync

Four features define the product. Users can upload files from any device, and those files can be very large, so assume a ceiling of 50 GB per file. Users can download anything they have stored from any device they sign in on. Changes propagate on their own: edit a spreadsheet on the desktop and the phone receives the new version with no manual refresh. Finally, users can share a file or folder, either by link or by inviting specific people.

Real-time collaborative editing, where two cursors type into one document together, is explicitly left out. That is an operational-transform or CRDT problem with very different constraints, and mixing it in would blur the storage and sync story that this question is really testing.

Durability ranks above everything else. A user will forgive a slow upload, but not a missing tax return, so once the system acknowledges a file it must survive disk failures, machine failures and even the loss of a data centre. Object stores with multi-zone replication exist precisely for this guarantee.

Availability comes next, and we choose it over strict consistency. If a phone briefly shows the version from a few seconds ago, nobody is harmed; if the service refuses to open files during a network partition, everybody is. Two further requirements shape the client: big uploads must be resumable, picking up from where they stopped instead of from byte zero, and sync must be quick enough that changes appear elsewhere within seconds rather than minutes.

Assume 500 million registered users, of whom 100 million are active on a given day. If each active user creates or modifies two files daily, that is 200 million uploads per day, or roughly 2,300 per second averaged over the day. With a typical file of about 1 MB, the system ingests around 200 TB of fresh data every day.

The total footprint is the eye-opening figure. If the average account holds about 10 GB, then 500 million accounts amount to roughly 5 EB. No relational database could hold that, and no fleet of application servers should be streaming it. Meanwhile, the metadata describing those files, such as names, paths, sizes, owners and chunk lists, is tiny by comparison. That contrast is the most important output of the estimate: keep file metadata in a database and file contents in object storage.

Four entities cover the domain: the file itself, its metadata record, the chunks its bytes are split into, and the user who owns it. Treating chunks as a first-class entity is what later enables resumable uploads and deduplication.

Uploading starts with POST /files, sending the name, the size and an array of chunk hashes. The server answers with a file_id and a set of upload URLs, one for each chunk it still needs. Downloading uses GET /files/{file_id}, which returns metadata plus a download URL instead of streaming bytes itself. Syncing relies on GET /changes?cursor={c}, which lists every change after the given position along with a fresh cursor the device stores for its next call. Note that none of these endpoints carry file contents; they only coordinate.

Desktop and mobile clients talk to an API gateway that authenticates requests and enforces rate limits. Behind it, the file service owns metadata: file names, sizes, owners, versions and ordered chunk lists, kept in a relational store such as Postgres where folder hierarchies and sharing permissions are easy to express with joins and constraints.

File bytes take a different route entirely. The client uploads chunks directly to object storage such as S3 using short-lived presigned URLs handed out by the file service, and downloads are served through a CDN. Our application servers therefore stay small and cheap, since they never touch the hundreds of terabytes moving each day.

Every metadata change is also appended to a per-user change log, for example a Kafka topic partitioned by user so events for one account stay in order. A sync service consumes those events and nudges the user's other connected devices with a small notification. The guiding split is easy to remember: coordination flows through our services, heavy content flows straight to storage.

A single 2 GB upload over a home connection will almost certainly be interrupted at some point, so the client breaks the file into fixed pieces of about 4 MB each, turning that video into roughly 500 chunks. It then computes a SHA-256 digest of each piece. The digest doubles as the chunk identifier, which means any two identical pieces of data share the same name everywhere in the system.

Next, the client registers the file by sending its list of digests. The file service creates a record with an uploading status and returns a presigned URL only for chunks that storage does not already hold. A presigned URL is a time-limited permission to write exactly one object, so the client can upload without our servers proxying the data. Several chunks go up in parallel to use the available bandwidth.

If the connection breaks at 90 percent, the client simply asks the service which chunks have arrived and sends the remainder; nothing already stored is repeated. When every chunk is confirmed, the file service flips the record to ready and emits a change event so other devices learn about the new file.

The naive way to learn about remote changes is for every device to poll on a timer. A long interval makes sync feel sluggish, while a short one multiplies requests across 100 million users into a constant storm of empty responses. Neither setting is acceptable.

The better approach keeps one lightweight persistent connection per device, using a WebSocket or a long poll. When the change log records an update, the sync service sends only a tiny signal. The device then calls the changes endpoint with its stored cursor, which acts like a bookmark into that user's log. Whether the laptop was offline for ten seconds or ten days, it receives exactly the events it missed and nothing more. The cost is holding many open connections, which a dedicated, horizontally scaled sync tier can absorb.

Applying a change is efficient too. The device compares the new chunk list with what it already has and fetches only the differing chunks, so editing one page of a large document moves a few megabytes, not the whole file. If two devices edited the same file while offline, the system does not attempt a clever merge: the first version to commit wins and the other is preserved as a conflicted copy, so no edits disappear.

Because each chunk is named by the hash of its own contents, deduplication comes almost for free. Identical chunks produce identical identifiers, so storage keeps a single copy. The upload flow already checks digests before sending data, which means copying a large file into another folder, or re-uploading something already stored, completes nearly instantly because no bytes need to travel.

Shared chunks complicate deletion, so each chunk carries a reference count of the file versions that point at it, and garbage collection removes a chunk only when that count reaches zero. Chunks are also compressed, and data that nobody has opened in months can move to colder, cheaper storage tiers. One caution: deduplicating across different accounts saves the most space but can leak information, since an instant upload reveals that someone else already holds that exact file. Scoping deduplication to a single account avoids that side channel.

Chunk size is a balancing act. Smaller chunks improve deduplication and make resumes cheaper, while larger chunks reduce request counts and the size of each file's chunk list. Favouring availability keeps files reachable at all times, at the cost of a device occasionally showing a slightly older version for a few seconds.

Sending uploads directly to S3 keeps application servers lean, but the file service must now track the status of each chunk and make sure presigned URLs expire. Saving conflicted copies rather than merging guarantees no work is lost, though the user has to reconcile the two versions manually. Taken together, the design rests on three habits: metadata lives in a database, bytes live in object storage, and only the parts that changed ever move across the network.