Design LeetCode: An Online Judge for Untrusted Code and Contests

Users need to browse the problem catalog, narrow it by topic or difficulty, and open a problem to read its statement and examples. They then write a solution in one of several supported languages, such as Java, Python or C++, and submit it.

The judge executes that submission against a hidden test suite and reports one of a small set of verdicts, such as Accepted, Wrong Answer, a timeout (TLE) or a crash during execution. Finally, timed contests let many users compete at once with a live ranking of every participant. Discussion forums, paid tiers and plagiarism detection are valuable features, but they sit beside the judging pipeline rather than inside it, so we leave them out.

Isolation is the top priority. A submission must never be able to damage the host, reach the network, read secrets or see another user's code. Every other requirement is negotiable before this one is.

After safety comes responsiveness. Someone who hits submit is staring at a spinner, so the verdict should arrive in under about 5 seconds. The system must also tolerate bursts: when a contest opens, traffic can multiply tenfold or more within a minute. And the contest ranking should be near real time, reflecting a newly accepted solution within a few seconds so participants trust what they see.

Assume about 1 million daily active users generating roughly 5 million submissions a day. Spread over 86,400 seconds, that averages to around 60 submissions per second, a load that any reasonably built backend handles comfortably without special tricks.

The average hides the real shape of the load. A weekly contest might draw around 100K participants, and submissions can briefly peak near 1,000 per second. If each run burns roughly 2 seconds of CPU across all its test cases, that peak needs about 2,000 cores busy at once. Code, verdicts and test files are small, so storage barely registers. The estimate points at a single conclusion: the code-running worker fleet is the bottleneck, and the architecture should make that fleet easy to grow and shrink.

The core entities are Problem (with its hidden test cases), Submission, Contest and Leaderboard. GET /problems/{id} delivers everything the editor needs to render a problem: its description, worked examples, and a starter stub per supported language.

Submitting is asynchronous by design. POST /problems/{id}/submissions takes the language, the source code and an optional contest ID, and immediately returns a submission ID rather than holding the connection open while code runs. The client then calls GET /submissions/{id} until the status moves from pending to a final verdict with runtime and memory figures. During contests, GET /contests/{id}/leaderboard returns the top of the ranking along with the caller's own position, so even someone in 40,000th place can see where they stand without paging through the whole list.

Requests pass through an API gateway responsible for authentication, throttling and routing. Behind it, the submission service validates the code, stores a submission row marked pending, returns the ID to the client, and then enqueues an execution job on Kafka or SQS. The service itself never executes user code.

A pool of code workers pulls jobs from that queue at its own pace. Each worker spins up an isolated container, runs the solution against every test case, and writes the verdict back to the database, where the polling client finds it. For contest submissions, the worker also bumps that contestant's entry in a per-contest Redis sorted set, and leaderboard reads go straight to Redis. Because the front-end services are stateless and the workers are fed by a queue, the worker pool can scale on queue length without anyone else noticing.

Running submissions directly on the application server is the simplest option and the most dangerous. A single malicious script could read environment variables, open outbound connections or crash the process that serves everyone else, and there is no clean way to cap its CPU or memory.

A dedicated virtual machine per run gives superb isolation, but VMs take time to boot and cost real money at a thousand runs per second. A hardened container strikes the balance: it launches in under a second and packs densely onto hosts. Its weakness is the shared kernel, so we reduce the attack surface aggressively. Containers get no network access, a read-only filesystem, an unprivileged user account, and a seccomp filter that rejects risky system calls.

Each run also gets hard ceilings on CPU seconds, memory, number of processes and bytes of output. Runaway loops and fork bombs are terminated before they can starve the host. We choose hardened containers, and for an extra layer we can run them under gVisor or inside Firecracker microVMs and always start from a fresh sandbox, so nothing leaks between submissions.

The queue is the first line of defense. When a thousand submissions land in one second, they simply wait a moment in line instead of overwhelming the workers. Workers are then autoscaled on queue depth: a growing backlog adds instances, and a draining one removes them, so we pay for peak capacity only during peaks.

Autoscaling reacts with some lag, but contests are on a published schedule, which removes the guesswork. About ten minutes before the start we pre-warm the fleet by launching extra workers and pulling every language image onto those machines, so the first submissions find hot capacity waiting.

Contest jobs also get a dedicated queue and worker pool. That way a surge of casual practice submissions cannot push a contestant's run behind thousands of others. Finally, every participant opens the same handful of problems in the same second, so problem pages are served from a CDN and the origin barely notices the stampede.

Recomputing a ranking with an SQL sort over 100K rows on every refresh would collapse under contest traffic. Instead, the moment a worker records an accepted contest solution, it computes a single composite score: points multiplied by one million, minus penalty seconds. Higher points always win, and among equal points the participant with less penalty time ranks higher, all expressed as one number.

That score is written with ZADD into a Redis sorted set dedicated to the contest, an update that costs O(log N). The first page of the leaderboard is ZREVRANGE from 0 to 49, which returns the top fifty in descending order. A user's own position comes from ZREVRANK, answered in milliseconds even with 100K entries.

Redis here is a fast, disposable view. Every verdict and score is also persisted in the database, which remains the source of truth, so if the Redis node is lost the sorted set can be rebuilt from stored results.

Clients poll for verdicts instead of receiving pushes. Polling once a second for a few seconds is simple and cheap; WebSockets would feel slightly snappier but require keeping many long-lived connections open. Hardened containers are faster and cheaper than microVMs, which in turn offer stronger isolation, so the choice depends on how much risk the platform tolerates.

The asynchronous queue absorbs spikes and decouples submission handling from execution, at the cost of a small added delay on every run. Keeping the leaderboard in memory makes ranking instant, but only because the database backs it for recovery. The shape of the whole design fits in one line: sandbox every submission, buffer execution behind a queue, and keep rankings in RAM.