Upstash Box sandbox benchmark
Our hardware probe found Upstash Box runs each sandbox as a Docker container. We timed it through one SDK against 8 other sandbox providers: cold start, real agent work, everyday calls, pause and snapshots, bursts, machine speed, reliability and cost per task. Each result is a median with its p95 and 95% interval, on two tracks: the same 4 vCPU, 8 GB machine for every provider, and each provider's default machine.
Sandbox bench runs on Sandbox SDK, which the same team (opencoredev) builds. We may work with some of these providers. No provider paid for these results or decided how they were tested.
Same machine: 4 vCPU, 8 GB
- Cold start, median
- 2.7 s 9th of 9
- Tool replay, median
- 27 s 7th of 9
Default machines
- Cold start, median
- 2.6 s 9th of 9
- Tool replay, median
- 27 s 7th of 9
Upstash Box on the same 4 vCPU, 8 GB machine
Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. Runs from 2026-10-22 to 2026-10-23, 9 providers. See every provider side by side on the full results.
- Asked for
- 4 vCPU, 8 GB ("medium")
- Measured inside
- 4 vCPU, 8 GB
- Cost per 1,000 tasks
- $0.34 (billed)
- Success rate
- 100.00% (1,075 of 1,075)
- Ran in: Ashburn, US.
- Cost: $0.00034 per task, from its bill (billed over 10 tasks, high confidence). A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.
- List price: $0.2 per hour for this machine, compute only.
- Pause and resume: supported; pause 0.84 s and resume 0.57 s median over 99 runs.
- Failures counted against Upstash Box: none.
Start-up
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Cold start (lower is better) | 2.7 sp95 11 s95%: 2.6 s to 2.7 s | 11 s | 2.6 s to 2.7 s | 9th of 9 · lowest median: Superserve 0.52 s |
Agent work
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Agent tool replay (lower is better) | 27 sp95 32 s95%: 26 s to 28 s | 32 s | 26 s to 28 s | 7th of 9 · lowest median: Daytona 12 s |
| npm install (lower is better) | 1.1 sp95 1.2 s95%: 1.1 s to 1.1 s | 1.2 s | 1.1 s to 1.1 s | 4th of 9 · lowest median: Daytona 0.75 s |
| pip install (lower is better) | 4.0 sp95 5.0 s95%: 4.0 s to 4.1 s | 5.0 s | 4.0 s to 4.1 s | 4th of 9 · lowest median: Tenki 3.3 s |
| git clone (lower is better) | 0.69 sp95 1.7 s95%: 0.67 s to 0.72 s | 1.7 s | 0.67 s to 0.72 s | 3rd of 9 · lowest median: Daytona 0.43 s |
| Dev server ready (lower is better) | 3.7 sp95 5.6 s95%: 3.7 s to 3.8 s | 5.6 s | 3.7 s to 3.8 s | 7th of 9 · lowest median: Superserve 1.3 s |
| Preview URL first byte (lower is better) | 0.31 sp95 0.73 s95%: 0.29 s to 0.34 s | 0.73 s | 0.29 s to 0.34 s | 5th of 9, tied with Upstash Ephemeral Box · lowest median: Superserve 0.18 s |
| Edit, rebuild, reload (lower is better) | 0.43 sp95 0.93 s95%: 0.42 s to 0.43 s | 0.93 s | 0.42 s to 0.43 s | 5th of 9 · lowest median: Freestyle 0.30 s |
Everyday calls
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Run a command (lower is better) | 0.33 sp95 0.41 s95%: 0.32 s to 0.33 s | 0.41 s | 0.32 s to 0.33 s | 8th of 9 · lowest median: E2B 0.13 s |
| Write + read 1 KB (lower is better) | 0.36 sp95 0.71 s95%: 0.35 s to 0.36 s | 0.71 s | 0.35 s to 0.36 s | 7th of 9 · lowest median: Superserve 0.10 s |
| Write + read 10 MB (lower is better) | 3.2 sp95 11 s95%: 3.1 s to 3.7 s | 11 s | 3.1 s to 3.7 s | 5th of 9 · lowest median: Superserve 0.66 s |
| Reconnect (lower is better) | 0.60 sp95 0.81 s95%: 0.59 s to 0.63 s | 0.81 s | 0.59 s to 0.63 s | 6th of 9 · lowest median: Daytona 0.30 s |
| Destroy (lower is better) | 1.0 sp95 12 s95%: 0.71 s to 1.7 s | 12 s | 0.71 s to 1.7 s | 9th of 9 · lowest median: E2B 0.11 s |
Pause and snapshots
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Pause + resume (lower is better) | 1.6 sp95 12 s95%: 1.3 s to 2.1 s | 12 s | 1.3 s to 2.1 s | 4th of 6, tied with E2B · lowest median: boxd 0.53 s |
| Take snapshot (lower is better) | 2.1 sp95 8.4 s95%: 2.1 s to 4.2 s | 8.4 s | 2.1 s to 4.2 s | 5th of 8 · lowest median: Superserve 0.39 s |
| Start from snapshot (lower is better) | 4.9 sp95 13 s95%: 4.8 s to 6.9 s | 13 s | 4.8 s to 6.9 s | 8th of 8 · lowest median: Superserve 0.55 s |
Bursts
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| 10 at once, all ready (lower is better) | 3.0 sp95 4.8 s95%: 2.7 s to 4.8 s | 4.8 s | 2.7 s to 4.8 s | 8th of 8, tied with Daytona · lowest median: Superserve 0.84 s |
| 25 at once, all ready (lower is better) | 5.4 sp95 5.7 s95%: 2.8 s to 5.7 s | 5.7 s | 2.8 s to 5.7 s | 8th of 9, tied with Boat, Daytona · lowest median: Superserve 0.79 s |
Machine speed
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| CPU, one core (higher is better) | 13.7M/sp95 13.8M/s95%: 13.7M/s to 13.7M/s | 13.8M/s | 13.7M/s to 13.7M/s | 5th of 9, tied with Upstash Ephemeral Box · highest median: Tenki 19.4M/s |
| CPU, all cores (higher is better) | 48.2M/sp95 50.5M/s95%: 47.8M/s to 49.1M/s | 50.5M/s | 47.8M/s to 49.1M/s | 3rd of 9 · highest median: Upstash Ephemeral Box 51M/s |
| Memory bandwidth (higher is better) | 33.8 GB/sp95 40.4 GB/s95%: 33.0 GB/s to 35.3 GB/s | 40.4 GB/s | 33.0 GB/s to 35.3 GB/s | 3rd of 9 · highest median: Tenki 45.9 GB/s |
| Disk write (higher is better) | 1.6 GB/sp95 2.2 GB/s95%: 1.2 GB/s to 1.8 GB/s | 2.2 GB/s | 1.2 GB/s to 1.8 GB/s | 2nd of 9, tied with Daytona · highest median: Upstash Ephemeral Box 2.4 GB/s |
| Disk read (higher is better) | 20.3 GB/sp95 22.8 GB/s95%: 19.9 GB/s to 20.7 GB/s | 22.8 GB/s | 19.9 GB/s to 20.7 GB/s | 2nd of 9 · highest median: Upstash Ephemeral Box 23.1 GB/s |
| 4K random write (higher is better) | 340.4 IOPSp95 547.4 IOPS95%: 304.7 IOPS to 397.6 IOPS | 547.4 IOPS | 304.7 IOPS to 397.6 IOPS | 9th of 9 · highest median: Upstash Ephemeral Box 45.1K IOPS |
| 4K random read (higher is better) | 501.9K IOPSp95 527.2K IOPS95%: 498.1K IOPS to 505.3K IOPS | 527.2K IOPS | 498.1K IOPS to 505.3K IOPS | 4th of 9 · highest median: Boat 796.4K IOPS |
Quick picks on this track, all providers
| Need | Pick | Runner-up |
|---|---|---|
| Fastest cold start | Superserve 0.52 s | Upstash Ephemeral Box 0.68 s |
| Fastest agent work (tool replay) | Daytona 12 s | Superserve 13 s |
| Fastest 25-sandbox fan-out | Superserve 0.79 s | E2B 1.1 s |
| Fastest reconnect | Daytona 0.30 s | Superserve 0.33 s |
| Cheapest task | Upstash Ephemeral Box $0.00026 per task, $0.26 per 1,000 (billed) | Upstash Box $0.00034 per task, $0.34 per 1,000 (billed) |
| Most reliable | E2B, Tenki, Superserve, Upstash Box, Daytona, Upstash Ephemeral Box, Boat 100.00% (tied) | Freestyle 99.91% |
Upstash Box on its default machine
Each provider's default sandbox, so machine sizes differ. Runs from 2026-10-22 to 2026-10-23, 9 providers. See every provider side by side on the full results.
- Asked for
- 2 vCPU, 4 GB
- Measured inside
- 2 vCPU, 4 GB
- Cost per 1,000 tasks
- $0.17 (billed)
- Success rate
- 100.00% (1,076 of 1,076)
- Ran in: Ashburn, US.
- Cost: $0.00017 per task, from its bill (billed over 10 tasks, high confidence). A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.
- List price: $0.1 per hour for this machine, compute only.
- Pause and resume: supported; pause 0.73 s and resume 0.59 s median over 100 runs.
- Failures counted against Upstash Box: none.
Start-up
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Cold start (lower is better) | 2.6 sp95 13 s95%: 2.6 s to 2.7 s | 13 s | 2.6 s to 2.7 s | 9th of 9 · lowest median: E2B 0.50 s |
Agent work
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Agent tool replay (lower is better) | 27 sp95 31 s95%: 26 s to 28 s | 31 s | 26 s to 28 s | 7th of 9 · lowest median: Superserve 13 s |
| npm install (lower is better) | 1.1 sp95 1.2 s95%: 1.1 s to 1.1 s | 1.2 s | 1.1 s to 1.1 s | 3rd of 9 · lowest median: Boat 0.86 s |
| pip install (lower is better) | 4.1 sp95 5.0 s95%: 4.0 s to 4.4 s | 5.0 s | 4.0 s to 4.4 s | 3rd of 9, tied with Tenki, Upstash Ephemeral Box · lowest median: Superserve 3.8 s |
| git clone (lower is better) | 0.70 sp95 1.3 s95%: 0.68 s to 0.77 s | 1.3 s | 0.68 s to 0.77 s | 3rd of 9 · lowest median: Upstash Ephemeral Box 0.61 s |
| Dev server ready (lower is better) | 3.8 sp95 4.8 s95%: 3.7 s to 3.8 s | 4.8 s | 3.7 s to 3.8 s | 7th of 9 · lowest median: Superserve 1.8 s |
| Preview URL first byte (lower is better) | 0.30 sp95 0.43 s95%: 0.29 s to 0.31 s | 0.43 s | 0.29 s to 0.31 s | 5th of 9, tied with Upstash Ephemeral Box · lowest median: Superserve 0.19 s |
| Edit, rebuild, reload (lower is better) | 0.43 sp95 1.1 s95%: 0.42 s to 0.44 s | 1.1 s | 0.42 s to 0.44 s | 5th of 9 · lowest median: Freestyle 0.31 s |
Everyday calls
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Run a command (lower is better) | 0.32 sp95 0.41 s95%: 0.32 s to 0.33 s | 0.41 s | 0.32 s to 0.33 s | 8th of 9 · lowest median: E2B 0.14 s |
| Write + read 1 KB (lower is better) | 0.36 sp95 1.1 s95%: 0.35 s to 0.36 s | 1.1 s | 0.35 s to 0.36 s | 7th of 9 · lowest median: Superserve 0.10 s |
| Write + read 10 MB (lower is better) | 2.9 sp95 7.5 s95%: 2.8 s to 3.1 s | 7.5 s | 2.8 s to 3.1 s | 6th of 9, tied with Freestyle · lowest median: Superserve 0.64 s |
| Reconnect (lower is better) | 0.60 sp95 0.76 s95%: 0.59 s to 0.64 s | 0.76 s | 0.59 s to 0.64 s | 6th of 9 · lowest median: Daytona 0.31 s |
| Destroy (lower is better) | 0.96 sp95 7.8 s95%: 0.80 s to 1.2 s | 7.8 s | 0.80 s to 1.2 s | 9th of 9, tied with Daytona · lowest median: E2B 0.11 s |
Pause and snapshots
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Pause + resume (lower is better) | 1.7 sp95 13 s95%: 1.3 s to 2.8 s | 13 s | 1.3 s to 2.8 s | 5th of 6 · lowest median: boxd 0.54 s |
| Take snapshot (lower is better) | 2.1 sp95 17 s95%: 2.1 s to 4.2 s | 17 s | 2.1 s to 4.2 s | 5th of 8 · lowest median: E2B 0.34 s |
| Start from snapshot (lower is better) | 4.8 sp95 15 s95%: 2.8 s to 4.8 s | 15 s | 2.8 s to 4.8 s | 8th of 8 · lowest median: Superserve 0.56 s |
Bursts
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| 10 at once, all ready (lower is better) | 5.0 sp95 6.8 s95%: 2.8 s to 6.8 s | 6.8 s | 2.8 s to 6.8 s | 9th of 9 · lowest median: Superserve 0.65 s |
| 25 at once, all ready (lower is better) | 15 sp95 17 s95%: 9.0 s to 17 s | 17 s | 9.0 s to 17 s | 8th of 8 · lowest median: Superserve 0.75 s |
Machine speed
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| CPU, one core (higher is better) | 13.7M/sp95 13.8M/s95%: 13.7M/s to 13.7M/s | 13.8M/s | 13.7M/s to 13.7M/s | 4th of 9 · highest median: Boat 18.8M/s |
| CPU, all cores (higher is better) | 25M/sp95 25.8M/s95%: 24.6M/s to 25.4M/s | 25.8M/s | 24.6M/s to 25.4M/s | 4th of 9 · highest median: Boat 51.7M/s |
| Memory bandwidth (higher is better) | 35.3 GB/sp95 40.1 GB/s95%: 33.9 GB/s to 36.2 GB/s | 40.1 GB/s | 33.9 GB/s to 36.2 GB/s | 3rd of 9 · highest median: Upstash Ephemeral Box 41.1 GB/s |
| Disk write (higher is better) | 1.7 GB/sp95 2.2 GB/s95%: 1.5 GB/s to 1.9 GB/s | 2.2 GB/s | 1.5 GB/s to 1.9 GB/s | 2nd of 9 · highest median: Upstash Ephemeral Box 2.4 GB/s |
| Disk read (higher is better) | 20.8 GB/sp95 22.8 GB/s95%: 20.4 GB/s to 21.1 GB/s | 22.8 GB/s | 20.4 GB/s to 21.1 GB/s | 2nd of 9, tied with Boat · highest median: Upstash Ephemeral Box 23.0 GB/s |
| 4K random write (higher is better) | 439.1 IOPSp95 556.2 IOPS95%: 379.4 IOPS to 480.7 IOPS | 556.2 IOPS | 379.4 IOPS to 480.7 IOPS | 8th of 9 · highest median: Upstash Ephemeral Box 43.3K IOPS |
| 4K random read (higher is better) | 505.2K IOPSp95 527.3K IOPS95%: 501.6K IOPS to 509.1K IOPS | 527.3K IOPS | 501.6K IOPS to 509.1K IOPS | 4th of 9 · highest median: Boat 772.6K IOPS |
Quick picks on this track, all providers
| Need | Pick | Runner-up |
|---|---|---|
| Fastest cold start | E2B 0.50 s | Superserve 0.53 s |
| Fastest agent work (tool replay) | Superserve 13 s | Daytona 15 s |
| Fastest 25-sandbox fan-out | Superserve 0.75 s | E2B 0.87 s |
| Fastest reconnect | Daytona 0.31 s | Superserve 0.32 s |
| Cheapest task | Upstash Ephemeral Box $0.00013 per task, $0.13 per 1,000 (billed) | Upstash Box $0.00017 per task, $0.17 per 1,000 (billed) |
| Most reliable | E2B, Tenki, Superserve, Upstash Box, Daytona, Boat 100.00% (tied) | boxd 99.81% |
How to read these numbers
- Upstash appears twice because Box (long-lived, can pause) and Ephemeral Box (short-lived, for one-off agent jobs) are separate products.
- Ephemeral Box has no pause, so it has no pause result.
- Every call starts from one server in Florida, US. Providers hosted farther away pay that round trip on every call; Boat (France) and boxd (Germany) pay a transatlantic one.
- There are two tracks. Same machine: 4 vCPU, 8 GB: Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. A provider marked “machine differs” measured something else inside. Default machines: Each provider's default sandbox, so machine sizes differ.
- Cost per task has a kind. Billed comes from the provider's bill, metered from its usage records at published rates, and estimate from our measured time at list price. Kinds are not directly comparable. Prices cover compute only at list rates: disk, transfer, subscriptions and credits are excluded.
- Pause and resume times come from the previous SDK run, because the newest runs skipped them. A burst size is published only when all three bursts finished.
- Failures caused by our own code are set aside, not counted. Every provider-side failure is kept.
- Ranks compare medians. Medians inside each other's 95% intervals count as tied, so a close rank can be noise.
More results
- Upstash Box vs E2B, side by side
- Upstash Box vs Daytona, side by side
- E2B sandbox benchmark
- Daytona sandbox benchmark
- Superserve sandbox benchmark
- Upstash Ephemeral Box sandbox benchmark
- Boat sandbox benchmark
- Freestyle sandbox benchmark
- Tenki sandbox benchmark
- Every sandbox provider we benchmark
- Every provider on the same 4 vCPU, 8 GB machine
- Every provider on its default machine
- Raw data: results.json · results.md · llms.txt