Upstash Ephemeral Box sandbox benchmark
Our hardware probe found Upstash Ephemeral Box runs each sandbox as a Container. We timed it through one SDK against 8 other sandbox providers: cold start, real agent work, everyday calls, pause and snapshots, bursts, machine speed, reliability and cost per task. Each result is a median with its p95 and 95% interval, on two tracks: the same 4 vCPU, 8 GB machine for every provider, and each provider's default machine.
Sandbox bench runs on Sandbox SDK, which the same team (opencoredev) builds. We may work with some of these providers. No provider paid for these results or decided how they were tested.
Same machine: 4 vCPU, 8 GB
- Cold start, median
- 0.68 s 2nd of 9
- Tool replay, median
- 22 s 5th of 9
Default machines
- Cold start, median
- 0.68 s 3rd of 9
- Tool replay, median
- 23 s 5th of 9
Upstash Ephemeral Box on the same 4 vCPU, 8 GB machine
Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. Runs from 2026-10-22 to 2026-10-23, 9 providers. See every provider side by side on the full results.
- Asked for
- 4 vCPU, 8 GB ("medium")
- Measured inside
- 4 vCPU, 8 GB
- Cost per 1,000 tasks
- $0.26 (billed)
- Success rate
- 100.00% (976 of 976)
- Ran in: Ashburn, US.
- Cost: $0.00026 per task, from its bill (billed over 10 tasks, high confidence). A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.
- List price: $0.2 per hour for this machine, compute only.
- Pause and resume: not supported.
- Failures counted against Upstash Ephemeral Box: none.
Start-up
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Cold start (lower is better) | 0.68 sp95 0.79 s95%: 0.68 s to 0.69 s | 0.79 s | 0.68 s to 0.69 s | 2nd of 9 · lowest median: Superserve 0.52 s |
Agent work
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Agent tool replay (lower is better) | 22 sp95 23 s95%: 22 s to 23 s | 23 s | 22 s to 23 s | 5th of 9 · lowest median: Daytona 12 s |
| npm install (lower is better) | 1.00 sp95 1.0 s95%: 0.99 s to 1.0 s | 1.0 s | 0.99 s to 1.0 s | 3rd of 9 · lowest median: Daytona 0.75 s |
| pip install (lower is better) | 5.6 sp95 6.4 s95%: 5.2 s to 5.9 s | 6.4 s | 5.2 s to 5.9 s | 8th of 9 · lowest median: Tenki 3.3 s |
| git clone (lower is better) | 0.61 sp95 0.67 s95%: 0.60 s to 0.63 s | 0.67 s | 0.60 s to 0.63 s | 2nd of 9 · lowest median: Daytona 0.43 s |
| Dev server ready (lower is better) | 3.1 sp95 3.3 s95%: 3.1 s to 3.2 s | 3.3 s | 3.1 s to 3.2 s | 6th of 9 · lowest median: Superserve 1.3 s |
| Preview URL first byte (lower is better) | 0.29 sp95 0.41 s95%: 0.28 s to 0.31 s | 0.41 s | 0.28 s to 0.31 s | 4th of 9, tied with Upstash Box · lowest median: Superserve 0.18 s |
| Edit, rebuild, reload (lower is better) | 0.35 sp95 0.38 s95%: 0.35 s to 0.36 s | 0.38 s | 0.35 s to 0.36 s | 4th of 9 · lowest median: Freestyle 0.30 s |
Everyday calls
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Run a command (lower is better) | 0.31 sp95 0.38 s95%: 0.30 s to 0.31 s | 0.38 s | 0.30 s to 0.31 s | 7th of 9 · lowest median: E2B 0.13 s |
| Write + read 1 KB (lower is better) | 0.29 sp95 0.32 s95%: 0.29 s to 0.29 s | 0.32 s | 0.29 s to 0.29 s | 6th of 9 · lowest median: Superserve 0.10 s |
| Write + read 10 MB (lower is better) | 2.5 sp95 3.6 s95%: 2.5 s to 2.6 s | 3.6 s | 2.5 s to 2.6 s | 4th of 9 · lowest median: Superserve 0.66 s |
| Reconnect (lower is better) | 0.52 sp95 0.60 s95%: 0.52 s to 0.53 s | 0.60 s | 0.52 s to 0.53 s | 5th of 9 · lowest median: Daytona 0.30 s |
| Destroy (lower is better) | 0.27 sp95 0.41 s95%: 0.26 s to 0.28 s | 0.41 s | 0.26 s to 0.28 s | 7th of 9 · lowest median: E2B 0.11 s |
Pause and snapshots
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Pause + resume (lower is better) | n/a | Upstash Ephemeral Box has no pause and resume | ||
| Take snapshot (lower is better) | 2.1 sp95 2.1 s95%: 2.1 s to 2.1 s | 2.1 s | 2.1 s to 2.1 s | 4th of 8 · lowest median: Superserve 0.39 s |
| Start from snapshot (lower is better) | 1.1 sp95 1.2 s95%: 1.1 s to 1.1 s | 1.2 s | 1.1 s to 1.1 s | 4th of 8, tied with E2B · lowest median: Superserve 0.55 s |
Bursts
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| 10 at once, all ready (lower is better) | 0.85 sp95 1.1 s95%: 0.75 s to 1.1 s | 1.1 s | 0.75 s to 1.1 s | 2nd of 8, tied with Superserve · lowest median: Superserve 0.84 s |
| 25 at once, all ready (lower is better) | 1.1 sp95 1.1 s95%: 0.99 s to 1.1 s | 1.1 s | 0.99 s to 1.1 s | 3rd of 9, tied with E2B · lowest median: Superserve 0.79 s |
Machine speed
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| CPU, one core (higher is better) | 13.7M/sp95 13.8M/s95%: 13.7M/s to 13.7M/s | 13.8M/s | 13.7M/s to 13.7M/s | 4th of 9, tied with Upstash Box · highest median: Tenki 19.4M/s |
| CPU, all cores (higher is better) | 51M/sp95 52M/s95%: 50.8M/s to 51.3M/s | 52M/s | 50.8M/s to 51.3M/s | 1st of 9 |
| Memory bandwidth (higher is better) | 42.0 GB/sp95 44.1 GB/s95%: 41.8 GB/s to 43.4 GB/s | 44.1 GB/s | 41.8 GB/s to 43.4 GB/s | 2nd of 9 · highest median: Tenki 45.9 GB/s |
| Disk write (higher is better) | 2.4 GB/sp95 2.9 GB/s95%: 2.4 GB/s to 2.4 GB/s | 2.9 GB/s | 2.4 GB/s to 2.4 GB/s | 1st of 9 |
| Disk read (higher is better) | 23.1 GB/sp95 23.9 GB/s95%: 22.9 GB/s to 23.2 GB/s | 23.9 GB/s | 22.9 GB/s to 23.2 GB/s | 1st of 9 |
| 4K random write (higher is better) | 45.1K IOPSp95 46.9K IOPS95%: 44.9K IOPS to 45.4K IOPS | 46.9K IOPS | 44.9K IOPS to 45.4K IOPS | 1st of 9 |
| 4K random read (higher is better) | 546.1K IOPSp95 581.6K IOPS95%: 540K IOPS to 560.5K IOPS | 581.6K IOPS | 540K IOPS to 560.5K IOPS | 3rd of 9 · highest median: Boat 796.4K IOPS |
Quick picks on this track, all providers
| Need | Pick | Runner-up |
|---|---|---|
| Fastest cold start | Superserve 0.52 s | Upstash Ephemeral Box 0.68 s |
| Fastest agent work (tool replay) | Daytona 12 s | Superserve 13 s |
| Fastest 25-sandbox fan-out | Superserve 0.79 s | E2B 1.1 s |
| Fastest reconnect | Daytona 0.30 s | Superserve 0.33 s |
| Cheapest task | Upstash Ephemeral Box $0.00026 per task, $0.26 per 1,000 (billed) | Upstash Box $0.00034 per task, $0.34 per 1,000 (billed) |
| Most reliable | E2B, Tenki, Superserve, Upstash Box, Daytona, Upstash Ephemeral Box, Boat 100.00% (tied) | Freestyle 99.91% |
Upstash Ephemeral Box on its default machine
Each provider's default sandbox, so machine sizes differ. Runs from 2026-10-22 to 2026-10-23, 9 providers. See every provider side by side on the full results.
- Asked for
- 2 vCPU, 4 GB
- Measured inside
- 2 vCPU, 4 GB
- Cost per 1,000 tasks
- $0.13 (billed)
- Success rate
- 99.07% (957 of 966)
- Ran in: Ashburn, US.
- Cost: $0.00013 per task, from its bill (billed over 10 tasks, high confidence). A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.
- Pause and resume: not supported.
- Failures counted against Upstash Ephemeral Box: Upstash Box exec failed (provider). (provider) (12 tests); Upstash Box files.write failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=files.write) (4 tests); Upstash Box connect failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=connect) (1 test)
Start-up
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Cold start (lower is better) | 0.68 sp95 0.81 s95%: 0.68 s to 0.69 s | 0.81 s | 0.68 s to 0.69 s | 3rd of 9 · lowest median: E2B 0.50 s |
Agent work
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Agent tool replay (lower is better) | 23 sp95 24 s95%: 23 s to 23 s | 24 s | 23 s to 23 s | 5th of 9 · lowest median: Superserve 13 s |
| npm install (lower is better) | 1.0 sp95 1.1 s95%: 1.0 s to 1.0 s | 1.1 s | 1.0 s to 1.0 s | 2nd of 9 · lowest median: Boat 0.86 s |
| pip install (lower is better) | 4.3 sp95 5.4 s95%: 4.1 s to 4.6 s | 5.4 s | 4.1 s to 4.6 s | 5th of 9, tied with Upstash Box, Tenki · lowest median: Superserve 3.8 s |
| git clone (lower is better) | 0.61 sp95 0.67 s95%: 0.61 s to 0.64 s | 0.67 s | 0.61 s to 0.64 s | 1st of 9, tied with Daytona |
| Dev server ready (lower is better) | 3.2 sp95 3.5 s95%: 3.2 s to 3.2 s | 3.5 s | 3.2 s to 3.2 s | 6th of 9 · lowest median: Superserve 1.8 s |
| Preview URL first byte (lower is better) | 0.29 sp95 0.39 s95%: 0.28 s to 0.30 s | 0.39 s | 0.28 s to 0.30 s | 4th of 9, tied with Upstash Box · lowest median: Superserve 0.19 s |
| Edit, rebuild, reload (lower is better) | 0.36 sp95 0.40 s95%: 0.36 s to 0.37 s | 0.40 s | 0.36 s to 0.37 s | 4th of 9 · lowest median: Freestyle 0.31 s |
Everyday calls
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Run a command (lower is better) | 0.31 sp95 0.39 s95%: 0.31 s to 0.31 s | 0.39 s | 0.31 s to 0.31 s | 7th of 9 · lowest median: E2B 0.14 s |
| Write + read 1 KB (lower is better) | 0.30 sp95 0.33 s95%: 0.29 s to 0.30 s | 0.33 s | 0.29 s to 0.30 s | 6th of 9 · lowest median: Superserve 0.10 s |
| Write + read 10 MB (lower is better) | 2.6 sp95 11 s95%: 2.6 s to 2.7 s | 11 s | 2.6 s to 2.7 s | 4th of 9 · lowest median: Superserve 0.64 s |
| Reconnect (lower is better) | 0.53 sp95 0.60 s95%: 0.53 s to 0.53 s | 0.60 s | 0.53 s to 0.53 s | 5th of 9 · lowest median: Daytona 0.31 s |
| Destroy (lower is better) | 0.28 sp95 0.42 s95%: 0.27 s to 0.28 s | 0.42 s | 0.27 s to 0.28 s | 7th of 9 · lowest median: E2B 0.11 s |
Pause and snapshots
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Pause + resume (lower is better) | n/a | Upstash Ephemeral Box has no pause and resume | ||
| Take snapshot (lower is better) | 2.1 sp95 2.1 s95%: 2.1 s to 2.1 s | 2.1 s | 2.1 s to 2.1 s | 4th of 8 · lowest median: E2B 0.34 s |
| Start from snapshot (lower is better) | 1.1 sp95 1.3 s95%: 0.93 s to 1.1 s | 1.3 s | 0.93 s to 1.1 s | 4th of 8 · lowest median: Superserve 0.56 s |
Bursts
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| 10 at once, all ready (lower is better) | 0.75 sp95 1.1 s95%: 0.74 s to 1.1 s | 1.1 s | 0.74 s to 1.1 s | 2nd of 9, tied with E2B · lowest median: Superserve 0.65 s |
| 25 at once, all ready (lower is better) | 1.0 sp95 1.1 s95%: 0.97 s to 1.1 s | 1.1 s | 0.97 s to 1.1 s | 3rd of 8 · lowest median: Superserve 0.75 s |
Machine speed
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| CPU, one core (higher is better) | 13.7M/sp95 13.8M/s95%: 13.7M/s to 13.7M/s | 13.8M/s | 13.7M/s to 13.7M/s | 5th of 9 · highest median: Boat 18.8M/s |
| CPU, all cores (higher is better) | 26M/sp95 26.5M/s95%: 26M/s to 26.1M/s | 26.5M/s | 26M/s to 26.1M/s | 3rd of 9, tied with Freestyle · highest median: Boat 51.7M/s |
| Memory bandwidth (higher is better) | 41.1 GB/sp95 44.1 GB/s95%: 40.5 GB/s to 42.7 GB/s | 44.1 GB/s | 40.5 GB/s to 42.7 GB/s | 1st of 9 |
| Disk write (higher is better) | 2.4 GB/sp95 2.9 GB/s95%: 2.4 GB/s to 2.4 GB/s | 2.9 GB/s | 2.4 GB/s to 2.4 GB/s | 1st of 9 |
| Disk read (higher is better) | 23.0 GB/sp95 24.0 GB/s95%: 22.9 GB/s to 23.3 GB/s | 24.0 GB/s | 22.9 GB/s to 23.3 GB/s | 1st of 9 |
| 4K random write (higher is better) | 43.3K IOPSp95 46.5K IOPS95%: 43K IOPS to 43.9K IOPS | 46.5K IOPS | 43K IOPS to 43.9K IOPS | 1st of 9 |
| 4K random read (higher is better) | 543.8K IOPSp95 577.6K IOPS95%: 536.3K IOPS to 557.1K IOPS | 577.6K IOPS | 536.3K IOPS to 557.1K IOPS | 3rd of 9 · highest median: Boat 772.6K IOPS |
Quick picks on this track, all providers
| Need | Pick | Runner-up |
|---|---|---|
| Fastest cold start | E2B 0.50 s | Superserve 0.53 s |
| Fastest agent work (tool replay) | Superserve 13 s | Daytona 15 s |
| Fastest 25-sandbox fan-out | Superserve 0.75 s | E2B 0.87 s |
| Fastest reconnect | Daytona 0.31 s | Superserve 0.32 s |
| Cheapest task | Upstash Ephemeral Box $0.00013 per task, $0.13 per 1,000 (billed) | Upstash Box $0.00017 per task, $0.17 per 1,000 (billed) |
| Most reliable | E2B, Tenki, Superserve, Upstash Box, Daytona, Boat 100.00% (tied) | boxd 99.81% |
How to read these numbers
- Upstash appears twice because Box (long-lived, can pause) and Ephemeral Box (short-lived, for one-off agent jobs) are separate products.
- Ephemeral Box has no pause, so it has no pause result.
- Every call starts from one server in Florida, US. Providers hosted farther away pay that round trip on every call; Boat (France) and boxd (Germany) pay a transatlantic one.
- There are two tracks. Same machine: 4 vCPU, 8 GB: Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. A provider marked “machine differs” measured something else inside. Default machines: Each provider's default sandbox, so machine sizes differ.
- Cost per task has a kind. Billed comes from the provider's bill, metered from its usage records at published rates, and estimate from our measured time at list price. Kinds are not directly comparable. Prices cover compute only at list rates: disk, transfer, subscriptions and credits are excluded.
- Pause and resume times come from the previous SDK run, because the newest runs skipped them. A burst size is published only when all three bursts finished.
- Failures caused by our own code are set aside, not counted. Every provider-side failure is kept.
- Ranks compare medians. Medians inside each other's 95% intervals count as tied, so a close rank can be noise.
More results
- E2B sandbox benchmark
- Daytona sandbox benchmark
- Superserve sandbox benchmark
- Upstash Box sandbox benchmark
- Boat sandbox benchmark
- Freestyle sandbox benchmark
- Tenki sandbox benchmark
- Every sandbox provider we benchmark
- Every provider on the same 4 vCPU, 8 GB machine
- Every provider on its default machine
- Raw data: results.json · results.md · llms.txt