Sandbox benchFull results

Upstash Box vs E2B

We timed Upstash Box and E2B through one SDK, alongside 7 other sandbox providers, on two tracks: the same 4 vCPU, 8 GB machine for every provider, and each provider's default machine. Every result is a median with its p95 and 95% interval. When the two medians sit inside each other's intervals, we call it a tie.

Sandbox bench runs on Sandbox SDK, which the same team (opencoredev) builds. We may work with some of these providers. No provider paid for these results or decided how they were tested.

Same machine: 4 vCPU, 8 GB

Upstash Box is ahead on 9 of the 24 tests both finished, E2B on 14, and 1 is tied within their 95% intervals.

  • Median cold start Upstash Box 2.7 s vs E2B 0.79 s (E2B ahead).
  • Median agent tool replay Upstash Box 27 s vs E2B 15 s (E2B ahead).

Default machines

Upstash Box is ahead on 9 of the 25 tests both finished, E2B on 16, and 0 are tied within their 95% intervals.

  • Median cold start Upstash Box 2.6 s vs E2B 0.50 s (E2B ahead).
  • Median agent tool replay Upstash Box 27 s vs E2B 16 s (E2B ahead).

Upstash Box vs E2B on the same 4 vCPU, 8 GB machine

Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. Runs from 2026-10-22 to 2026-10-23, 9 providers. Upstash Box is ahead on 9 of the 24 tests both finished, E2B on 14, and 1 is tied within their 95% intervals. Ranks are out of every provider on this track; see them all on the full results.

Upstash Box

Measured inside
4 vCPU, 8 GB
Cost per 1,000 tasks
$0.34 (billed)
Success rate
100.00% (1,075 of 1,075)
Pause and resume
pause 0.84 s, resume 0.57 s (median of 99)
  • Asked for: 4 vCPU, 8 GB ("medium").
  • Ran in: Ashburn, US.
  • Cost: $0.00034 per task, from its bill (billed over 10 tasks, high confidence).
  • Failures counted against Upstash Box: none.

E2B

Measured inside
4 vCPU, 7.8 GB
Cost per 1,000 tasks
$1.56 (metered)
Success rate
100.00% (1,073 of 1,073)
Pause and resume
pause 1.29 s, resume 0.51 s (median of 100)
  • Asked for: 4 vCPU, 8 GB, template r1h7gud12rf1f6iu29r0.
  • Ran in: The Dalles, US.
  • Cost: $0.0016 per task, their own usage records at published rates (metered over 10 tasks, medium confidence).
  • Failures counted against E2B: none.

Upstash Box's cost is billed and E2B's is metered, so the two aren't directly comparable. A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.

Start-up

TestUpstash BoxE2BAhead
Cold start (lower is better)2.7 s 9th of 9p95 11 s95%: 2.6 s to 2.7 s0.79 s 4th of 9p95 2.2 s95%: 0.71 s to 0.82 sE2B

Agent work

TestUpstash BoxE2BAhead
Agent tool replay (lower is better)27 s 7th of 9p95 32 s95%: 26 s to 28 s15 s 3rd of 9p95 17 s95%: 15 s to 16 sE2B
npm install (lower is better)1.1 s 4th of 9p95 1.2 s95%: 1.1 s to 1.1 s1.8 s 9th of 9p95 2.9 s95%: 1.7 s to 1.9 sUpstash Box
pip install (lower is better)4.0 s 4th of 9p95 5.0 s95%: 4.0 s to 4.1 s4.9 s 7th of 9p95 8.2 s95%: 4.2 s to 5.0 sUpstash Box
git clone (lower is better)0.69 s 3rd of 9p95 1.7 s95%: 0.67 s to 0.72 s1.2 s 6th of 9p95 2.1 s95%: 1.1 s to 1.3 sUpstash Box
Dev server ready (lower is better)3.7 s 7th of 9p95 5.6 s95%: 3.7 s to 3.8 s2.2 s 3rd of 9p95 3.4 s95%: 2.1 s to 2.3 sE2B
Preview URL first byte (lower is better)0.31 s 5th of 9p95 0.73 s95%: 0.29 s to 0.34 s0.27 s 3rd of 9p95 0.49 s95%: 0.27 s to 0.28 sE2B
Edit, rebuild, reload (lower is better)0.43 s 5th of 9p95 0.93 s95%: 0.42 s to 0.43 s0.33 s 3rd of 9p95 0.56 s95%: 0.32 s to 0.34 sE2B

Everyday calls

TestUpstash BoxE2BAhead
Run a command (lower is better)0.33 s 8th of 9p95 0.41 s95%: 0.32 s to 0.33 s0.13 s 1st of 9p95 0.19 s95%: 0.13 s to 0.14 sE2B
Write + read 1 KB (lower is better)0.36 s 7th of 9p95 0.71 s95%: 0.35 s to 0.36 s0.23 s 4th of 9p95 0.38 s95%: 0.23 s to 0.24 sE2B
Write + read 10 MB (lower is better)3.2 s 5th of 9p95 11 s95%: 3.1 s to 3.7 s1.3 s 3rd of 9p95 2.3 s95%: 1.3 s to 1.4 sE2B
Reconnect (lower is better)0.60 s 6th of 9p95 0.81 s95%: 0.59 s to 0.63 s0.46 s 3rd of 9p95 1.8 s95%: 0.45 s to 0.46 sE2B
Destroy (lower is better)1.0 s 9th of 9p95 12 s95%: 0.71 s to 1.7 s0.11 s 1st of 9p95 0.15 s95%: 0.11 s to 0.12 sE2B

Pause and snapshots

TestUpstash BoxE2BAhead
Pause + resume (lower is better)1.6 s 4th of 6p95 12 s95%: 1.3 s to 2.1 s1.8 s 5th of 6p95 3.5 s95%: 1.6 s to 2.0 sTied
Take snapshot (lower is better)2.1 s 5th of 8p95 8.4 s95%: 2.1 s to 4.2 s1.8 s 3rd of 8p95 2.2 s95%: 1.1 s to 2.0 sE2B
Start from snapshot (lower is better)4.9 s 8th of 8p95 13 s95%: 4.8 s to 6.9 s1.1 s 3rd of 8p95 3.2 s95%: 0.82 s to 1.6 sE2B

Bursts

TestUpstash BoxE2BAhead
10 at once, all ready (lower is better)3.0 s 8th of 8p95 4.8 s95%: 2.7 s to 4.8 sIn one of its three bursts of 10, two creates failed on E2B's side (E2B reported a timeout), so only 2 of 3 bursts finished and the size is not published; coming in the next updaten/a
25 at once, all ready (lower is better)5.4 s 8th of 9p95 5.7 s95%: 2.8 s to 5.7 s1.1 s 2nd of 9p95 1.3 s95%: 0.94 s to 1.3 sE2B

Machine speed

TestUpstash BoxE2BAhead
CPU, one core (higher is better)13.7M/s 5th of 9p95 13.8M/s95%: 13.7M/s to 13.7M/s10.3M/s 7th of 9p95 15.3M/s95%: 9.8M/s to 10.6M/sUpstash Box
CPU, all cores (higher is better)48.2M/s 3rd of 9p95 50.5M/s95%: 47.8M/s to 49.1M/s26.5M/s 8th of 9p95 37.8M/s95%: 24.8M/s to 27M/sUpstash Box
Memory bandwidth (higher is better)33.8 GB/s 3rd of 9p95 40.4 GB/s95%: 33.0 GB/s to 35.3 GB/s12.3 GB/s 9th of 9p95 14.0 GB/s95%: 11.7 GB/s to 12.7 GB/sUpstash Box
Disk write (higher is better)1.6 GB/s 2nd of 9p95 2.2 GB/s95%: 1.2 GB/s to 1.8 GB/s867 MB/s 4th of 9p95 1.0 GB/s95%: 750 MB/s to 902 MB/sUpstash Box
Disk read (higher is better)20.3 GB/s 2nd of 9p95 22.8 GB/s95%: 19.9 GB/s to 20.7 GB/s6.2 GB/s 8th of 9p95 7.5 GB/s95%: 5.9 GB/s to 6.4 GB/sUpstash Box
4K random write (higher is better)340.4 IOPS 9th of 9p95 547.4 IOPS95%: 304.7 IOPS to 397.6 IOPS6.9K IOPS 5th of 9p95 8.9K IOPS95%: 6.6K IOPS to 7.1K IOPSE2B
4K random read (higher is better)501.9K IOPS 4th of 9p95 527.2K IOPS95%: 498.1K IOPS to 505.3K IOPS453K IOPS 5th of 9p95 601.5K IOPS95%: 436.3K IOPS to 476.2K IOPSUpstash Box

Upstash Box vs E2B on each one's default machine

Each provider's default sandbox, so machine sizes differ. Runs from 2026-10-22 to 2026-10-23, 9 providers. Upstash Box is ahead on 9 of the 25 tests both finished, E2B on 16, and 0 are tied within their 95% intervals. Ranks are out of every provider on this track; see them all on the full results.

Upstash Box

Measured inside
2 vCPU, 4 GB
Cost per 1,000 tasks
$0.17 (billed)
Success rate
100.00% (1,076 of 1,076)
Pause and resume
pause 0.73 s, resume 0.59 s (median of 100)
  • Asked for: 2 vCPU, 4 GB.
  • Ran in: Ashburn, US.
  • Cost: $0.00017 per task, from its bill (billed over 10 tasks, high confidence).
  • Failures counted against Upstash Box: none.

E2B

Measured inside
2 vCPU, 473 MB
Cost per 1,000 tasks
$0.49 (metered)
Success rate
100.00% (1,076 of 1,076)
Pause and resume
pause 0.26 s, resume 0.45 s (median of 100)
  • Asked for: 2 vCPU, 512 MB, template base.
  • Machine: E2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.
  • Ran in: The Dalles, US.
  • Cost: $0.00049 per task, their own usage records at published rates (metered over 10 tasks, medium confidence).
  • Failures counted against E2B: none.

Upstash Box's cost is billed and E2B's is metered, so the two aren't directly comparable. A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.

Start-up

TestUpstash BoxE2BAhead
Cold start (lower is better)2.6 s 9th of 9p95 13 s95%: 2.6 s to 2.7 s0.50 s 1st of 9p95 2.0 s95%: 0.48 s to 0.51 sE2B

Agent work

TestUpstash BoxE2BAhead
Agent tool replay (lower is better)27 s 7th of 9p95 31 s95%: 26 s to 28 s16 s 3rd of 9p95 18 s95%: 16 s to 17 sE2B
npm install (lower is better)1.1 s 3rd of 9p95 1.2 s95%: 1.1 s to 1.1 s2.1 s 9th of 9p95 2.8 s95%: 2.0 s to 2.3 sUpstash Box
pip install (lower is better)4.1 s 3rd of 9p95 5.0 s95%: 4.0 s to 4.4 s5.0 s 8th of 9p95 6.0 s95%: 4.8 s to 5.1 sUpstash Box
git clone (lower is better)0.70 s 3rd of 9p95 1.3 s95%: 0.68 s to 0.77 s1.2 s 5th of 9p95 2.6 s95%: 1.1 s to 1.2 sUpstash Box
Dev server ready (lower is better)3.8 s 7th of 9p95 4.8 s95%: 3.7 s to 3.8 s2.3 s 4th of 9p95 3.3 s95%: 2.2 s to 2.6 sE2B
Preview URL first byte (lower is better)0.30 s 5th of 9p95 0.43 s95%: 0.29 s to 0.31 s0.27 s 2nd of 9p95 0.53 s95%: 0.26 s to 0.28 sE2B
Edit, rebuild, reload (lower is better)0.43 s 5th of 9p95 1.1 s95%: 0.42 s to 0.44 s0.34 s 2nd of 9p95 0.59 s95%: 0.33 s to 0.36 sE2B

Everyday calls

TestUpstash BoxE2BAhead
Run a command (lower is better)0.32 s 8th of 9p95 0.41 s95%: 0.32 s to 0.33 s0.14 s 1st of 9p95 0.15 s95%: 0.14 s to 0.14 sE2B
Write + read 1 KB (lower is better)0.36 s 7th of 9p95 1.1 s95%: 0.35 s to 0.36 s0.24 s 5th of 9p95 0.26 s95%: 0.23 s to 0.24 sE2B
Write + read 10 MB (lower is better)2.9 s 6th of 9p95 7.5 s95%: 2.8 s to 3.1 s1.3 s 3rd of 9p95 2.4 s95%: 1.3 s to 1.4 sE2B
Reconnect (lower is better)0.60 s 6th of 9p95 0.76 s95%: 0.59 s to 0.64 s0.46 s 3rd of 9p95 0.68 s95%: 0.46 s to 0.46 sE2B
Destroy (lower is better)0.96 s 9th of 9p95 7.8 s95%: 0.80 s to 1.2 s0.11 s 1st of 9p95 0.12 s95%: 0.11 s to 0.11 sE2B

Pause and snapshots

TestUpstash BoxE2BAhead
Pause + resume (lower is better)1.7 s 5th of 6p95 13 s95%: 1.3 s to 2.8 s0.74 s 3rd of 6p95 3.6 s95%: 0.72 s to 0.84 sE2B
Take snapshot (lower is better)2.1 s 5th of 8p95 17 s95%: 2.1 s to 4.2 s0.34 s 1st of 8p95 0.67 s95%: 0.32 s to 0.39 sE2B
Start from snapshot (lower is better)4.8 s 8th of 8p95 15 s95%: 2.8 s to 4.8 s0.75 s 3rd of 8p95 1.2 s95%: 0.55 s to 0.92 sE2B

Bursts

TestUpstash BoxE2BAhead
10 at once, all ready (lower is better)5.0 s 9th of 9p95 6.8 s95%: 2.8 s to 6.8 s1.1 s 4th of 9p95 1.1 s95%: 0.62 s to 1.1 sE2B
25 at once, all ready (lower is better)15 s 8th of 8p95 17 s95%: 9.0 s to 17 s0.87 s 2nd of 8p95 1.2 s95%: 0.83 s to 1.2 sE2B

Machine speed

TestUpstash BoxE2BAhead
CPU, one core (higher is better)13.7M/s 4th of 9p95 13.8M/s95%: 13.7M/s to 13.7M/s10M/s 7th of 9p95 15.3M/s95%: 9.7M/s to 10.6M/sUpstash Box
CPU, all cores (higher is better)25M/s 4th of 9p95 25.8M/s95%: 24.6M/s to 25.4M/s14.1M/s 7th of 9p95 20.2M/s95%: 13.9M/s to 14.9M/sUpstash Box
Memory bandwidth (higher is better)35.3 GB/s 3rd of 9p95 40.1 GB/s95%: 33.9 GB/s to 36.2 GB/s12.3 GB/s 9th of 9p95 14.2 GB/s95%: 12.1 GB/s to 12.7 GB/sUpstash Box
Disk write (higher is better)1.7 GB/s 2nd of 9p95 2.2 GB/s95%: 1.5 GB/s to 1.9 GB/s949 MB/s 4th of 9p95 1.1 GB/s95%: 885 MB/s to 996 MB/sUpstash Box
Disk read (higher is better)20.8 GB/s 2nd of 9p95 22.8 GB/s95%: 20.4 GB/s to 21.1 GB/s6.2 GB/s 8th of 9p95 7.6 GB/s95%: 6.1 GB/s to 6.5 GB/sUpstash Box
4K random write (higher is better)439.1 IOPS 8th of 9p95 556.2 IOPS95%: 379.4 IOPS to 480.7 IOPS7.2K IOPS 5th of 9p95 8.8K IOPS95%: 6.8K IOPS to 7.5K IOPSE2B
4K random read (higher is better)505.2K IOPS 4th of 9p95 527.3K IOPS95%: 501.6K IOPS to 509.1K IOPS467K IOPS 5th of 9p95 643K IOPS95%: 451.7K IOPS to 479.1K IOPSUpstash Box

How to read these numbers

  • Upstash appears twice because Box (long-lived, can pause) and Ephemeral Box (short-lived, for one-off agent jobs) are separate products.
  • Ephemeral Box has no pause, so it has no pause result.
  • Every call starts from one server in Florida, US. Providers hosted farther away pay that round trip on every call; Boat (France) and boxd (Germany) pay a transatlantic one.
  • There are two tracks. Same machine: 4 vCPU, 8 GB: Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. A provider marked “machine differs” measured something else inside. Default machines: Each provider's default sandbox, so machine sizes differ.
  • Cost per task has a kind. Billed comes from the provider's bill, metered from its usage records at published rates, and estimate from our measured time at list price. Kinds are not directly comparable. Prices cover compute only at list rates: disk, transfer, subscriptions and credits are excluded.
  • Pause and resume times come from the previous SDK run, because the newest runs skipped them. A burst size is published only when all three bursts finished.
  • Failures caused by our own code are set aside, not counted. Every provider-side failure is kept.
  • Ranks compare medians. Medians inside each other's 95% intervals count as tied, so a close rank can be noise.

More results