Freestyle sandbox benchmark
Our hardware probe found Freestyle runs each sandbox as a Firecracker microVM. We timed it through one SDK against 8 other sandbox providers: cold start, real agent work, everyday calls, pause and snapshots, bursts, machine speed, reliability and cost per task. Each result is a median with its p95 and 95% interval, on two tracks: the same 4 vCPU, 8 GB machine for every provider, and each provider's default machine.
Sandbox bench runs on Sandbox SDK, which the same team (opencoredev) builds. We may work with some of these providers. No provider paid for these results or decided how they were tested.
Same machine: 4 vCPU, 8 GB
- Cold start, median
- 0.69 s 3rd of 9
- Tool replay, median
- 24 s 6th of 9
Default machines
- Cold start, median
- 0.72 s 4th of 9
- Tool replay, median
- 26 s 6th of 9
Freestyle on the same 4 vCPU, 8 GB machine
Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. Runs from 2026-10-22 to 2026-10-23, 9 providers. See every provider side by side on the full results.
- Asked for
- 4 vCPU, 8 GB
- Measured inside
- 4 vCPU, 7.8 GB
- Cost per 1,000 tasks
- $1.97 (estimate)
- Success rate
- 99.91% (1,070 of 1,071)
- Cost: $0.0020 per task, our measured time at list price (estimate over 10 tasks, low confidence). A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.
- List price: $0.264 per hour for this machine, compute only.
- Pause and resume: supported; pause 0.32 s and resume 0.35 s median over 100 runs.
- Failures counted against Freestyle: Preview HTTP 502, no first byte (1 test)
Start-up
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Cold start (lower is better) | 0.69 sp95 2.0 s95%: 0.68 s to 0.72 s | 2.0 s | 0.68 s to 0.72 s | 3rd of 9 · lowest median: Superserve 0.52 s |
Agent work
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Agent tool replay (lower is better) | 24 sp95 42 s95%: 22 s to 29 s | 42 s | 22 s to 29 s | 6th of 9 · lowest median: Daytona 12 s |
| npm install (lower is better) | 1.7 sp95 3.5 s95%: 1.6 s to 1.9 s | 3.5 s | 1.6 s to 1.9 s | 8th of 9, tied with E2B · lowest median: Daytona 0.75 s |
| pip install (lower is better) | 7.5 sp95 20 s95%: 7.4 s to 8.4 s | 20 s | 7.4 s to 8.4 s | 9th of 9 · lowest median: Tenki 3.3 s |
| git clone (lower is better) | 1.5 sp95 2.6 s95%: 1.4 s to 1.7 s | 2.6 s | 1.4 s to 1.7 s | 7th of 9, tied with boxd · lowest median: Daytona 0.43 s |
| Dev server ready (lower is better) | 2.2 sp95 4.8 s95%: 2.1 s to 2.5 s | 4.8 s | 2.1 s to 2.5 s | 4th of 9, tied with E2B · lowest median: Superserve 1.3 s |
| Preview URL first byte (lower is better) | 0.48 sp95 0.65 s95%: 0.45 s to 0.51 s | 0.65 s | 0.45 s to 0.51 s | 7th of 9 · lowest median: Superserve 0.18 s |
| Edit, rebuild, reload (lower is better) | 0.30 sp95 1.6 s95%: 0.29 s to 0.32 s | 1.6 s | 0.29 s to 0.32 s | 1st of 9, tied with Daytona |
Everyday calls
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Run a command (lower is better) | 0.16 sp95 0.26 s95%: 0.16 s to 0.16 s | 0.26 s | 0.16 s to 0.16 s | 3rd of 9 · lowest median: E2B 0.13 s |
| Write + read 1 KB (lower is better) | 0.21 sp95 2.4 s95%: 0.21 s to 0.22 s | 2.4 s | 0.21 s to 0.22 s | 3rd of 9 · lowest median: Superserve 0.10 s |
| Write + read 10 MB (lower is better) | 4.8 sp95 19 s95%: 4.2 s to 5.2 s | 19 s | 4.2 s to 5.2 s | 7th of 9 · lowest median: Superserve 0.66 s |
| Reconnect (lower is better) | 0.49 sp95 0.79 s95%: 0.48 s to 0.51 s | 0.79 s | 0.48 s to 0.51 s | 4th of 9 · lowest median: Daytona 0.30 s |
| Destroy (lower is better) | 0.19 sp95 0.39 s95%: 0.18 s to 0.21 s | 0.39 s | 0.18 s to 0.21 s | 6th of 9, tied with Superserve · lowest median: E2B 0.11 s |
Pause and snapshots
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Pause + resume (lower is better) | 0.72 sp95 2.6 s95%: 0.68 s to 0.81 s | 2.6 s | 0.68 s to 0.81 s | 2nd of 6 · lowest median: boxd 0.53 s |
| Take snapshot (lower is better) | 0.43 sp95 1.1 s95%: 0.37 s to 0.54 s | 1.1 s | 0.37 s to 0.54 s | 2nd of 8 · lowest median: Superserve 0.39 s |
| Start from snapshot (lower is better) | 0.75 sp95 1.1 s95%: 0.71 s to 0.82 s | 1.1 s | 0.71 s to 0.82 s | 2nd of 8 · lowest median: Superserve 0.55 s |
Bursts
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| 10 at once, all ready (lower is better) | 1.1 sp95 8.0 s95%: 1.00 s to 8.0 s | 8.0 s | 1.00 s to 8.0 s | 3rd of 8, tied with Tenki · lowest median: Superserve 0.84 s |
| 25 at once, all ready (lower is better) | 2.0 sp95 4.8 s95%: 1.4 s to 4.8 s | 4.8 s | 1.4 s to 4.8 s | 5th of 9 · lowest median: Superserve 0.79 s |
Machine speed
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| CPU, one core (higher is better) | 9.7M/sp95 10.4M/s95%: 9.6M/s to 9.9M/s | 10.4M/s | 9.6M/s to 9.9M/s | 8th of 9 · highest median: Tenki 19.4M/s |
| CPU, all cores (higher is better) | 27.7M/sp95 30.6M/s95%: 27M/s to 28.4M/s | 30.6M/s | 27M/s to 28.4M/s | 7th of 9 · highest median: Upstash Ephemeral Box 51M/s |
| Memory bandwidth (higher is better) | 14.7 GB/sp95 20.5 GB/s95%: 13.5 GB/s to 16.3 GB/s | 20.5 GB/s | 13.5 GB/s to 16.3 GB/s | 8th of 9 · highest median: Tenki 45.9 GB/s |
| Disk write (higher is better) | 198 MB/sp95 249 MB/s95%: 187 MB/s to 215 MB/s | 249 MB/s | 187 MB/s to 215 MB/s | 8th of 9 · highest median: Upstash Ephemeral Box 2.4 GB/s |
| Disk read (higher is better) | 7.0 GB/sp95 8.8 GB/s95%: 6.4 GB/s to 7.4 GB/s | 8.8 GB/s | 6.4 GB/s to 7.4 GB/s | 7th of 9 · highest median: Upstash Ephemeral Box 23.1 GB/s |
| 4K random write (higher is better) | 6.1K IOPSp95 8.1K IOPS95%: 5.6K IOPS to 6.3K IOPS | 8.1K IOPS | 5.6K IOPS to 6.3K IOPS | 6th of 9, tied with Boat · highest median: Upstash Ephemeral Box 45.1K IOPS |
| 4K random read (higher is better) | 258.8K IOPSp95 338.1K IOPS95%: 245.4K IOPS to 271.7K IOPS | 338.1K IOPS | 245.4K IOPS to 271.7K IOPS | 8th of 9 · highest median: Boat 796.4K IOPS |
Quick picks on this track, all providers
| Need | Pick | Runner-up |
|---|---|---|
| Fastest cold start | Superserve 0.52 s | Upstash Ephemeral Box 0.68 s |
| Fastest agent work (tool replay) | Daytona 12 s | Superserve 13 s |
| Fastest 25-sandbox fan-out | Superserve 0.79 s | E2B 1.1 s |
| Fastest reconnect | Daytona 0.30 s | Superserve 0.33 s |
| Cheapest task | Upstash Ephemeral Box $0.00026 per task, $0.26 per 1,000 (billed) | Upstash Box $0.00034 per task, $0.34 per 1,000 (billed) |
| Most reliable | E2B, Tenki, Superserve, Upstash Box, Daytona, Upstash Ephemeral Box, Boat 100.00% (tied) | Freestyle 99.91% |
Freestyle on its default machine
Each provider's default sandbox, so machine sizes differ. Runs from 2026-10-22 to 2026-10-23, 9 providers. See every provider side by side on the full results.
- Asked for
- 4 vCPU, 8 GB
- Measured inside
- 4 vCPU, 7.8 GB
- Cost per 1,000 tasks
- $1.78 (estimate)
- Success rate
- 99.81% (1,072 of 1,074)
- Cost: $0.0018 per task, our measured time at list price (estimate over 10 tasks, low confidence). A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.
- List price: $0.264 per hour for this machine, compute only.
- Pause and resume: supported; pause 0.27 s and resume 0.32 s median over 100 runs.
- Failures counted against Freestyle: Preview HTTP 502, no first byte (1 test); Native operation timed out. (timeout) (1 test)
Start-up
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Cold start (lower is better) | 0.72 sp95 3.0 s95%: 0.71 s to 0.75 s | 3.0 s | 0.71 s to 0.75 s | 4th of 9 · lowest median: E2B 0.50 s |
Agent work
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Agent tool replay (lower is better) | 26 sp95 44 s95%: 22 s to 28 s | 44 s | 22 s to 28 s | 6th of 9 · lowest median: Superserve 13 s |
| npm install (lower is better) | 1.6 sp95 3.3 s95%: 1.6 s to 1.8 s | 3.3 s | 1.6 s to 1.8 s | 8th of 9 · lowest median: Boat 0.86 s |
| pip install (lower is better) | 7.5 sp95 14 s95%: 7.4 s to 9.7 s | 14 s | 7.4 s to 9.7 s | 9th of 9 · lowest median: Superserve 3.8 s |
| git clone (lower is better) | 1.5 sp95 2.8 s95%: 1.4 s to 1.8 s | 2.8 s | 1.4 s to 1.8 s | 7th of 9, tied with boxd · lowest median: Upstash Ephemeral Box 0.61 s |
| Dev server ready (lower is better) | 2.2 sp95 5.0 s95%: 2.0 s to 2.5 s | 5.0 s | 2.0 s to 2.5 s | 3rd of 9, tied with E2B · lowest median: Superserve 1.8 s |
| Preview URL first byte (lower is better) | 0.48 sp95 1.7 s95%: 0.45 s to 0.53 s | 1.7 s | 0.45 s to 0.53 s | 7th of 9 · lowest median: Superserve 0.19 s |
| Edit, rebuild, reload (lower is better) | 0.31 sp95 3.8 s95%: 0.29 s to 0.34 s | 3.8 s | 0.29 s to 0.34 s | 1st of 9 |
Everyday calls
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Run a command (lower is better) | 0.16 sp95 0.26 s95%: 0.16 s to 0.17 s | 0.26 s | 0.16 s to 0.17 s | 3rd of 9 · lowest median: E2B 0.14 s |
| Write + read 1 KB (lower is better) | 0.21 sp95 0.55 s95%: 0.21 s to 0.21 s | 0.55 s | 0.21 s to 0.21 s | 3rd of 9, tied with Tenki · lowest median: Superserve 0.10 s |
| Write + read 10 MB (lower is better) | 2.9 sp95 11 s95%: 2.7 s to 3.4 s | 11 s | 2.7 s to 3.4 s | 5th of 9, tied with Upstash Box · lowest median: Superserve 0.64 s |
| Reconnect (lower is better) | 0.50 sp95 0.83 s95%: 0.48 s to 0.51 s | 0.83 s | 0.48 s to 0.51 s | 4th of 9 · lowest median: Daytona 0.31 s |
| Destroy (lower is better) | 0.19 sp95 0.46 s95%: 0.18 s to 0.21 s | 0.46 s | 0.18 s to 0.21 s | 6th of 9, tied with Superserve · lowest median: E2B 0.11 s |
Pause and snapshots
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| Pause + resume (lower is better) | 0.64 sp95 2.8 s95%: 0.60 s to 0.69 s | 2.8 s | 0.60 s to 0.69 s | 2nd of 6 · lowest median: boxd 0.54 s |
| Take snapshot (lower is better) | 0.39 sp95 1.3 s95%: 0.33 s to 0.46 s | 1.3 s | 0.33 s to 0.46 s | 3rd of 8 · lowest median: E2B 0.34 s |
| Start from snapshot (lower is better) | 0.73 sp95 18 s95%: 0.70 s to 0.90 s | 18 s | 0.70 s to 0.90 s | 2nd of 8, tied with E2B · lowest median: Superserve 0.56 s |
Bursts
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| 10 at once, all ready (lower is better) | 1.9 sp95 2.6 s95%: 1.7 s to 2.6 s | 2.6 s | 1.7 s to 2.6 s | 5th of 9, tied with Daytona, boxd · lowest median: Superserve 0.65 s |
| 25 at once, all ready (lower is better) | 4.6 sp95 9.7 s95%: 2.9 s to 9.7 s | 9.7 s | 2.9 s to 9.7 s | 6th of 8 · lowest median: Superserve 0.75 s |
Machine speed
| Test | Median | p95 | 95% interval | Rank |
|---|---|---|---|---|
| CPU, one core (higher is better) | 9.6M/sp95 10.4M/s95%: 9.5M/s to 9.9M/s | 10.4M/s | 9.5M/s to 9.9M/s | 8th of 9 · highest median: Boat 18.8M/s |
| CPU, all cores (higher is better) | 26.1M/sp95 30.8M/s95%: 24.9M/s to 27.1M/s | 30.8M/s | 24.9M/s to 27.1M/s | 2nd of 9, tied with Upstash Ephemeral Box · highest median: Boat 51.7M/s |
| Memory bandwidth (higher is better) | 14.4 GB/sp95 20.5 GB/s95%: 13.0 GB/s to 15.8 GB/s | 20.5 GB/s | 13.0 GB/s to 15.8 GB/s | 8th of 9 · highest median: Upstash Ephemeral Box 41.1 GB/s |
| Disk write (higher is better) | 203 MB/sp95 244 MB/s95%: 191 MB/s to 210 MB/s | 244 MB/s | 191 MB/s to 210 MB/s | 8th of 9 · highest median: Upstash Ephemeral Box 2.4 GB/s |
| Disk read (higher is better) | 6.5 GB/sp95 8.7 GB/s95%: 6.0 GB/s to 7.3 GB/s | 8.7 GB/s | 6.0 GB/s to 7.3 GB/s | 7th of 9 · highest median: Upstash Ephemeral Box 23.0 GB/s |
| 4K random write (higher is better) | 5.6K IOPSp95 7.9K IOPS95%: 4.7K IOPS to 6.4K IOPS | 7.9K IOPS | 4.7K IOPS to 6.4K IOPS | 6th of 9, tied with Boat · highest median: Upstash Ephemeral Box 43.3K IOPS |
| 4K random read (higher is better) | 267K IOPSp95 335.7K IOPS95%: 252.7K IOPS to 283.8K IOPS | 335.7K IOPS | 252.7K IOPS to 283.8K IOPS | 8th of 9 · highest median: Boat 772.6K IOPS |
Quick picks on this track, all providers
| Need | Pick | Runner-up |
|---|---|---|
| Fastest cold start | E2B 0.50 s | Superserve 0.53 s |
| Fastest agent work (tool replay) | Superserve 13 s | Daytona 15 s |
| Fastest 25-sandbox fan-out | Superserve 0.75 s | E2B 0.87 s |
| Fastest reconnect | Daytona 0.31 s | Superserve 0.32 s |
| Cheapest task | Upstash Ephemeral Box $0.00013 per task, $0.13 per 1,000 (billed) | Upstash Box $0.00017 per task, $0.17 per 1,000 (billed) |
| Most reliable | E2B, Tenki, Superserve, Upstash Box, Daytona, Boat 100.00% (tied) | boxd 99.81% |
How to read these numbers
- Freestyle, Tenki and Boat costs are list-price estimates on our measured sandbox lifetime, because none gives a per-sandbox bill (Boat drops it when a sandbox is deleted).
- Every call starts from one server in Florida, US. Providers hosted farther away pay that round trip on every call; Boat (France) and boxd (Germany) pay a transatlantic one.
- There are two tracks. Same machine: 4 vCPU, 8 GB: Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size. A provider marked “machine differs” measured something else inside. Default machines: Each provider's default sandbox, so machine sizes differ.
- Cost per task has a kind. Billed comes from the provider's bill, metered from its usage records at published rates, and estimate from our measured time at list price. Kinds are not directly comparable. Prices cover compute only at list rates: disk, transfer, subscriptions and credits are excluded.
- Pause and resume times come from the previous SDK run, because the newest runs skipped them. A burst size is published only when all three bursts finished.
- Failures caused by our own code are set aside, not counted. Every provider-side failure is kept.
- Ranks compare medians. Medians inside each other's 95% intervals count as tied, so a close rank can be noise.
More results
- E2B sandbox benchmark
- Daytona sandbox benchmark
- Superserve sandbox benchmark
- Upstash Box sandbox benchmark
- Upstash Ephemeral Box sandbox benchmark
- Boat sandbox benchmark
- Tenki sandbox benchmark
- Every sandbox provider we benchmark
- Every provider on the same 4 vCPU, 8 GB machine
- Every provider on its default machine
- Raw data: results.json · results.md · llms.txt