Sandbox bench: cloud sandboxes for agents, measured

Where the time goes

Step by step, from median timings. Colour is the provider's time; grey is ours.

TraceCold start · every provider, median of each step

Fastest

497ms

Superserve · provider time

Slowest

3.08s

Upstash Box · provider time

Spread

6.2×

slowest vs fastest

Failures

0

of 700 attempts

Provider

0s5s10s

create()First commandOurs: Sandbox SDKRanked by provider time · pick a provider for its full trace

TraceCold start · every provider, median of each step

Fastest

647ms

Freestyle · provider time

Slowest

2.59s

Upstash Box · provider time

Spread

4.0×

slowest vs fastest

Failures

1

of 578 attempts

Provider

0s5s10s

create()First commandOurs: Sandbox SDKRanked by provider time · pick a provider for its full trace

Start-up

Cold start is the wait before an agent can do anything: from create() until the first command runs, the provider's own time. Ready through Sandbox SDK adds the setup our SDK does after that, marked as ours.

Cold start

create() to first command, without our SDK's setup · Median seconds · Lower is better

Superservemedian 0.51 sp95 3.4 s95% CI 0.50 s–0.53 s100 attemptsClick for where the time goes

E2Bmedian 0.56 sp95 3.2 s95% CI 0.55 s–0.57 s100 attemptsClick for where the time goes

Freestylemedian 0.63 sp95 1.4 s95% CI 0.61 s–0.64 s100 attemptsClick for where the time goes

Tenkimedian 0.87 sp95 4.1 s95% CI 0.85 s–0.89 s100 attemptsClick for where the time goes

boxdmedian 1.4 sp95 4.2 s95% CI 1.4 s–1.4 s100 attemptsClick for where the time goes

Daytonamedian 1.6 sp95 5.8 s95% CI 1.4 s–1.8 s100 attemptsClick for where the time goes

Upstash Boxmedian 3.0 sp95 23 s95% CI 2.7 s–4.2 s100 attemptsClick for where the time goes

0.51

Superservemedian 0.51 sp95 3.4 s95% CI 0.50 s–0.53 s100 attemptsClick for where the time goes

0.56

E2Bmedian 0.56 sp95 3.2 s95% CI 0.55 s–0.57 s100 attemptsClick for where the time goes

0.63

Freestylemedian 0.63 sp95 1.4 s95% CI 0.61 s–0.64 s100 attemptsClick for where the time goes

0.87

Tenkimedian 0.87 sp95 4.1 s95% CI 0.85 s–0.89 s100 attemptsClick for where the time goes

1.4

boxdmedian 1.4 sp95 4.2 s95% CI 1.4 s–1.4 s100 attemptsClick for where the time goes

1.6

Daytonamedian 1.6 sp95 5.8 s95% CI 1.4 s–1.8 s100 attemptsClick for where the time goes

3.0

Upstash Boxmedian 3.0 sp95 23 s95% CI 2.7 s–4.2 s100 attemptsClick for where the time goes

Notes (2)

1E2B: ran E2B's default base template, 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Ready through Sandbox SDK

create() until our SDK's handle is ready · Median seconds of each part · Lower is better

Provider cold start, the number on the Cold start tabOur SDK's extra setup, not the provider's

Superserveprovider cold start 0.51 sour SDK setup +0.81 sready through the SDK, median 1.3 s100 cold startsClick for where the time goes

E2Bprovider cold start 0.56 sour SDK setup +0.78 sready through the SDK, median 1.3 s100 cold startsClick for where the time goes

Freestyleprovider cold start 0.63 sour SDK setup +0.97 sready through the SDK, median 1.6 s100 cold startsClick for where the time goes

Tenkiprovider cold start 0.87 sour SDK setup +1.0 sready through the SDK, median 2.0 s100 cold startsClick for where the time goes

boxdprovider cold start 1.4 sour SDK setup +1.8 sready through the SDK, median 3.2 s100 cold startsClick for where the time goes

Daytonaprovider cold start 1.6 sour SDK setup +2.6 sready through the SDK, median 5.8 s100 cold startsClick for where the time goes

Upstash Boxprovider cold start 3.0 sour SDK setup +7.3 sready through the SDK, median 12 s100 cold startsClick for where the time goes

1.3

Superserveprovider cold start 0.51 sour SDK setup +0.81 sready through the SDK, median 1.3 s100 cold startsClick for where the time goes

1.3

E2Bprovider cold start 0.56 sour SDK setup +0.78 sready through the SDK, median 1.3 s100 cold startsClick for where the time goes

1.6

Freestyleprovider cold start 0.63 sour SDK setup +0.97 sready through the SDK, median 1.6 s100 cold startsClick for where the time goes

1.9

Tenkiprovider cold start 0.87 sour SDK setup +1.0 sready through the SDK, median 2.0 s100 cold startsClick for where the time goes

3.2

boxdprovider cold start 1.4 sour SDK setup +1.8 sready through the SDK, median 3.2 s100 cold startsClick for where the time goes

4.2

Daytonaprovider cold start 1.6 sour SDK setup +2.6 sready through the SDK, median 5.8 s100 cold startsClick for where the time goes

10

Upstash Boxprovider cold start 3.0 sour SDK setup +7.3 sready through the SDK, median 12 s100 cold startsClick for where the time goes

Notes (2)

The hatched part is work Sandbox SDK does after the sandbox can run commands, such as preparing the workspace. It is ours, so it is kept out of cold start. Each part is its own median, so the stack can differ a little from the median total shown on hover.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Cold start

create() to first command, without our SDK's setup · Median seconds · Lower is better

Freestylemedian 0.66 sp95 2.8 s95% CI 0.64 s–0.68 s100 attemptsClick for where the time goes

E2Bmedian 0.95 sp95 3.1 s95% CI 0.92 s–1.0 s100 attemptsClick for where the time goes

Tenkimedian 0.96 sp95 3.0 s95% CI 0.91 s–1.1 s78 attemptsClick for where the time goes

Daytonamedian 1.1 sp95 3.0 s95% CI 1.0 s–1.3 s100 attemptsClick for where the time goes

boxdmedian 1.4 sp95 3.9 s95% CI 1.4 s–1.4 s100 attemptsClick for where the time goes

Upstash Boxmedian 2.7 sp95 9.1 s95% CI 2.6 s–2.7 s100 attempts, 1 failedClick for where the time goes

0.66

Freestylemedian 0.66 sp95 2.8 s95% CI 0.64 s–0.68 s100 attemptsClick for where the time goes

0.95

E2Bmedian 0.95 sp95 3.1 s95% CI 0.92 s–1.0 s100 attemptsClick for where the time goes

0.96

Tenkimedian 0.96 sp95 3.0 s95% CI 0.91 s–1.1 s78 attemptsClick for where the time goes

1.1

Daytonamedian 1.1 sp95 3.0 s95% CI 1.0 s–1.3 s100 attemptsClick for where the time goes

1.4

boxdmedian 1.4 sp95 3.9 s95% CI 1.4 s–1.4 s100 attemptsClick for where the time goes

2.7

Upstash Boxmedian 2.7 sp95 9.1 s95% CI 2.6 s–2.7 s100 attempts, 1 failedClick for where the time goes

Notes (3)

1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Ready through Sandbox SDK

create() until our SDK's handle is ready · Median seconds of each part · Lower is better

Provider cold start, the number on the Cold start tabOur SDK's extra setup, not the provider's

Freestyleprovider cold start 0.66 sour SDK setup +0.96 sready through the SDK, median 1.7 s100 cold startsClick for where the time goes

E2Bprovider cold start 0.95 sour SDK setup +0.90 sready through the SDK, median 2.1 s100 cold startsClick for where the time goes

Tenkiprovider cold start 0.96 sour SDK setup +1.0 sready through the SDK, median 2.0 s78 cold startsClick for where the time goes

Daytonaprovider cold start 1.1 sour SDK setup +1.8 sready through the SDK, median 3.4 s100 cold startsClick for where the time goes

boxdprovider cold start 1.4 sour SDK setup +1.8 sready through the SDK, median 3.2 s100 cold startsClick for where the time goes

Upstash Boxprovider cold start 2.7 sour SDK setup +5.2 sready through the SDK, median 8.9 s99 cold startsClick for where the time goes

1.6

Freestyleprovider cold start 0.66 sour SDK setup +0.96 sready through the SDK, median 1.7 s100 cold startsClick for where the time goes

1.9

E2Bprovider cold start 0.95 sour SDK setup +0.90 sready through the SDK, median 2.1 s100 cold startsClick for where the time goes

2.0

Tenkiprovider cold start 0.96 sour SDK setup +1.0 sready through the SDK, median 2.0 s78 cold startsClick for where the time goes

3.0

Daytonaprovider cold start 1.1 sour SDK setup +1.8 sready through the SDK, median 3.4 s100 cold startsClick for where the time goes

3.2

boxdprovider cold start 1.4 sour SDK setup +1.8 sready through the SDK, median 3.2 s100 cold startsClick for where the time goes

7.8

Upstash Boxprovider cold start 2.7 sour SDK setup +5.2 sready through the SDK, median 8.9 s99 cold startsClick for where the time goes

Notes (4)

1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
The hatched part is work Sandbox SDK does after the sandbox can run commands, such as preparing the workspace. It is ours, so it is kept out of cold start. Each part is its own median, so the stack can differ a little from the median total shown on hover.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Agent work

Real coding-agent tasks, so CPU, disk and network count as much as the API.

Agent tool replay

100 tool calls in a row: write, read, edit, list, grep, run a test · Median seconds · Lower is better

Daytonamedian 11 sp95 31 s95% CI 10 s–14 s30 attemptsClick for where the time goes

Superservemedian 12 sp95 15 s95% CI 12 s–13 s30 attemptsClick for where the time goes

E2Bmedian 15 sp95 17 s95% CI 15 s–15 s30 attemptsClick for where the time goes

Tenkimedian 17 sp95 21 s95% CI 17 s–18 s30 attemptsClick for where the time goes

Freestylemedian 20 sp95 23 s95% CI 19 s–20 s30 attemptsClick for where the time goes

Upstash Boxmedian 24 sp95 30 s95% CI 24 s–25 s30 attempts, 1 failedClick for where the time goes

boxdmedian 31 sp95 34 s95% CI 31 s–32 s30 attemptsClick for where the time goes

11

Daytonamedian 11 sp95 31 s95% CI 10 s–14 s30 attemptsClick for where the time goes

12

Superservemedian 12 sp95 15 s95% CI 12 s–13 s30 attemptsClick for where the time goes

15

E2Bmedian 15 sp95 17 s95% CI 15 s–15 s30 attemptsClick for where the time goes

17

Tenkimedian 17 sp95 21 s95% CI 17 s–18 s30 attemptsClick for where the time goes

20

Freestylemedian 20 sp95 23 s95% CI 19 s–20 s30 attemptsClick for where the time goes

24

Upstash Boxmedian 24 sp95 30 s95% CI 24 s–25 s30 attempts, 1 failedClick for where the time goes

31

boxdmedian 31 sp95 34 s95% CI 31 s–32 s30 attemptsClick for where the time goes

Notes (2)

1boxd: runs in Germany, so every call from our Florida server crosses the Atlantic.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Agent tool replay

100 tool calls in a row: write, read, edit, list, grep, run a test · Median seconds · Lower is better

Daytonamedian 11 sp95 29 s95% CI 9.9 s–12 s30 attemptsClick for where the time goes

E2Bmedian 15 sp95 16 s95% CI 15 s–15 s30 attemptsClick for where the time goes

Freestylemedian 20 sp95 28 s95% CI 20 s–22 s30 attemptsClick for where the time goes

Tenkimedian 21 sp95 34 s95% CI 20 s–25 s30 attemptsClick for where the time goes

Upstash Boxmedian 28 sp95 54 s95% CI 24 s–30 s30 attemptsClick for where the time goes

boxdmedian 33 sp95 49 s95% CI 32 s–41 s30 attemptsClick for where the time goes

11

Daytonamedian 11 sp95 29 s95% CI 9.9 s–12 s30 attemptsClick for where the time goes

15

E2Bmedian 15 sp95 16 s95% CI 15 s–15 s30 attemptsClick for where the time goes

20

Freestylemedian 20 sp95 28 s95% CI 20 s–22 s30 attemptsClick for where the time goes

21

Tenkimedian 21 sp95 34 s95% CI 20 s–25 s30 attemptsClick for where the time goes

28

Upstash Boxmedian 28 sp95 54 s95% CI 24 s–30 s30 attemptsClick for where the time goes

33

boxdmedian 33 sp95 49 s95% CI 32 s–41 s30 attemptsClick for where the time goes

Notes (3)

1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest. Runs in Germany, so every call from our Florida server crosses the Atlantic.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Everyday calls

What an agent does hundreds of times per task: run a command, move files, reattach to a sandbox.

Run a command

A no-op command, mostly network and API overhead · Median seconds · Lower is better

Daytonamedian 0.12 sp95 0.29 s95% CI 0.11 s–0.13 s100 attempts

E2Bmedian 0.12 sp95 0.28 s95% CI 0.12 s–0.12 s100 attempts

Freestylemedian 0.15 sp95 0.20 s95% CI 0.14 s–0.15 s100 attempts

Superservemedian 0.15 sp95 0.23 s95% CI 0.15 s–0.16 s100 attempts

Tenkimedian 0.17 sp95 0.19 s95% CI 0.16 s–0.17 s100 attempts

Upstash Boxmedian 0.22 sp95 0.32 s95% CI 0.22 s–0.22 s100 attempts

boxdmedian 0.29 sp95 0.29 s95% CI 0.29 s–0.29 s100 attempts

0.12

Daytonamedian 0.12 sp95 0.29 s95% CI 0.11 s–0.13 s100 attempts

0.12

E2Bmedian 0.12 sp95 0.28 s95% CI 0.12 s–0.12 s100 attempts

0.15

Freestylemedian 0.15 sp95 0.20 s95% CI 0.14 s–0.15 s100 attempts

0.15

Superservemedian 0.15 sp95 0.23 s95% CI 0.15 s–0.16 s100 attempts

0.17

Tenkimedian 0.17 sp95 0.19 s95% CI 0.16 s–0.17 s100 attempts

0.22

Upstash Boxmedian 0.22 sp95 0.32 s95% CI 0.22 s–0.22 s100 attempts

0.29

boxdmedian 0.29 sp95 0.29 s95% CI 0.29 s–0.29 s100 attempts

Notes (1)

Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Run a command

A no-op command, mostly network and API overhead · Median seconds · Lower is better

Daytonamedian 0.11 sp95 0.28 s95% CI 0.10 s–0.11 s100 attempts

E2Bmedian 0.12 sp95 0.14 s95% CI 0.12 s–0.12 s100 attempts

Freestylemedian 0.15 sp95 0.22 s95% CI 0.14 s–0.15 s100 attempts

Tenkimedian 0.17 sp95 0.20 s95% CI 0.16 s–0.17 s78 attempts

Upstash Boxmedian 0.23 sp95 0.50 s95% CI 0.22 s–0.23 s99 attempts

boxdmedian 0.28 sp95 0.29 s95% CI 0.28 s–0.28 s100 attempts

0.11

Daytonamedian 0.11 sp95 0.28 s95% CI 0.10 s–0.11 s100 attempts

0.12

E2Bmedian 0.12 sp95 0.14 s95% CI 0.12 s–0.12 s100 attempts

0.15

Freestylemedian 0.15 sp95 0.22 s95% CI 0.14 s–0.15 s100 attempts

0.17

Tenkimedian 0.17 sp95 0.20 s95% CI 0.16 s–0.17 s78 attempts

0.23

Upstash Boxmedian 0.23 sp95 0.50 s95% CI 0.22 s–0.23 s99 attempts

0.28

boxdmedian 0.28 sp95 0.29 s95% CI 0.28 s–0.28 s100 attempts

Notes (3)

1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Pause & snapshots

Pausing stops paying for idle time while an agent waits on a human, with every process kept. Where stop and start restarts the sandbox instead, we mark pause as not supported. Snapshots start many sandboxes from one prepared state.

Resume

Resume a paused sandbox until a command runs · Median seconds · Lower is better

Freestylemedian 0.32 s100 attemptsClick for where the time goes

boxdmedian 0.41 s100 attemptsClick for where the time goes

E2Bmedian 0.45 s100 attemptsClick for where the time goes

Superservemedian 0.54 s100 attemptsClick for where the time goes

Upstash Boxmedian 0.59 s100 attemptsClick for where the time goes

Tenkimedian 2.3 s100 attemptsClick for where the time goes

0.32

Freestylemedian 0.32 s100 attemptsClick for where the time goes

0.41

boxdmedian 0.41 s100 attemptsClick for where the time goes

0.45

E2Bmedian 0.45 s100 attemptsClick for where the time goes

0.54

Superservemedian 0.54 s100 attemptsClick for where the time goes

0.59

Upstash Boxmedian 0.59 s100 attemptsClick for where the time goes

2.3

Tenkimedian 2.3 s100 attemptsClick for where the time goes

Notes (2)

Daytona: not supported. Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process. We don't time a restart as a pause.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Resume

Resume a paused sandbox until a command runs · Median seconds · Lower is better

Freestylemedian 0.35 s100 attemptsClick for where the time goes

boxdmedian 0.40 s99 attemptsClick for where the time goes

E2Bmedian 0.51 s100 attemptsClick for where the time goes

Upstash Boxmedian 0.57 s99 attemptsClick for where the time goes

Tenkimedian 2.4 s77 attemptsClick for where the time goes

0.35

Freestylemedian 0.35 s100 attemptsClick for where the time goes

0.40

boxdmedian 0.40 s99 attemptsClick for where the time goes

0.51

E2Bmedian 0.51 s100 attemptsClick for where the time goes

0.57

Upstash Boxmedian 0.57 s99 attemptsClick for where the time goes

2.4

Tenkimedian 2.4 s77 attemptsClick for where the time goes

Notes (4)

1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
Daytona: not supported. Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process. We don't time a restart as a pause.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Bursts

10 or 25 sandboxes started at once, as an agent fanning out would, timed until the last one runs a command.

10 at once, all ready

10 creates at once, until all 10 run a command · Median seconds · Lower is better

Superservemedian 0.72 sp95 0.72 s95% CI 0.64 s–0.72 s3 attempts

E2Bmedian 1.0 sp95 1.2 s95% CI 0.98 s–1.2 s3 attempts

Freestylemedian 1.1 sp95 1.1 s95% CI 0.85 s–1.1 s3 attempts

Tenkimedian 1.2 sp95 1.8 s95% CI 1.1 s–1.8 s3 attempts

boxdmedian 2.2 sp95 2.9 s95% CI 2.0 s–2.9 s3 attempts

Daytonamedian 2.8 sp95 4.4 s95% CI 2.2 s–4.4 s3 attempts

Upstash Boxmedian 3.8 sp95 13 s95% CI 2.8 s–13 s3 attempts

0.72

Superservemedian 0.72 sp95 0.72 s95% CI 0.64 s–0.72 s3 attempts

1.0

E2Bmedian 1.0 sp95 1.2 s95% CI 0.98 s–1.2 s3 attempts

1.1

Freestylemedian 1.1 sp95 1.1 s95% CI 0.85 s–1.1 s3 attempts

1.2

Tenkimedian 1.2 sp95 1.8 s95% CI 1.1 s–1.8 s3 attempts

2.2

boxdmedian 2.2 sp95 2.9 s95% CI 2.0 s–2.9 s3 attempts

2.8

Daytonamedian 2.8 sp95 4.4 s95% CI 2.2 s–4.4 s3 attempts

3.8

Upstash Boxmedian 3.8 sp95 13 s95% CI 2.8 s–13 s3 attempts

Notes (1)

Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

10 at once, all ready

10 creates at once, until all 10 run a command · Median seconds · Lower is better

Freestylemedian 1.0 sp95 1.0 s95% CI 0.89 s–1.0 s3 attempts

Tenkimedian 1.2 sp95 1.3 s95% CI 1.2 s–1.3 s3 attempts

E2Bmedian 1.4 sp95 1.6 s95% CI 1.2 s–1.6 s3 attempts

Upstash Boxmedian 21 sp95 25 s95% CI 4.8 s–25 s3 attempts

1.0

Freestylemedian 1.0 sp95 1.0 s95% CI 0.89 s–1.0 s3 attempts

1.2

Tenkimedian 1.2 sp95 1.3 s95% CI 1.2 s–1.3 s3 attempts

1.4

E2Bmedian 1.4 sp95 1.6 s95% CI 1.2 s–1.6 s3 attempts

21

Upstash Boxmedian 21 sp95 25 s95% CI 4.8 s–25 s3 attempts

Notes (3)

No result for Daytona (not run: our Daytona account's 10 vCPU quota can't hold ten 4 vCPU sandboxes at once); boxd (ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest. One of its three bursts of 10 was cut short by our own ownership check (1 create failed on our side), so it is not counted; coming in the next update).
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Machines

What each provider ran: what we asked for, what we measured inside, and the hardware underneath.

What each provider ran

Each provider's default sandbox, and the vCPU and RAM measured inside it

E2B
Asked for 2 vCPU, 512 MB, template base
Measured 2 vCPU, 473 MB
E2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.
Daytona
Asked for 1 vCPU, 1 GB
Measured 1 vCPU, 1 GB
Upstash Box
Asked for 2 vCPU, 4 GB
Measured 2 vCPU, 4 GB
boxd
Asked for 2 vCPU, 8 GB
Measured 2 vCPU, 7.8 GB
Freestyle
Asked for 4 vCPU, 8 GB
Measured 4 vCPU, 7.8 GB
Tenki
Asked for provider default
Measured 2 vCPU, 3.8 GB
Tenki documents a 5 GiB default disk but not a default vCPU or RAM, so those are what we measured.
Superserve
Asked for 1 vCPU, 1 GB, template sandbox-bench-node22-py311-1x1-pinned
Measured 1 vCPU, 994 MB
Superserve ran a template we built with Node 22 and Python 3.11 preinstalled, at its default 1 vCPU, 1 GB RAM and 4 GB disk. Superserve's own templates have Node (node-22) or Python (python-3.11), not both, and our tests need both.

Provider	Asked for	Measured inside
E2B	2 vCPU, 512 MB, template baseE2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.	2 vCPU, 473 MB
Daytona	1 vCPU, 1 GB	1 vCPU, 1 GB
Upstash Box	2 vCPU, 4 GB	2 vCPU, 4 GB
boxd	2 vCPU, 8 GB	2 vCPU, 7.8 GB
Freestyle	4 vCPU, 8 GB	4 vCPU, 7.8 GB
Tenki	provider defaultTenki documents a 5 GiB default disk but not a default vCPU or RAM, so those are what we measured.	2 vCPU, 3.8 GB
Superserve	1 vCPU, 1 GB, template sandbox-bench-node22-py311-1x1-pinnedSuperserve ran a template we built with Node 22 and Python 3.11 preinstalled, at its default 1 vCPU, 1 GB RAM and 4 GB disk. Superserve's own templates have Node (node-22) or Python (python-3.11), not both, and our tests need both.	1 vCPU, 994 MB

Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Hardware inside each sandbox

Chip, memory, cloud and region, and how each sandbox is isolated

Named means the CPU reports its own model. Inferred means we worked it out from what the sandbox shows, such as model number, cache and clock, or from the chip's published spec. Measured memory was read inside the sandbox. From a hardware probe on Oct 22, 2026, checked against the benchmark's fingerprints. Open a row for what each sandbox hides.

E2BChipIntel Xeon Platinum 8373CIce Lake, 2.6 GHz, 54 MB L3 · inferredMemoryDDR4inferredWhereGoogle Cloudus-west1, The Dalles, OregonRuns asFirecracker microVMnested inside a Google Cloud VM
Hidden or different from the real machine
The CPU name, microcode and clock leaves are hidden. The chip is identified from its model number, cache size and 2.6 GHz clock.
The default 2 vCPUs are one physical core with two hyperthreads, so two-thread work scales well below twice one thread.
No DMI or PCI devices, so the memory type can't be read inside; DDR4 follows from the chip.
Memory
DDR4-3200, the chip's memory type (inferred)
Disk
23 GB virtio disk, ext4
Kernel
Linux 6.1, Debian 12
Size
E2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.
DaytonaChipAMD EPYC 9354P, 9354, 9254 or 9275FZen 4 (Genoa); the 9275F is Zen 5 (Turin). Hosts vary · named by the CPUMemoryDDR5measuredWhereBare-metal serversLocation not visible; control plane on AWS us-east-1Runs asDocker containershares the host's kernel; cgroups set its CPU and RAM
Hidden or different from the real machine
Nothing at the CPU level is hidden: the real CPU name, microcode, server model (Dell PowerEdge R6615 in the probe, a Supermicro AS-2115GT-HNTR in the benchmark) and disks are visible.
nproc and lscpu show every host thread (48 or 64) and /proc/meminfo shows the host's RAM, while the default sandbox may use 1 CPU and 1 GB. Tools that size thread pools or caches from those numbers oversubscribe.
Hosts differ in clock speed, so single-thread results vary from sandbox to sandbox. Neighbours share the host kernel.
Outbound internet is blocked on our account tier, so the sandbox's own network location could not be read.
Memory
Registered DDR5, read from the memory controller on a 9254 host (measured)
Disk
3 GB container disk (10 GB at 4 vCPU) on the host's NVMe RAID
Kernel
Host kernel 6.17 (Ubuntu 24.04), Debian 13 userland
Chips in this track
9354P ×59, 9254 ×43, 9354 ×27, 9275F ×1, from the benchmark's fingerprints
Upstash BoxChipAWS Graviton4Arm Neoverse V2, 2.8 GHz; Graviton4 from the m8g instance type · inferredMemoryDDR5inferredWhereAWS EC2us-east-1, Ashburn, VirginiaRuns asDocker containeron a shared m8g.16xlarge VM; cgroups set its CPU and RAM
Hidden or different from the real machine
The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.
nproc and /proc/meminfo show the host's 64 CPUs and 256 GB, while a box may use 2 CPUs and 4 GB (4 and 8 at the medium size).
No DMI tables or clock readings inside, so memory type and clock come from AWS's published spec.
Neighbours share the host kernel and the chip's L3 cache.
Memory
DDR5-5600, from AWS's Graviton4 spec (inferred)
Disk
5 GB overlay root and a 5 GB /workspace, on EBS
Kernel
Host kernel 7.0 (Ubuntu 24.04 for AWS), Debian 12 userland
boxdChipAMD EPYC 9554P, likelyZen 4 (Genoa), 3.1 GHz · inferredMemoryDDR5inferredWhereLatitude.sh bare metalFrankfurt, GermanyRuns asFirecracker microVMwith Docker installed inside the VM
Hidden or different from the real machine
The CPU name is trimmed to "AMD EPYC" and the microcode and topology are hidden. The chip is inferred from its 3.1 GHz clock and the Genoa servers Latitude.sh sells; a dual-socket 9554 can't be ruled out.
No DMI, so the memory type is inferred.
AVX-512 is exposed, which can help vectorised work against hosts that hide it.
The disk reports itself as rotational although it is virtual, so tools that tune for hard drives may behave differently.
Memory
DDR5-4800, the Genoa platform's memory (inferred)
Disk
100 GB virtio disk, ext4
Kernel
Linux 6.18 (custom build), Ubuntu 24.04
FreestyleChipAMD EPYC 7763, likelyZen 3 (Milan), 2.45 GHz · inferredMemoryDDR4inferredWhereFreestyle's own network (AS36320)San Francisco Bay AreaRuns asFirecracker microVM4 vCPU, 8 GB, no hyperthreads exposed
Hidden or different from the real machine
The CPU name, microcode and topology are hidden. The chip is inferred from its 2.45 GHz clock, cache and Zen 3 features.
Memory is added after boot: about 3 GB is free at first, 7.9 GB later.
No DMI, so the memory type is inferred.
Memory
DDR4-3200, the Milan platform's memory (inferred)
Disk
32 GB virtio disk, ext4
Kernel
Linux 6.1
TenkiChipAMD EPYC 7B13, as reportedZen 3 (Milan), 2.25 GHz. The 4 vCPU VM also showed EPYC 9575F (Zen 5, Turin) cores · conflictingMemoryNot knownNo memory data inside the VMWhereLuxor (AS19538)SeattleRuns asCloud Hypervisor microVMwith a container inside
Hidden or different from the real machine
On the 4 vCPU VM, two vCPUs report an EPYC 7B13 (Milan) and two an EPYC 9575F (Turin). That fits a snapshot taken on a Milan host and restored on a Turin one, so the real host may be newer than the name says.
CPU features are capped at the Milan set on every vCPU, so no AVX-512 even on Turin.
The clock reads 2.25 GHz, the 7B13's base, which may be carried over from the snapshot.
Memory
No memory data inside the VM
Disk
5 GB virtio disk, ext4
Kernel
Linux 6.18
Size
Tenki documents a 5 GiB default disk but not a default vCPU or RAM, so those are what we measured.
SuperserveChipIntel Xeon, Sapphire RapidsLikely Xeon Platinum 8481C; 2.2 GHz, 105 MB L3 · inferredMemoryDDR5inferredWhereGoogle Cloudus-east4, Northern Virginia, from its IP rangeRuns asFirecracker microVMlikely inside a Google Cloud VM
Hidden or different from the real machine
The CPU name and microcode are hidden. The chip family is clear from its model number; the exact SKU is not.
The old 4.14 guest kernel can't turn on the chip's AMX units, so AMX workloads can't use them.
No DMI, so the memory type is inferred.
Memory
DDR5, the Sapphire Rapids platform's memory (inferred)
Disk
4 GB virtio disk, ext4
Kernel
Linux 4.14, Debian 12
Size
Superserve ran a template we built with Node 22 and Python 3.11 preinstalled, at its default 1 vCPU, 1 GB RAM and 4 GB disk. Superserve's own templates have Node (node-22) or Python (python-3.11), not both, and our tests need both.

What each provider ran

Asked for 4 vCPU, 8 GB; measured inside each sandbox

E2BMatches
Asked for 4 vCPU, 8 GB, template r1h7gud12rf1f6iu29r0
Measured 4 vCPU, 7.8 GB
DaytonaMatches
Asked for 4 vCPU, 8 GB, template daytona-large
Measured 4 vCPU, 8 GB
Upstash BoxMatches
Asked for 4 vCPU, 8 GB ("medium")
Measured 4 vCPU, 8 GB
boxdDiffers
Asked for 4 vCPU, 16 GB
Measured 4 vCPU, 16 GB
Ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
FreestyleMatches
Asked for 4 vCPU, 8 GB
Measured 4 vCPU, 7.8 GB
TenkiMatches
Asked for 4 vCPU, 8 GB
Measured 4 vCPU, 7.8 GB

Provider	Asked for	Measured inside	4 vCPU, 8 GB?
E2B	4 vCPU, 8 GB, template r1h7gud12rf1f6iu29r0	4 vCPU, 7.8 GB	Matches
Daytona	4 vCPU, 8 GB, template daytona-large	4 vCPU, 8 GB	Matches
Upstash Box	4 vCPU, 8 GB ("medium")	4 vCPU, 8 GB	Matches
boxd	4 vCPU, 16 GBRan 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.	4 vCPU, 16 GB	Differs
Freestyle	4 vCPU, 8 GB	4 vCPU, 7.8 GB	Matches
Tenki	4 vCPU, 8 GB	4 vCPU, 7.8 GB	Matches

Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Hardware inside each sandbox

Chip, memory, cloud and region, and how each sandbox is isolated

Named means the CPU reports its own model. Inferred means we worked it out from what the sandbox shows, such as model number, cache and clock, or from the chip's published spec. Measured memory was read inside the sandbox. From a hardware probe on Oct 22, 2026, checked against the benchmark's fingerprints. Open a row for what each sandbox hides.

E2BChipIntel Xeon Platinum 8373CIce Lake, 2.6 GHz, 54 MB L3 · inferredMemoryDDR4inferredWhereGoogle Cloudus-west1, The Dalles, OregonRuns asFirecracker microVMnested inside a Google Cloud VM
Hidden or different from the real machine
The CPU name, microcode and clock leaves are hidden. The chip is identified from its model number, cache size and 2.6 GHz clock.
The default 2 vCPUs are one physical core with two hyperthreads, so two-thread work scales well below twice one thread.
No DMI or PCI devices, so the memory type can't be read inside; DDR4 follows from the chip.
Memory
DDR4-3200, the chip's memory type (inferred)
Disk
23 GB virtio disk, ext4
Kernel
Linux 6.1, Debian 12
At 4 vCPU
The 4 vCPU, 8 GB template ran on the same CPU in the same region: two cores with two threads each.
DaytonaChipAMD EPYC 9354P, 9354, 9254 or 9275FZen 4 (Genoa); the 9275F is Zen 5 (Turin). Hosts vary · named by the CPUMemoryDDR5measuredWhereBare-metal serversLocation not visible; control plane on AWS us-east-1Runs asDocker containershares the host's kernel; cgroups set its CPU and RAM
Hidden or different from the real machine
Nothing at the CPU level is hidden: the real CPU name, microcode, server model (Dell PowerEdge R6615 in the probe, a Supermicro AS-2115GT-HNTR in the benchmark) and disks are visible.
nproc and lscpu show every host thread (48 or 64) and /proc/meminfo shows the host's RAM, while the default sandbox may use 1 CPU and 1 GB. Tools that size thread pools or caches from those numbers oversubscribe.
Hosts differ in clock speed, so single-thread results vary from sandbox to sandbox. Neighbours share the host kernel.
Outbound internet is blocked on our account tier, so the sandbox's own network location could not be read.
Memory
Registered DDR5, read from the memory controller on a 9254 host (measured)
Disk
3 GB container disk (10 GB at 4 vCPU) on the host's NVMe RAID
Kernel
Host kernel 6.17 (Ubuntu 24.04), Debian 13 userland
Chips in this track
9254 ×35, 9354P ×34, 9354 ×24, 9275F ×7, from the benchmark's fingerprints
Upstash BoxChipAWS Graviton4Arm Neoverse V2, 2.8 GHz; Graviton4 from the m8g instance type · inferredMemoryDDR5inferredWhereAWS EC2us-east-1, Ashburn, VirginiaRuns asDocker containeron a shared m8g.16xlarge VM; cgroups set its CPU and RAM
Hidden or different from the real machine
The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.
nproc and /proc/meminfo show the host's 64 CPUs and 256 GB, while a box may use 2 CPUs and 4 GB (4 and 8 at the medium size).
No DMI tables or clock readings inside, so memory type and clock come from AWS's published spec.
Neighbours share the host kernel and the chip's L3 cache.
Memory
DDR5-5600, from AWS's Graviton4 spec (inferred)
Disk
5 GB overlay root and a 5 GB /workspace, on EBS
Kernel
Host kernel 7.0 (Ubuntu 24.04 for AWS), Debian 12 userland
At 4 vCPU
The medium size (4 vCPU, 8 GB) runs on the same m8g.16xlarge hosts.
boxdChipAMD EPYC 9554P, likelyZen 4 (Genoa), 3.1 GHz · inferredMemoryDDR5inferredWhereLatitude.sh bare metalFrankfurt, GermanyRuns asFirecracker microVMwith Docker installed inside the VM
Hidden or different from the real machine
The CPU name is trimmed to "AMD EPYC" and the microcode and topology are hidden. The chip is inferred from its 3.1 GHz clock and the Genoa servers Latitude.sh sells; a dual-socket 9554 can't be ruled out.
No DMI, so the memory type is inferred.
AVX-512 is exposed, which can help vectorised work against hosts that hide it.
The disk reports itself as rotational although it is virtual, so tools that tune for hard drives may behave differently.
Memory
DDR5-4800, the Genoa platform's memory (inferred)
Disk
100 GB virtio disk, ext4
Kernel
Linux 6.18 (custom build), Ubuntu 24.04
At 4 vCPU
The 4 vCPU, 16 GB size ran on the same CPU type.
FreestyleChipAMD EPYC 7763, likelyZen 3 (Milan), 2.45 GHz · inferredMemoryDDR4inferredWhereFreestyle's own network (AS36320)San Francisco Bay AreaRuns asFirecracker microVM4 vCPU, 8 GB, no hyperthreads exposed
Hidden or different from the real machine
The CPU name, microcode and topology are hidden. The chip is inferred from its 2.45 GHz clock, cache and Zen 3 features.
Memory is added after boot: about 3 GB is free at first, 7.9 GB later.
No DMI, so the memory type is inferred.
Memory
DDR4-3200, the Milan platform's memory (inferred)
Disk
32 GB virtio disk, ext4
Kernel
Linux 6.1
TenkiChipAMD EPYC 7B13, as reportedZen 3 (Milan), 2.25 GHz. The 4 vCPU VM also showed EPYC 9575F (Zen 5, Turin) cores · conflictingMemoryNot knownNo memory data inside the VMWhereLuxor (AS19538)SeattleRuns asCloud Hypervisor microVMwith a container inside
Hidden or different from the real machine
On the 4 vCPU VM, two vCPUs report an EPYC 7B13 (Milan) and two an EPYC 9575F (Turin). That fits a snapshot taken on a Milan host and restored on a Turin one, so the real host may be newer than the name says.
CPU features are capped at the Milan set on every vCPU, so no AVX-512 even on Turin.
The clock reads 2.25 GHz, the 7B13's base, which may be carried over from the snapshot.
Memory
No memory data inside the VM
Disk
5 GB virtio disk, ext4
Kernel
Linux 6.18
At 4 vCPU
The 4 vCPU, 8 GB VM is the one that showed both chips.

Reliability

How often each provider failed, and why. Only failures the provider caused are counted; Method lists the ones we set aside.

Failed attempts

Per 1,000 attempts across the tests on this page · Lower is better

E2B0 of 1,076 attempts failed

Daytona0 of 973 attempts failed

Freestyle0 of 1,068 attempts failed

Superserve0 of 1,076 attempts failed

Tenki1 of 1,075 attempts failed

boxd1 of 1,073 attempts failed

Upstash Box3 of 1,070 attempts failed

0.0

E2B0 of 1,076 attempts failed

0.0

Daytona0 of 973 attempts failed

0.0

Freestyle0 of 1,068 attempts failed

0.0

Superserve0 of 1,076 attempts failed

0.9

Tenki1 of 1,075 attempts failed

0.9

boxd1 of 1,073 attempts failed

2.8

Upstash Box3 of 1,070 attempts failed

Every failure and its cause · 5 runs across 3 providers

Tenki
- 1× Tenki request failed (not_found). (not_found, 5, operation=snapshot)Take snapshot
boxd
- 1× boxd request failed. (provider)Write + read 10 MB
Upstash Box
- 1× Upstash Box exec failed (provider). (provider)Agent tool replay
- 1× Upstash Box files.write failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=files.write)npm install
- 1× Guest exit 143: Cloning into '/workspace/bench-clone'..git clone

Provider	Runs	Cause	Tests
Tenki	1	Tenki request failed (not_found). (not_found, 5, operation=snapshot)	Take snapshot
boxd	1	boxd request failed. (provider)	Write + read 10 MB
Upstash Box	1	Upstash Box exec failed (provider). (provider)	Agent tool replay
Upstash Box	1	Upstash Box files.write failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=files.write)	npm install
Upstash Box	1	Guest exit 143: Cloning into '/workspace/bench-clone'..	git clone

Failed attempts

Per 1,000 attempts across the tests on this page · Lower is better

E2B0 of 1,076 attempts failed

Daytona0 of 910 attempts failed

Tenki0 of 897 attempts failed

Freestyle1 of 1,073 attempts failed

Upstash Box1 of 1,062 attempts failed

boxd2 of 1,064 attempts failed

0.0

E2B0 of 1,076 attempts failed

0.0

Daytona0 of 910 attempts failed

0.0

Tenki0 of 897 attempts failed

0.9

Freestyle1 of 1,073 attempts failed

0.9

Upstash Box1 of 1,062 attempts failed

1.9

boxd2 of 1,064 attempts failed

Every failure and its cause · 4 runs across 3 providers

Freestyle
- 1× Preview HTTP 502, no first bytePreview URL first byte
Upstash Box
- 1× Upstash Box exec failed (provider). (provider)Cold start
boxd
- 1× boxd request failed. (invalid_argument)Pause + resume
- 1× boxd request failed. (provider)Write + read 10 MB

Provider	Runs	Cause	Tests
Freestyle	1	Preview HTTP 502, no first byte	Preview URL first byte
Upstash Box	1	Upstash Box exec failed (provider). (provider)	Cold start
boxd	1	boxd request failed. (invalid_argument)	Pause + resume
boxd	1	boxd request failed. (provider)	Write + read 10 MB

Cost

What the same agent task cost on each provider, and what each charges per hour.

Cost per 1,000 tasks

One task = create a sandbox, run 100 tool calls, delete it · US dollars · Lower is better

Upstash Box$0.23 per 1,000 tasksFrom Upstash Box's bill, 10 tasks

Daytona$0.24 per 1,000 tasksFrom Daytona's bill, 10 tasks

Superserve$0.25 per 1,000 tasksFrom Superserve's list price, 10 tasks

E2B$0.49 per 1,000 tasksFrom E2B's usage record, 10 tasks

Tenki$0.90 per 1,000 tasksFrom Tenki's usage record, 10 tasks

boxd$1.00 per 1,000 tasksFrom boxd's usage record, 10 tasks

Freestyle$1.71 per 1,000 tasksFrom Freestyle's list price, 10 tasks

$0.23

Upstash Box$0.23 per 1,000 tasksFrom Upstash Box's bill, 10 tasks

$0.24

Daytona$0.24 per 1,000 tasksFrom Daytona's bill, 10 tasks

$0.25

Superserve$0.25 per 1,000 tasksFrom Superserve's list price, 10 tasks

$0.49

E2B$0.49 per 1,000 tasksFrom E2B's usage record, 10 tasks

$0.90

Tenki$0.90 per 1,000 tasksFrom Tenki's usage record, 10 tasks

$1.00

boxd$1.00 per 1,000 tasksFrom boxd's usage record, 10 tasks

$1.71

Freestyle$1.71 per 1,000 tasksFrom Freestyle's list price, 10 tasks

Not from a bill yet. E2B, Tenki and boxd: their own usage records at published rates; Superserve and Freestyle: our measured time at list price.

Notes (3)

Each task ran in its own sandbox, up to 10 per provider, in a separate cost run. The bar is the median.
Usage charges only: credits, tax and plan minimums are left out.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Agent work vs cost

Time for 100 tool calls against cost per 1,000 tasks · Lower left is faster and cheaper

Faster and cheaper than the median

Cost per 1,000 tasks (USD)

Median time for 100 tool calls, seconds (log scale)

Not from a bill yet. E2B, Tenki and boxd: their own usage records at published rates; Superserve and Freestyle: our measured time at list price.

Notes (3)

Time is the median from the main tests. Cost is from the separate cost run. Hover or focus a dot for its numbers.
Boat and boxd run in Europe, so every call from our Florida server crosses the Atlantic.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.

Cost per 1,000 tasks

One task = create a sandbox, run 100 tool calls, delete it · US dollars · Lower is better

Upstash Box$0.47 per 1,000 tasksFrom Upstash Box's bill, 8 tasks

Daytona$1.19 per 1,000 tasksFrom Daytona's bill, 10 tasks

E2B$1.55 per 1,000 tasksFrom E2B's usage record, 10 tasks

Freestyle$1.78 per 1,000 tasksFrom Freestyle's list price, 10 tasks

Tenki$1.89 per 1,000 tasksFrom Tenki's usage record, 10 tasks

boxd$2.00 per 1,000 tasksFrom boxd's usage record, 10 tasks

$0.47

Upstash Box$0.47 per 1,000 tasksFrom Upstash Box's bill, 8 tasks

$1.19

Daytona$1.19 per 1,000 tasksFrom Daytona's bill, 10 tasks

$1.55

E2B$1.55 per 1,000 tasksFrom E2B's usage record, 10 tasks

$1.78

Freestyle$1.78 per 1,000 tasksFrom Freestyle's list price, 10 tasks

$1.89

Tenki$1.89 per 1,000 tasksFrom Tenki's usage record, 10 tasks

$2.00

boxd$2.00 per 1,000 tasksFrom boxd's usage record, 10 tasks

Not from a bill yet. E2B, Tenki and boxd: their own usage records at published rates; Freestyle: our measured time at list price.

Notes (5)

1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
Each task ran in its own sandbox, up to 10 per provider, in a separate cost run. The bar is the median.
Usage charges only: credits, tax and plan minimums are left out.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Agent work vs cost

Time for 100 tool calls against cost per 1,000 tasks · Lower left is faster and cheaper

Faster and cheaper than the median

Cost per 1,000 tasks (USD)

Median time for 100 tool calls, seconds (log scale)

Not from a bill yet. E2B, Tenki and boxd: their own usage records at published rates; Freestyle: our measured time at list price.

Notes (5)

Time is the median from the main tests. Cost is from the separate cost run. Hover or focus a dot for its numbers.
boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.
Boat and boxd run in Europe, so every call from our Florida server crosses the Atlantic.
Boat: coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that.
Superserve: not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does.

Method

How the numbers were made, and what could bias them.

Our SDKEvery provider ran through Sandbox SDK, which we make; version 0.2.0 is public on npm.
The 1.0 used here will be published soon. Each adapter calls the provider's official SDK or API, and uses its native path for commands and files where one exists.
One locationEvery test ran from one server in Florida, US.
Distance to each provider's region affects every call. The Hardware tab under Machines shows where each sandbox ran.
Two tracksDefault machines compares each provider's default sandbox; Same machine asks every provider for 4 vCPU and 8 GB.
Default sizes differ: E2B's default template is 2 vCPU and 512 MB, though its pricing page says 4 GB. boxd has no 4 vCPU, 8 GB size and ran 4 vCPU, 16 GB. Superserve is left out of Same machine because its account caps templates at 2 vCPU. Every chart marks a machine that differs from its track.
Cold startTimed from create() until a first command runs; our SDK's own setup is timed separately.
The command is a bare true. Setup after that point is shown as Ready through Sandbox SDK and is never part of cold start or bursts.
Runs100 runs per quick test, 30 per agent or snapshot test, and medians inside each other's 95% intervals count as tied.
Bursts run 3 times per size, each timed until every sandbox in it runs a command, and a size is published only when all three finished. Tenki's Same machine run stopped at 78 of 100 cold starts; each chart shows its attempt count. Default timings come from one run; snapshots, bursts, Same machine and cost come from a later run whose code changes touched only cleanup. Download speed is left out because the test host rate-limited several providers. Boat joins in the next update.
CostOne task per sandbox, up to 10 per provider, priced from each provider's bill wherever it gives one.
Upstash Box and Daytona give a per-sandbox charge; Daytona's can still change for 48 hours. E2B, boxd and Tenki give a usage record, priced at their published rates. Freestyle has no billing API and Superserve reports usage only per account, so those use our measured time at list price. Credits, tax and plan minimums are left out. The main test sandboxes are not priced, because providers ran different mixes of tests in them.
FailuresOnly failures the provider caused count against it; ones caused by us are set aside.
That includes failures we can't pin on either side, such as our SDK timing out while waiting. Attempts blocked by our own plan limits were rerun. Set aside:
- Default machines, boxd: 1 run, 21 tests. Our ownership check failed during one burst of 25 (16 creates, 5 deletes), so that burst is not counted.
- Default machines, Freestyle: 8 runs, 8 tests. Our SDK timed out or wrapped the error moving 10 MB, so we can't tell whose side failed.
- Default machines, Tenki: 1 run, 2 tests. Our cleanup check could not confirm the sandbox after Tenki's snapshot failed, so we can't tell whose side the delete failure was on.
- Same machine, Upstash Box: 3 runs, 5 tests. Our ownership check could not confirm the sandbox before deleting it, so we can't tell whose side failed.
- Same machine, Upstash Box: 3 runs, 4 tests. Our ownership check failed on 4 snapshot cleanup steps; none is counted.
- Same machine, Upstash Box: 3 runs, 3 tests. Our ownership check failed on 3 burst deletes; none is counted.
- Same machine, boxd: 12 runs, 12 tests. Our ownership check failed on 4 snapshot attempts, and our 30-second cleanup deadline ran out 8 times; none is counted.
- Same machine, boxd: 4 runs, 14 tests. Our ownership check failed on 1 burst create and 13 burst deletes; none is counted.
- Same machine, Freestyle: 3 runs, 3 tests. Our SDK timed out or wrapped the error moving 10 MB, so we can't tell whose side failed.
- Same machine, Tenki: 1 run, 1 test. The run was stopped during this attempt, so it is not counted.
Plan limitsWe used self-serve accounts, and a chart names any limit that changed its result.
Superserve caps templates at 2 vCPU, so it is not in Same machine. Boat allows 200 creates a day. Daytona's 10 vCPU quota stopped its 4 vCPU snapshot and burst tests. We will add results if a paid plan or quota increase lifts these.
SuperserveSuperserve ran a custom template we built with Node 22 and Python 3.11.
Its stock templates offer one or the other, and our tests need both. Before this template, our setup installed Python on every create.
IndependentNo provider paid for these results or decided how they were tested.
Some may have seen an early copy to check for errors. When a provider pointed out a mistake of ours, we fixed it the same way for every provider. We may work with some of these companies; that did not change how tests ran or how results are reported.

Every number with its full method: results.md

Cloud sandboxes, measured

Cold start→

Agent work, 100 tool calls→

Cost per 1,000 tasks→

Start from snapshot→

Cold start→

Agent work, 100 tool calls→

Cost per 1,000 tasks→

Start from snapshot→

Where the time goes

Start-up

Cold start

Ready through Sandbox SDK

Every cold start

Cold start

Ready through Sandbox SDK

Every cold start

Agent work

Agent tool replay

npm install

pip install

git clone

Dev server ready

Preview URL first byte

Edit, rebuild, reload

Agent tool replay

npm install

pip install

git clone

Dev server ready

Preview URL first byte

Edit, rebuild, reload

Everyday calls

Run a command

Write + read 1 KB

Write + read 10 MB

Reconnect

Destroy

Run a command

Write + read 1 KB

Write + read 10 MB

Reconnect

Destroy

Pause & snapshots

Resume

Pause

Pause + resume

Take snapshot

Start from snapshot

Resume

Pause

Pause + resume

Take snapshot

Start from snapshot

Bursts

10 at once, all ready

25 at once, all ready

10 at once, all ready

25 at once, all ready

Machines

What each provider ran

Default machine size

vCPU

RAM

Disk

Hardware inside each sandbox

CPU, one core

CPU, all cores

Memory bandwidth

Disk write

Disk read

4K random write

4K random read

What each provider ran

Hardware inside each sandbox

CPU, one core

CPU, all cores

Memory bandwidth

Disk write

Disk read

Cold start

Agent work, 100 tool calls

Cost per 1,000 tasks

Start from snapshot

Cold start

Agent work, 100 tool calls

Cost per 1,000 tasks

Start from snapshot