Sandbox bench

Cloud sandboxes, measured

Oct 22–26, 2026 · 8 providers (9 configs) · 2 caller locations: Florida, US & Frankfurt, DE · 100 runs per main test, 20 per location test

Caller locations:All locations (EU-Central vs US-East)EU-Central · Frankfurt, DEUS-East · Florida, USUS-West · soon

Oct 22–26, 2026 · 8 providers (9 configs) · 2 caller locations: Florida, US & Frankfurt, DE · 100 runs per main test, 20 per location test

Caller locations:All locations (EU-Central vs US-East)EU-Central · Frankfurt, DEUS-East · Florida, USUS-West · soon

Upstash appears twice: Box can pause; Ephemeral Box starts in 0.68 s but can't pause.

Upstash appears twice: Box can pause; Ephemeral Box starts in 0.68 s but can't pause.

By location

How network distance changes latency: the same tests run from EU-Central (Frankfurt) and US-East (Florida).

Cold start by caller location

Median time from each caller location · Colored bar marks the faster location · Lower is better

Oct 26, 2026 · 5–20 runs per test

E2BSandbox in Oregon
EU-Central
0.67 s
US-East
0.48 s
×1.4 faster from US-East
DaytonaAPI in Virginia
EU-Central
1.5 s
US-East
0.99 s
×1.5 faster from US-East
Upstash BoxSandbox in Virginia
EU-Central
3.0 s
US-East
2.5 s
×1.2 faster from US-East
BoatSandbox in Northern France
EU-Central
0.78 s
US-East
1.1 s
×1.5 faster from EU-Central
boxdSandbox in Frankfurt
EU-Central
1.6 s
US-East
1.0 s
×1.6 faster from US-East
FreestyleSandbox in SF Bay Area
EU-Central
1.1 s
US-East
0.81 s
×1.3 faster from US-East
TenkiSandbox in Seattle
EU-Central
1.1 s
US-East
0.91 s
×1.2 faster from US-East
SuperserveSandbox in Virginia
EU-Central
0.71 s
US-East
0.49 s
×1.5 faster from US-East

Run a command

fastest location
E2B
0.12 sUS-East×1.5
Daytona
0.10 sUS-East×1.6
Upstash Box
0.22 sUS-East×2.5
Boat
0.11 sEU-Central×2.9
boxd
0.17 sEU-Central×1.7
Freestyle
0.16 sUS-East×1.6
Tenki
0.17 sUS-East×1.4
Superserve
0.15 sUS-East×1.6

Write + read 10 MB

fastest location
E2B
1.3 sUS-East×2.1
Daytona
1.8 sUS-East×1.1
Upstash Box
2.1 sUS-East×1.9
Boat
3.5 sEU-Central×2.0
boxd
0.74 sEU-Central×3.9
Freestyle
2.1 sUS-East×2.3
Tenki
2.0 sUS-East×1.5
Superserve
0.61 sUS-East×3.6

Agent tool replay

fastest location
E2B
14 sUS-East×1.6
Daytona
17 sUS-East×1.3
Upstash Box
21 sUS-East×1.8
Boat
23 sEU-Central×1.8
boxd
17 sEU-Central×1.8
Freestyle
19 sUS-East×1.7
Tenki
15 sUS-East×1.6
Superserve
11 sUS-East×2.0

Boat (Northern France) and boxd (Frankfurt) run their sandboxes in Europe; the remaining providers run in US-East. In-sandbox CPU, memory, disk, and package install speed do not change with caller location and are omitted here.

Cold start

Median s · lower is better

  • 1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.

Agent work, 100 tool calls

Median s · lower is better

  • 1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.

Cost per 1,000 tasks

USD · lower is better

  • 1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.

Start from snapshot

Median s · lower is better

  • 1boxd: ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.

Cold start

Median s · lower is better

Agent work, 100 tool calls

Median s · lower is better

Cost per 1,000 tasks

USD · lower is better

Start from snapshot

Median s · lower is better

Where the time goes

Step by step, from median timings, in each provider's own time.

TraceCold start · every provider, median of each step
Fastest
519ms
Superserve · provider time
Slowest
2.62s
Upstash Box · provider time
Spread
5.1×
slowest vs fastest
Failures
0
of 900 attempts
Provider
0s1s2s3s
create()First commandRanked by provider time · pick a provider for its full trace.
TraceCold start · every provider, median of each step
Fastest
495ms
E2B · provider time
Slowest
2.62s
Upstash Box · provider time
Spread
5.3×
slowest vs fastest
Failures
0
of 900 attempts
Provider
0s1s2s3s
create()First commandOurs: Sandbox SDKRanked by provider time · pick a provider for its full trace.

Start-up

Cold start is the wait before an agent can do anything: from create() until the first command runs, the provider's own time.

Cold start

create() to first command, without our SDK's setup · Median seconds · Lower is better

Cold start

create() to first command, without our SDK's setup · Median seconds · Lower is better

Agent work

Real coding-agent tasks, so CPU, disk and network count as much as the API.

Agent tool replay

100 tool calls in a row: write, read, edit, list, grep, run a test · Median seconds · Lower is better

Agent tool replay

100 tool calls in a row: write, read, edit, list, grep, run a test · Median seconds · Lower is better

Everyday calls

What an agent does hundreds of times per task: run a command, move files, reattach to a sandbox.

Run a command

A no-op command, mostly network and API overhead · Median seconds · Lower is better

Run a command

A no-op command, mostly network and API overhead · Median seconds · Lower is better

Pause & snapshots

Pausing stops paying for idle time while an agent waits on a human, with every process kept. Where stop and start restarts the sandbox instead, we mark pause as not supported. Snapshots start many sandboxes from one prepared state.

Resume

Resume a paused sandbox until a command runs · Median seconds · Lower is better

Resume

Resume a paused sandbox until a command runs · Median seconds · Lower is better

Bursts

10 or 25 sandboxes started at once, as an agent fanning out would, timed until the last one runs a command.

10 at once, all ready

10 creates at once, until all 10 run a command · Median seconds · Lower is better

10 at once, all ready

10 creates at once, until all 10 run a command · Median seconds · Lower is better

Machines

What each provider ran: what we asked for, what we measured inside, and the hardware underneath.

What each provider ran

Asked for 4 vCPU, 8 GB; measured inside each sandbox

  • E2BMatches

    Asked for 4 vCPU, 8 GB, template r1h7gud12rf1f6iu29r0

    Measured 4 vCPU, 7.8 GB

  • FreestyleMatches

    Asked for 4 vCPU, 8 GB

    Measured 4 vCPU, 7.8 GB

  • TenkiMatches

    Asked for 4 vCPU, 8 GB

    Measured 4 vCPU, 7.8 GB

  • boxdDiffers

    Asked for 4 vCPU, 16 GB

    Measured 4 vCPU, 16 GB

    Ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.

  • SuperserveMatches

    Asked for 4 vCPU, 8 GB, template sandbox-bench-node22-py311-4x8-pinned

    Measured 4 vCPU, 7.8 GB

  • Upstash BoxMatches

    Asked for 4 vCPU, 8 GB ("medium")

    Measured 4 vCPU, 8 GB

  • DaytonaMatches

    Asked for 4 vCPU, 8 GB, template daytona-large

    Measured 4 vCPU, 8 GB

  • Upstash Ephemeral BoxMatches

    Asked for 4 vCPU, 8 GB ("medium")

    Measured 4 vCPU, 8 GB

  • BoatMatches

    Asked for 4 vCPU, 8 GB

    Measured 4 vCPU, 7.8 GB

What each provider ran

Each provider's default sandbox, and the vCPU and RAM measured inside it

  • E2B

    Asked for 2 vCPU, 512 MB, template base

    Measured 2 vCPU, 473 MB

    E2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.

  • Freestyle

    Asked for 4 vCPU, 8 GB

    Measured 4 vCPU, 7.8 GB

  • Tenki

    Asked for provider default

    Measured 2 vCPU, 3.8 GB

    Tenki documents a 5 GiB default disk but not a default vCPU or RAM, so those are what we measured.

  • boxd

    Asked for 2 vCPU, 8 GB

    Measured 2 vCPU, 7.8 GB

  • Superserve

    Asked for 1 vCPU, 1 GB, template sandbox-bench-node22-py311-1x1-pinned

    Measured 1 vCPU, 994 MB

    Superserve ran a template we built with Node 22 and Python 3.11 preinstalled, at its default 1 vCPU, 1 GB RAM and 4 GB disk. Superserve's own templates have Node (node-22) or Python (python-3.11), not both, and our tests need both.

  • Upstash Box

    Asked for 2 vCPU, 4 GB

    Measured 2 vCPU, 4 GB

  • Daytona

    Asked for 1 vCPU, 1 GB

    Measured 1 vCPU, 1 GB

  • Upstash Ephemeral Box

    Asked for 2 vCPU, 4 GB

    Measured 2 vCPU, 4 GB

  • Boat

    Asked for 4 vCPU, 8 GB

    Measured 4 vCPU, 7.8 GB

Reliability

How often each provider failed, and why. Only failures the provider caused are counted; Method lists the ones we set aside.

Failed attempts

Per 1,000 attempts across the tests on this page · Lower is better

Failed attempts

Per 1,000 attempts across the tests on this page · Lower is better

Cost

What the same agent task cost on each provider, and what each charges per hour.

Cost per 1,000 tasks

One task = create a sandbox, run 100 tool calls, delete it · US dollars · Lower is better

Not from a bill yet. E2B and boxd: their own usage records at published rates; Boat, Tenki and Freestyle: our measured time at list price.

Agent work vs cost

Time for 100 tool calls against cost per 1,000 tasks · Lower left is faster and cheaper

Faster and cheaper than the median
Cost per 1,000 tasks (USD)

Median time for 100 tool calls, seconds (log scale)

Not from a bill yet. E2B and boxd: their own usage records at published rates; Boat, Tenki and Freestyle: our measured time at list price.

Cost per 1,000 tasks

One task = create a sandbox, run 100 tool calls, delete it · US dollars · Lower is better

Not from a bill yet. E2B and boxd: their own usage records at published rates; Boat, Tenki and Freestyle: our measured time at list price.

Agent work vs cost

Time for 100 tool calls against cost per 1,000 tasks · Lower left is faster and cheaper

Faster and cheaper than the median
Cost per 1,000 tasks (USD)

Median time for 100 tool calls, seconds (log scale)

Not from a bill yet. E2B and boxd: their own usage records at published rates; Boat, Tenki and Freestyle: our measured time at list price.

Method

How the numbers were made, and what could bias them.

Every number with its full method: results.md