{"about":"Sandbox bench: cloud sandbox providers for AI agents, timed through one SDK (Sandbox SDK, built by us; 0.2.0 is public on npm and the 1.0 used here will be published soon) from one server in Florida, US, on two tracks: the same 4 vCPU, 8 GB machine for every provider (the fair comparison, and the one the site opens on), and each provider's default machine. Medians, p95s and 95% confidence intervals per provider; every provider-side failure kept, failures caused by our own code set aside.","howToRead":"Pick a track first: `tracks[\"same-spec\"]` asks every provider for 4 vCPU and 8 GB, so prefer it for comparisons; `tracks.default` compares each provider's default sandbox, so sizes differ. In same-spec, `allocation.equalSpec: false` means the machine measured inside was not 4 vCPU, 8 GB, so compare that provider with care. median and p95 are in `unit`; `better` says which direction wins; results are sorted best first. attempts counts every try, failed included. A null median means the provider does not support the test, every attempt failed, or the result is excluded (see `status`). Cold start is the provider's own time, create() until the first command; Our SDK's setup after create is not included; on the faster SDK it measured 0.1 to 0.3 ms median per provider (20 creates each, separate run). `costPerTask.kind` is billed (provider charge), metered (provider usage at published rates) or estimate (list price, where billing gives no amount); a task is one sandbox that is created, runs the 100-call tool replay and is deleted, from a separate cost run: never compare them as if they were the same kind without saying so.","task":"A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.","commonSize":"4 vCPU, 8 GB","comingNextUpdate":["Snapshot and restore","Bursts of 10 and 25","The same-machine (4 vCPU, 8 GB) track"],"tracks":{"same-spec":{"id":"same-spec","label":"Same machine: 4 vCPU, 8 GB","description":"Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size.","published":true,"run":{"source":"m","start":"2026-10-22T18:05:53.912Z","end":"2026-10-23T23:43:51.006Z","runs":["sbx53-same-spec-burst-r8b2","sbx53-same-spec-burst-r8c2","sbx53-same-spec-burst-r8u","sbx53-same-spec-burst25-r8u","sbx53-same-spec-cost-r8b2","sbx53-same-spec-cost-r8c2","sbx53-same-spec-cost-r8c-superserve","sbx53-same-spec-cost-r8u","sbx53-same-spec-main-r8b3","sbx53-same-spec-main-r8c2","sbx53-same-spec-main-r8u","sbx53-same-spec-snapshot-r8b2","sbx53-same-spec-snapshot-r8c2","sbx53-same-spec-snapshot-r8u","sbx53-same-spec-burst-r8y-1","sbx53-same-spec-burst-r8y-2","sbx53-same-spec-burst25-r8y-1","sbx53-same-spec-burst25-r8y-2","sbx53-same-spec-burst25-r8y-3","sbx53-same-spec-burst-r8y-3","sbx53-same-spec-cost-r8y","sbx53-same-spec-main-r8y","sbx53-same-spec-snapshot-r8y","sbx53-same-spec-main-r8e","sbx53-same-spec-snapshot-r8e","sbx53-same-spec-cost-r8e","sbx53-same-spec-burst-r8e","sbx53-same-spec-main-r8boat2","sbx53-same-spec-cost-r8boat2","sbx53-same-spec-burst-r8boat2-1","sbx53-same-spec-burst25-r8boat2-1","sbx53-same-spec-burst-r8boat2-2","sbx53-same-spec-burst25-r8boat2-2","sbx53-same-spec-burst-r8boat2-3","sbx53-same-spec-burst25-r8boat2-3"],"observations":15576},"absent":[],"providers":[{"id":"e2b","name":"E2B","allocation":{"requested":{"size":null,"template":"r1h7gud12rf1f6iu29r0","compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8},"text":"4 vCPU, 8 GB, template r1h7gud12rf1f6iu29r0"},"measured":[{"cpu":4,"memoryGiB":7.77,"diskGiB":21.535}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"The Dalles, US","cpu":"Intel Xeon Processor @ 2.60GHz","hardware":{"chip":"Intel Xeon Platinum 8373C","family":"Ice Lake, 2.6 GHz, 54 MB L3","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the chip's memory type"},"cloud":"Google Cloud","region":"us-west1, The Dalles, Oregon","runsAs":"Firecracker microVM, nested inside a Google Cloud VM","hidden":["The CPU name, microcode and clock leaves are hidden. The chip is identified from its model number, cache size and 2.6 GHz clock.","The default 2 vCPUs are one physical core with two hyperthreads, so two-thread work scales well below twice one thread.","No DMI or PCI devices, so the memory type can't be read inside; DDR4 follows from the chip."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"metered","perTaskUsd":0.001565,"per1000TasksUsd":1.565,"confidence":"medium","tasks":10,"source":"https://api.e2b.app/events/sandboxes/{id}","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"10 GiB Hobby / 20 GiB Pro included disk. Subscription excluded. Per-second billing; no separate creation fee listed.","usdNote":null,"source":"https://e2b.dev/pricing"},"pause":{"supported":true,"split":{"pause":1.2916321539999918,"resume":0.5057368029998616,"n":100}},"reliability":{"attempts":1073,"succeeded":1073,"successRate":1},"failures":[],"setAside":[]},{"id":"freestyle","name":"Freestyle","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":32},"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.755,"diskGiB":31.201}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":null,"cpu":null,"hardware":{"chip":"AMD EPYC 7763, likely","family":"Zen 3 (Milan), 2.45 GHz","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the Milan platform's memory"},"cloud":"Freestyle's own network (AS36320)","region":"San Francisco Bay Area","runsAs":"Firecracker microVM, 4 vCPU, 8 GB, no hyperthreads exposed","hidden":["The CPU name, microcode and topology are hidden. The chip is inferred from its 2.45 GHz clock, cache and Zen 3 features.","Memory is added after boot: about 3 GB is free at first, 7.9 GB later.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"estimate","perTaskUsd":0.001967,"per1000TasksUsd":1.967,"confidence":"low","tasks":10,"source":"https://www.freestyle.sh/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.264,"perVcpuHour":0.066,"perHourUsd":0.264,"commonSizeUsd":0.264,"note":"Published overage rates, allowances/subscriptions excluded. $0.02/GB transfer excluded. Billing increment/minimum and creation fees unspecified; one-second estimate.","usdNote":null,"source":"https://www.freestyle.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.3201344599998556,"resume":0.34602604700019585,"n":100}},"reliability":{"attempts":1071,"succeeded":1070,"successRate":0.99907},"failures":[{"kind":"provider-operation","reason":"Preview HTTP 502, no first byte","count":1,"runs":1,"tests":["Preview URL first byte"]}],"setAside":[{"reason":"Our SDK timed out or wrapped the error moving 10 MB, so we can't tell whose side failed","count":5,"runs":5}]},{"id":"tenki","name":"Tenki","allocation":{"requested":{"size":{"cpu":4,"memoryMb":8192},"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":null,"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.797,"diskGiB":4.84}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":null,"cpu":"AMD EPYC 7B13 64-Core Processor","hardware":{"chip":"AMD EPYC 7B13, as reported","family":"Zen 3 (Milan), 2.25 GHz. The 4 vCPU VM also showed EPYC 9575F (Zen 5, Turin) cores","chipBasis":"conflicting","memory":{"type":null,"note":"No memory data inside the VM"},"cloud":"Luxor (AS19538)","region":"Seattle","runsAs":"Cloud Hypervisor microVM, with a container inside","hidden":["On the 4 vCPU VM, two vCPUs report an EPYC 7B13 (Milan) and two an EPYC 9575F (Turin). That fits a snapshot taken on a Milan host and restored on a Turin one, so the real host may be newer than the name says.","CPU features are capped at the Milan set on every vCPU, so no AVX-512 even on Turin.","The clock reads 2.25 GHz, the 7B13's base, which may be carried over from the snapshot."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"estimate","perTaskUsd":0.001794,"per1000TasksUsd":1.794,"confidence":"low","tasks":10,"source":"https://tenki.cloud/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"Per-second, no minimum. Separate persistent storage $0.000000076/GiB-second excluded. Credits excluded.","usdNote":null,"source":"https://tenki.cloud/pricing"},"pause":{"supported":true,"split":{"pause":60.53191615900025,"resume":2.449111862000078,"n":77}},"reliability":{"attempts":1053,"succeeded":1053,"successRate":1},"failures":[],"setAside":[]},{"id":"boxd","name":"boxd","allocation":{"requested":{"size":{"cpu":4,"memoryMb":16384},"template":null,"compute":{"cpu":4,"memoryGiB":16},"provisioned":null,"text":"4 vCPU, 16 GB"},"measured":[{"cpu":4,"memoryGiB":15.64,"diskGiB":97.872}],"measuredText":"4 vCPU, 16 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":16}},"allocationNote":"ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.","advertisedDefault":null,"ranIn":"Frankfurt am Main, DE","cpu":"AMD EPYC","hardware":{"chip":"AMD EPYC 9554P, likely","family":"Zen 4 (Genoa), 3.1 GHz","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-4800, the Genoa platform's memory"},"cloud":"Latitude.sh bare metal","region":"Frankfurt, Germany","runsAs":"Firecracker microVM, with Docker installed inside the VM","hidden":["The CPU name is trimmed to \"AMD EPYC\" and the microcode and topology are hidden. The chip is inferred from its 3.1 GHz clock and the Genoa servers Latitude.sh sells; a dual-socket 9554 can't be ruled out.","No DMI, so the memory type is inferred.","AVX-512 is exposed, which can help vectorised work against hosts that hide it.","The disk reports itself as rotational although it is virtual, so tools that tune for hard drives may behave differently."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"metered","perTaskUsd":0.001999,"per1000TasksUsd":1.999,"confidence":"medium","tasks":10,"source":"boxd orgs.usage / orgs.billingActivityPage","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"EUR","perHour":0.348,"perVcpuHour":0.087,"perHourUsd":0.391,"commonSizeUsd":null,"note":"EUR; RAM in use and written disk billed, so allocated resources are an upper bound. Metered per second, settled every 15 minutes (not a rounding increment). FX budget uses conservative 2 USD/EUR, not a market quote.","usdNote":"Converted at the ECB rate of 1.1234 USD per EUR on October 21, 2026.","source":"https://boxd.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.12380898099998013,"resume":0.4039313779999502,"n":99}},"reliability":{"attempts":1068,"succeeded":1065,"successRate":0.99719},"failures":[{"kind":"provider-operation","reason":"boxd request failed. (provider)","count":2,"runs":2,"tests":["Write + read 10 MB"]},{"kind":"provider-operation","reason":"boxd request failed. (invalid_argument)","count":1,"runs":1,"tests":["Pause + resume"]}],"setAside":[]},{"id":"superserve","name":"Superserve","allocation":{"requested":{"size":null,"template":"sandbox-bench-node22-py311-4x8-pinned","compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":4},"text":"4 vCPU, 8 GB, template sandbox-bench-node22-py311-4x8-pinned"},"measured":[{"cpu":4,"memoryGiB":7.798,"diskGiB":3.859}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"Washington, US","cpu":"Intel Xeon Processor","hardware":{"chip":"Intel Xeon, Sapphire Rapids","family":"Likely Xeon Platinum 8481C; 2.2 GHz, 105 MB L3","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5, the Sapphire Rapids platform's memory"},"cloud":"Google Cloud","region":"us-east4, Northern Virginia, from its IP range","runsAs":"Firecracker microVM, likely inside a Google Cloud VM","hidden":["The CPU name and microcode are hidden. The chip family is clear from its model number; the exact SKU is not.","The old 4.14 guest kernel can't turn on the chip's AMX units, so AMX workloads can't use them.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.001325,"per1000TasksUsd":1.325,"confidence":"high","tasks":10,"source":"https://api.superserve.ai/billing/usage-series (millisecond range) + /activity","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"Published vCPU and GiB-hour rates converted to seconds. Credits excluded. One-second rounding assumed; snapshot storage, disk, transfer and any minimum charge excluded. Adapter default superserve/node-22 is 1 vCPU, 1 GiB RAM, 4 GiB disk; resources are template-owned. SDK runtime preparation is included in cold timing.","usdNote":null,"source":"https://www.superserve.ai/pricing"},"pause":{"supported":true,"split":{"pause":0.3940402660002001,"resume":0.5288790809999918,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]},{"id":"upstash","name":"Upstash Box","allocation":{"requested":{"size":"medium","template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":null,"text":"4 vCPU, 8 GB (\"medium\")"},"measured":[{"cpu":4,"memoryGiB":8,"diskGiB":10}],"measuredText":"4 vCPU, 8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"Ashburn, US","cpu":null,"hardware":{"chip":"AWS Graviton4","family":"Arm Neoverse V2, 2.8 GHz; Graviton4 from the m8g instance type","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-5600, from AWS's Graviton4 spec"},"cloud":"AWS EC2","region":"us-east-1, Ashburn, Virginia","runsAs":"Docker container, on a shared m8g.16xlarge VM; cgroups set its CPU and RAM","hidden":["The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.","nproc and /proc/meminfo show the host's 64 CPUs and 256 GB, while a box may use 2 CPUs and 4 GB (4 and 8 at the medium size).","No DMI tables or clock readings inside, so memory type and clock come from AWS's published spec.","Neighbours share the host kernel and the chip's L3 cache."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.0003371,"per1000TasksUsd":0.3371,"confidence":"high","tasks":10,"source":"https://us-east-1.box.upstash.com/v2/box/{id}","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":0.2,"perVcpuHour":0.05,"perHourUsd":0.2,"commonSizeUsd":0.2,"note":"PAYG ACTIVE CPU billing: Small $0.10, Medium $0.20, Large $0.40 per active box-hour; tier price × wall time is an upper-bound estimate, not metered usage. Disk allocated capacity is an upper bound for written data. 30-day month. No RAM fee listed. Increment/minimum unspecified.","usdNote":null,"source":"https://upstash.com/pricing/box"},"pause":{"supported":true,"split":{"pause":0.8366179980000016,"resume":0.5705873199999333,"n":99}},"reliability":{"attempts":1075,"succeeded":1075,"successRate":1},"failures":[],"setAside":[]},{"id":"daytona","name":"Daytona","allocation":{"requested":{"size":null,"template":"daytona-large","compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":10},"text":"4 vCPU, 8 GB, template daytona-large"},"measured":[{"cpu":4,"memoryGiB":8,"diskGiB":10}],"measuredText":"4 vCPU, 8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":null,"cpu":"AMD EPYC 9254 24-Core Processor","hardware":{"chip":"AMD EPYC 9354P, 9354, 9254 or 9275F","family":"Zen 4 (Genoa); the 9275F is Zen 5 (Turin). Hosts vary","chipBasis":"named by the CPU","memory":{"type":"DDR5","basis":"measured","detail":"Registered DDR5, read from the memory controller on a 9254 host"},"cloud":"Bare-metal servers","region":"Location not visible; control plane on AWS us-east-1","runsAs":"Docker container, shares the host's kernel; cgroups set its CPU and RAM","hidden":["Nothing at the CPU level is hidden: the real CPU name, microcode, server model (Dell PowerEdge R6615 in the probe, a Supermicro AS-2115GT-HNTR in the benchmark) and disks are visible.","nproc and lscpu show every host thread (48 or 64) and /proc/meminfo shows the host's RAM, while the default sandbox may use 1 CPU and 1 GB. Tools that size thread pools or caches from those numbers oversubscribe.","Hosts differ in clock speed, so single-thread results vary from sandbox to sandbox. Neighbours share the host kernel.","Outbound internet is blocked on our account tier, so the sandbox's own network location could not be read."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.00136,"per1000TasksUsd":1.36,"confidence":"low","tasks":10,"source":"https://analytics.app.daytona.io/organization/{org}/usage/sandbox","basis":"Billed over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"Hourly published rates converted to seconds. Assumed one-second increment; minimum and creation fee not stated. Snapshot storage excluded.","usdNote":null,"source":"https://www.daytona.io/pricing"},"pause":{"supported":false,"note":"Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process."},"reliability":{"attempts":976,"succeeded":976,"successRate":1},"failures":[],"setAside":[]},{"id":"upstash-ephemeral","name":"Upstash Ephemeral Box","allocation":{"requested":{"size":"medium","template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":null,"text":"4 vCPU, 8 GB (\"medium\")"},"measured":[{"cpu":4,"memoryGiB":8,"diskGiB":20}],"measuredText":"4 vCPU, 8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"Ashburn, US","cpu":null,"hardware":{"chip":"AWS Graviton4","family":"Arm Neoverse V2; Graviton4 from the m8gd instance type","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-5600, from AWS's Graviton4 spec"},"cloud":"AWS EC2","region":"us-east-1, Ashburn, Virginia","runsAs":"Container, Ephemeral Box, on a shared AWS m8gd.12xlarge host","hidden":["The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.","Our deeper hardware probe ran on Upstash's regular Box. For Ephemeral Box we have the benchmark's own fingerprints: the AWS host type, the shared host kernel and the Arm architecture.","Memory type and clock come from AWS's published Graviton4 spec, not from inside the box."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.0002554,"per1000TasksUsd":0.2554,"confidence":"high","tasks":10,"source":"https://us-east-1.box.upstash.com/v2/box/{id}","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":0.2,"perVcpuHour":0.05,"perHourUsd":0.2,"commonSizeUsd":0.2,"note":"PAYG ACTIVE CPU billing: Small $0.10, Medium $0.20, Large $0.40 per active box-hour; tier price × wall time is an upper-bound estimate, not metered usage. Disk allocated capacity is an upper bound for written data. 30-day month. No RAM fee listed. Increment/minimum unspecified.","usdNote":null,"source":"https://upstash.com/pricing/box"},"pause":{"supported":false,"split":null},"reliability":{"attempts":976,"succeeded":976,"successRate":1},"failures":[],"setAside":[{"reason":"Our own scenario deadline ran out; not counted","count":3,"runs":3}]},{"id":"boat","name":"Boat","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"documentedDisk":{"value":50,"unit":"GB","scope":"User files; excludes operating system and pre-installed tools","source":"https://docs.boat.dev/machines"}},"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578},{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578},{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"Lille, FR","cpu":"AMD Ryzen 9 9950X 16-Core Processor","hardware":{"chip":"AMD Ryzen 9 9950X","family":"Zen 5 (Granite Ridge), 4.3 GHz base, up to 5.7 GHz","chipBasis":"named by the CPU","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5, the only memory the chip takes"},"cloud":"OVHcloud dedicated servers","region":"Northern France","runsAs":"QEMU/KVM virtual machine, on a desktop-class chip","hidden":["A desktop chip with a high boost clock, so single-thread results depend on boost and neighbours.","The cache sizes the guest sees are synthetic and differ from the real chip."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"estimate","perTaskUsd":0.00052,"per1000TasksUsd":0.52,"confidence":"low","tasks":10,"source":"https://docs.boat.dev/machines","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.036,"perVcpuHour":0.009,"perHourUsd":0.036,"commonSizeUsd":0.036,"note":"Default VM $0.036/hour (4 vCPU, 8 GB, 50 GB). Flat rate for each size; Small, Large and Xlarge are also offered. $20 monthly minimum/packs excluded; billed per second.","usdNote":null,"source":"https://docs.boat.dev/machines"},"pause":{"supported":false,"note":"Boat has no pause: stop and start restarts the machine and ends every running process."},"reliability":{"attempts":915,"succeeded":915,"successRate":1},"failures":[],"setAside":[{"reason":"Our own scenario deadline ran out; not counted","count":1,"runs":1}]}],"quickPicks":[{"need":"Fastest cold start","pick":"Superserve 0.52 s","runnerUp":"Upstash Ephemeral Box 0.68 s"},{"need":"Fastest agent work (tool replay)","pick":"Daytona 12 s","runnerUp":"Superserve 13 s"},{"need":"Fastest 25-sandbox fan-out","pick":"Superserve 0.79 s","runnerUp":"E2B 1.1 s"},{"need":"Fastest reconnect","pick":"Daytona 0.30 s","runnerUp":"Superserve 0.33 s"},{"need":"Cheapest task","pick":"Upstash Ephemeral Box $0.00026 per task, $0.26 per 1,000 (billed)","runnerUp":"Upstash Box $0.00034 per task, $0.34 per 1,000 (billed)"},{"need":"Most reliable","pick":"E2B, Tenki, Superserve, Upstash Box, Daytona, Upstash Ephemeral Box, Boat 100.00% (tied)","runnerUp":"Freestyle 99.91%"}],"metrics":[{"id":"cold","group":"Start-up","label":"Cold start","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.52,"p95":0.636,"ci95":[0.518,0.524],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.684,"p95":0.792,"ci95":[0.678,0.688],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.693,"p95":1.961,"ci95":[0.68,0.722],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.793,"p95":2.219,"ci95":[0.715,0.821],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.089,"p95":3.291,"ci95":[0.942,1.165],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":1.125,"p95":1.648,"ci95":[1.037,1.172],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.152,"p95":1.34,"ci95":[1.145,1.164],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":1.25,"p95":2.091,"ci95":[1.132,1.334],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":2.684,"p95":10.98,"ci95":[2.624,2.717],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"tool-replay","group":"Agent work","label":"Agent tool replay","unit":"s","better":"lower","results":[{"provider":"daytona","median":12.274,"p95":30.118,"ci95":[12.011,13.046],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":13.147,"p95":14.572,"ci95":[13.035,13.27],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":15.468,"p95":17.471,"ci95":[15.236,16.241],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":20.953,"p95":30.956,"ci95":[19.43,21.824],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":22.4,"p95":22.851,"ci95":[22.318,22.546],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":24.072,"p95":41.703,"ci95":[21.983,28.817],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":26.886,"p95":32.184,"ci95":[26.489,27.855],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":33.537,"p95":40.141,"ci95":[33.258,33.822],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":43.256,"p95":48.625,"ci95":[42.911,44.293],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"npm-install","group":"Agent work","label":"npm install","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.747,"p95":0.908,"ci95":[0.727,0.772],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":0.848,"p95":1.864,"ci95":[0.84,0.858],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.997,"p95":1.039,"ci95":[0.992,1.001],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":1.077,"p95":1.153,"ci95":[1.07,1.094],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":1.097,"p95":1.817,"ci95":[0.987,1.441],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.099,"p95":1.842,"ci95":[1.022,1.431],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.425,"p95":1.682,"ci95":[1.327,1.523],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.712,"p95":3.482,"ci95":[1.625,1.857],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.769,"p95":2.891,"ci95":[1.668,1.862],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"pip-install","group":"Agent work","label":"pip install","unit":"s","better":"lower","results":[{"provider":"tenki","median":3.326,"p95":5.003,"ci95":[3.141,4.481],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":3.765,"p95":4.553,"ci95":[3.698,4.115],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":3.948,"p95":8.961,"ci95":[3.856,4.416],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":4.042,"p95":5.01,"ci95":[3.986,4.127],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":4.222,"p95":5.538,"ci95":[4.048,4.347],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":4.539,"p95":5.006,"ci95":[4.49,4.546],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":4.866,"p95":8.172,"ci95":[4.219,5.046],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":5.638,"p95":6.381,"ci95":[5.218,5.861],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":7.509,"p95":20.298,"ci95":[7.394,8.385],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"git-clone","group":"Agent work","label":"git clone","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.433,"p95":1.042,"ci95":[0.417,0.451],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.615,"p95":0.672,"ci95":[0.596,0.631],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.69,"p95":1.704,"ci95":[0.665,0.723],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":1.018,"p95":2.705,"ci95":[0.849,1.055],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.176,"p95":2.146,"ci95":[1.1,1.287],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.179,"p95":2.116,"ci95":[1.14,1.28],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":1.521,"p95":2.572,"ci95":[1.396,1.658],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.531,"p95":1.94,"ci95":[1.481,1.632],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":2.13,"p95":2.35,"ci95":[2.119,2.149],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"dev-server","group":"Agent work","label":"Dev server ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":1.321,"p95":1.66,"ci95":[1.318,1.344],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.942,"p95":4.55,"ci95":[1.916,1.994],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":2.186,"p95":3.414,"ci95":[2.102,2.307],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":2.247,"p95":4.785,"ci95":[2.115,2.466],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":2.556,"p95":6.611,"ci95":[2.499,2.608],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":3.131,"p95":3.264,"ci95":[3.101,3.172],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":3.74,"p95":5.558,"ci95":[3.679,3.812],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":3.999,"p95":4.928,"ci95":[3.976,4.072],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":4.028,"p95":4.83,"ci95":[3.876,4.098],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"preview-first-byte","group":"Agent work","label":"Preview URL first byte","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.181,"p95":0.231,"ci95":[0.178,0.191],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":0.253,"p95":0.44,"ci95":[0.247,0.268],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.273,"p95":0.487,"ci95":[0.265,0.284],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.294,"p95":0.412,"ci95":[0.284,0.306],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.306,"p95":0.733,"ci95":[0.292,0.341],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.377,"p95":0.489,"ci95":[0.372,0.388],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.482,"p95":0.651,"ci95":[0.448,0.511],"attempts":30,"failed":1,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.836,"p95":0.993,"ci95":[0.815,0.859],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.865,"p95":1.119,"ci95":[0.822,0.966],"attempts":29,"failed":0,"status":"measured","equalSpec":true}]},{"id":"edit-loop","group":"Agent work","label":"Edit, rebuild, reload","unit":"s","better":"lower","results":[{"provider":"freestyle","median":0.302,"p95":1.558,"ci95":[0.291,0.322],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":0.313,"p95":0.757,"ci95":[0.298,0.336],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.333,"p95":0.561,"ci95":[0.323,0.344],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.352,"p95":0.379,"ci95":[0.348,0.359],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.427,"p95":0.932,"ci95":[0.42,0.432],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.493,"p95":0.895,"ci95":[0.454,0.507],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.515,"p95":0.612,"ci95":[0.503,0.536],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.073,"p95":1.482,"ci95":[1.07,1.078],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":1.112,"p95":1.404,"ci95":[1.102,1.144],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"command","group":"Everyday calls","label":"Run a command","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.134,"p95":0.193,"ci95":[0.133,0.136],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":0.134,"p95":0.303,"ci95":[0.13,0.138],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.16,"p95":0.257,"ci95":[0.158,0.164],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.183,"p95":0.207,"ci95":[0.177,0.187],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.188,"p95":0.222,"ci95":[0.187,0.191],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.303,"p95":0.314,"ci95":[0.302,0.303],"attempts":98,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.308,"p95":0.383,"ci95":[0.305,0.311],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.325,"p95":0.408,"ci95":[0.321,0.33],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":0.335,"p95":0.393,"ci95":[0.333,0.338],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"files-1kb","group":"Everyday calls","label":"Write + read 1 KB","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.097,"p95":0.11,"ci95":[0.096,0.099],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":0.176,"p95":0.522,"ci95":[0.169,0.181],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.21,"p95":2.386,"ci95":[0.208,0.219],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.234,"p95":0.383,"ci95":[0.23,0.236],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.265,"p95":0.337,"ci95":[0.218,0.292],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.291,"p95":0.317,"ci95":[0.289,0.294],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.357,"p95":0.707,"ci95":[0.354,0.363],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.505,"p95":0.522,"ci95":[0.505,0.507],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.799,"p95":0.966,"ci95":[0.789,0.815],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"files-10mb","group":"Everyday calls","label":"Write + read 10 MB","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.664,"p95":0.855,"ci95":[0.642,0.684],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":1.048,"p95":2.723,"ci95":[1.003,1.096],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.325,"p95":2.268,"ci95":[1.297,1.363],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":2.537,"p95":3.556,"ci95":[2.511,2.576],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":3.229,"p95":10.778,"ci95":[3.057,3.655],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":3.448,"p95":7.082,"ci95":[3.427,3.553],"attempts":97,"failed":2,"status":"measured","equalSpec":false},{"provider":"freestyle","median":4.779,"p95":19.071,"ci95":[4.221,5.212],"attempts":95,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":7.544,"p95":26.769,"ci95":[7.14,7.848],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":7.733,"p95":12.885,"ci95":[6.654,8.49],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"reconnect","group":"Everyday calls","label":"Reconnect","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.298,"p95":0.706,"ci95":[0.285,0.32],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.331,"p95":0.385,"ci95":[0.326,0.337],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.46,"p95":1.828,"ci95":[0.454,0.464],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.495,"p95":0.79,"ci95":[0.478,0.508],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.523,"p95":0.6,"ci95":[0.52,0.527],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.599,"p95":0.814,"ci95":[0.588,0.628],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.642,"p95":0.726,"ci95":[0.632,0.653],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":0.691,"p95":1,"ci95":[0.683,0.695],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.946,"p95":0.992,"ci95":[0.944,0.949],"attempts":99,"failed":0,"status":"measured","equalSpec":false}]},{"id":"destroy","group":"Everyday calls","label":"Destroy","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.114,"p95":0.15,"ci95":[0.114,0.115],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.123,"p95":0.157,"ci95":[0.114,0.126],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.156,"p95":0.169,"ci95":[0.155,0.157],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.182,"p95":0.218,"ci95":[0.181,0.185],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.185,"p95":0.418,"ci95":[0.178,0.196],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.193,"p95":0.385,"ci95":[0.18,0.215],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.265,"p95":0.407,"ci95":[0.257,0.276],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":0.846,"p95":2.534,"ci95":[0.802,0.9],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":1.033,"p95":12.074,"ci95":[0.706,1.666],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"pause-resume","group":"Pause and snapshots","label":"Pause + resume","unit":"s","better":"lower","results":[{"provider":"boxd","median":0.528,"p95":1.078,"ci95":[0.526,0.53],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.725,"p95":2.609,"ci95":[0.682,0.808],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.928,"p95":1.349,"ci95":[0.91,0.953],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":1.636,"p95":11.733,"ci95":[1.271,2.053],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.809,"p95":3.493,"ci95":[1.634,1.962],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":63.779,"p95":84.195,"ci95":[60.58,67.704],"attempts":77,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process.","equalSpec":true},{"provider":"upstash-ephemeral","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Upstash Ephemeral Box has no pause and resume","equalSpec":true},{"provider":"boat","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Boat has no pause: stop and start restarts the machine and ends every running process.","equalSpec":true}]},{"id":"snapshot","group":"Pause and snapshots","label":"Take snapshot","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.385,"p95":0.408,"ci95":[0.379,0.39],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.429,"p95":1.108,"ci95":[0.374,0.54],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.833,"p95":2.192,"ci95":[1.092,2.02],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":2.122,"p95":2.132,"ci95":[2.12,2.123],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":2.139,"p95":8.373,"ci95":[2.132,4.197],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":3.817,"p95":4.266,"ci95":[3.813,3.85],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":6.873,"p95":8.802,"ci95":[6.739,6.888],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":7.353,"p95":49.311,"ci95":[6.824,11.523],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported","equalSpec":true}]},{"id":"create-from-snapshot","group":"Pause and snapshots","label":"Start from snapshot","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.546,"p95":0.635,"ci95":[0.542,0.558],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.752,"p95":1.143,"ci95":[0.708,0.82],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.122,"p95":3.185,"ci95":[0.821,1.561],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":1.126,"p95":1.215,"ci95":[1.108,1.134],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.142,"p95":1.195,"ci95":[1.127,1.145],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.882,"p95":2.902,"ci95":[1.687,1.958],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":2.376,"p95":4.631,"ci95":[2.284,3.002],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":4.893,"p95":13.097,"ci95":[4.812,6.881],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported","equalSpec":true}]},{"id":"burst-10","group":"Bursts","label":"10 at once, all ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.84,"p95":1.568,"ci95":[0.581,1.568],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.854,"p95":1.09,"ci95":[0.753,1.09],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":1.15,"p95":7.978,"ci95":[0.997,7.978],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.194,"p95":1.203,"ci95":[1.065,1.203],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":1.774,"p95":1.858,"ci95":[1.641,1.858],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":2.532,"p95":2.999,"ci95":[2.506,2.999],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":2.946,"p95":9.93,"ci95":[2.259,9.93],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":2.966,"p95":4.828,"ci95":[2.74,4.828],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"stopped-early: in one of its three bursts of 10, two creates failed on E2B's side (E2B reported a timeout), so only 2 of 3 bursts finished and the size is not published; coming in the next update","equalSpec":true}]},{"id":"burst-25","group":"Bursts","label":"25 at once, all ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.79,"p95":1.019,"ci95":[0.752,1.019],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.109,"p95":1.282,"ci95":[0.942,1.282],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":1.124,"p95":1.128,"ci95":[0.995,1.128],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.193,"p95":1.283,"ci95":[1.18,1.283],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":2.037,"p95":4.783,"ci95":[1.441,4.783],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":2.868,"p95":91.559,"ci95":[2.45,91.559],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":5.255,"p95":5.623,"ci95":[2.334,5.623],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":5.405,"p95":5.732,"ci95":[2.83,5.732],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":5.933,"p95":7.661,"ci95":[5.924,7.661],"attempts":3,"failed":0,"status":"measured","equalSpec":false}]},{"id":"cpu-single","group":"Machine speed","label":"CPU, one core","unit":"ops/s","better":"higher","results":[{"provider":"tenki","median":19389801.399,"p95":20883728.38,"ci95":[17233015.355,19780810.813],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":19008462.701,"p95":21528929.456,"ci95":[18132239.036,19416272.673],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":18594166.82,"p95":18969949.57,"ci95":[18556426.205,18625074.293],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":13713334.928,"p95":13771325.536,"ci95":[13705574.473,13721962.163],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":13710323.35,"p95":13765155.972,"ci95":[13699910.961,13720266.633],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":12500550.807,"p95":12956061.462,"ci95":[12393994.324,12568367.678],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":10251510.56,"p95":15304689.118,"ci95":[9812453.69,10586720.16],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":9734199.883,"p95":10391729.55,"ci95":[9614330.552,9935928.466],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":9203127.491,"p95":10071902.478,"ci95":[9055442.433,9349823.468],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"cpu-all","group":"Machine speed","label":"CPU, all cores","unit":"ops/s","better":"higher","results":[{"provider":"upstash-ephemeral","median":50971602.039,"p95":52000982.735,"ci95":[50833655.166,51259146.005],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":50221612.911,"p95":70516909.929,"ci95":[48211429.488,51975659.799],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":48194747.852,"p95":50507719.291,"ci95":[47764670.318,49132415.556],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":43603045.202,"p95":59778516.71,"ci95":[42396617.174,44874063.745],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":36620672.536,"p95":50318522.537,"ci95":[34519769.17,39471763.598],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":29234225.256,"p95":31565477.963,"ci95":[28643418.285,29667677.873],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":27694134.573,"p95":30566164.929,"ci95":[26972322.965,28383751.349],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":26538239.281,"p95":37838605.859,"ci95":[24839733.419,27024762.425],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":25891074.345,"p95":28534089.651,"ci95":[25326459.643,26302225.417],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"memory","group":"Machine speed","label":"Memory bandwidth","unit":"MB/s","better":"higher","results":[{"provider":"tenki","median":45885.83,"p95":51681.928,"ci95":[39082.931,47872.641],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":41968.771,"p95":44108.977,"ci95":[41768.127,43445.468],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":33813.023,"p95":40388.076,"ci95":[32966.088,35282.269],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":31575.622,"p95":47650.692,"ci95":[29208.144,34030.639],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":29883.933,"p95":33205.612,"ci95":[29341.051,30888.712],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":23174.544,"p95":25169.091,"ci95":[22651.758,23454.057],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":16526.611,"p95":17224.461,"ci95":[16254.29,16830.239],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":14740.736,"p95":20495.572,"ci95":[13534.836,16331.047],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":12301.749,"p95":14016.831,"ci95":[11724.035,12682.019],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"disk-write","group":"Machine speed","label":"Disk write","unit":"MB/s","better":"higher","results":[{"provider":"upstash-ephemeral","median":2383.636,"p95":2874.851,"ci95":[2373.118,2438.858],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":1551.886,"p95":2208.41,"ci95":[1215.897,1839.678],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":1380.861,"p95":2935.928,"ci95":[1249.307,1862.095],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":866.805,"p95":1040.304,"ci95":[749.784,902.371],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":744.398,"p95":823.027,"ci95":[741.521,748.193],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":443.502,"p95":598.897,"ci95":[407.25,461.209],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":423.175,"p95":909.63,"ci95":[373.329,498.983],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":198.151,"p95":249.083,"ci95":[187.26,214.794],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":157.063,"p95":855.377,"ci95":[154.427,161.553],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"disk-read","group":"Machine speed","label":"Disk read","unit":"MB/s","better":"higher","results":[{"provider":"upstash-ephemeral","median":23092.711,"p95":23890.318,"ci95":[22869.677,23195.196],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":20318.198,"p95":22819.306,"ci95":[19875.325,20679.429],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":19270.955,"p95":27688.008,"ci95":[17715.953,21526.894],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":17306.776,"p95":18932.84,"ci95":[16697.735,17737.535],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":12077.76,"p95":20536.663,"ci95":[10478.657,14319.661],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":11586.333,"p95":14998.465,"ci95":[11469.15,11935.375],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":6970.516,"p95":8821.793,"ci95":[6389.497,7385.795],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":6203.025,"p95":7452.042,"ci95":[5924.815,6381.831],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":4788.841,"p95":8048.98,"ci95":[4613.883,5397.632],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"iops-write","group":"Machine speed","label":"4K random write","unit":"IOPS","better":"higher","results":[{"provider":"upstash-ephemeral","median":45130.98,"p95":46944.463,"ci95":[44945.389,45422.758],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":20931.147,"p95":30020.422,"ci95":[19576.743,22081.079],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":16975.311,"p95":33346.711,"ci95":[16525.147,17527.769],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":7979.277,"p95":12763.286,"ci95":[7467.048,8700.105],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":6888.833,"p95":8888.347,"ci95":[6609.418,7066.659],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":6114.298,"p95":8069.523,"ci95":[5591.298,6318.835],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":5666.051,"p95":9375.102,"ci95":[4368.665,6636.327],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":370.626,"p95":443.687,"ci95":[357.468,376.473],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":340.386,"p95":547.421,"ci95":[304.65,397.592],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"iops-read","group":"Machine speed","label":"4K random read","unit":"IOPS","better":"higher","results":[{"provider":"boat","median":796398.784,"p95":1027493.478,"ci95":[731978.601,853689.037],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":604306.629,"p95":719169.808,"ci95":[577458.371,640208.068],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":546103.474,"p95":581571.251,"ci95":[539993.806,560548.617],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":501911.593,"p95":527207.164,"ci95":[498149.517,505327.427],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":452995.521,"p95":601548.989,"ci95":[436332.895,476182.504],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":438862.654,"p95":465780.582,"ci95":[431599.852,444456.017],"attempts":99,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":343015.972,"p95":403555.667,"ci95":[337260.263,364131.318],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":258827.694,"p95":338060.923,"ci95":[245408.886,271711.45],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":220965.863,"p95":617031.02,"ci95":[217487.526,226656.987],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]}]},"default":{"id":"default","label":"Default machines","description":"Each provider's default sandbox, so machine sizes differ.","published":true,"run":{"source":"m","start":"2026-10-22T18:05:53.908Z","end":"2026-10-23T23:39:32.627Z","runs":["sbx53-default-burst-r8b2","sbx53-default-burst-r8c2","sbx53-default-burst-r8u","sbx53-default-burst25-r8u","sbx53-default-cost-r8b2","sbx53-default-cost-r8c2","sbx53-default-cost-r8c-superserve","sbx53-default-cost-r8u","sbx53-default-main-r8b3","sbx53-default-main-r8c2","sbx53-default-main-r8u","sbx53-default-snapshot-r8b2","sbx53-default-snapshot-r8c2","sbx53-default-snapshot-r8u","sbx53-default-burst-r8y-1","sbx53-default-burst-r8y-2","sbx53-default-burst-r8y-3","sbx53-default-cost-r8y","sbx53-default-main-r8y","sbx53-default-snapshot-r8y","sbx53-default-main-r8e","sbx53-default-snapshot-r8e","sbx53-default-cost-r8e","sbx53-default-burst-r8e","sbx53-default-main-r8boat2","sbx53-default-cost-r8boat2","sbx53-default-burst-r8boat2-1","sbx53-default-burst25-r8boat2-1","sbx53-default-burst-r8boat2-2","sbx53-default-burst25-r8boat2-2","sbx53-default-burst-r8boat2-3","sbx53-default-burst25-r8boat2-3"],"observations":15609},"absent":[],"providers":[{"id":"e2b","name":"E2B","allocation":{"requested":{"size":null,"template":"base","compute":{"cpu":2,"memoryGiB":0.5},"provisioned":{"cpu":2,"memoryGiB":0.5},"text":"2 vCPU, 512 MB, template base"},"measured":[{"cpu":2,"memoryGiB":0.462,"diskGiB":21.535}],"measuredText":"2 vCPU, 473 MB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":0.5}},"allocationNote":"E2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.","advertisedDefault":{"vcpu":2,"memoryGiB":0.5,"diskGiB":21.534767150878903,"source":"https://e2b.dev/pricing"},"ranIn":"The Dalles, US","cpu":"Intel Xeon Processor @ 2.60GHz","hardware":{"chip":"Intel Xeon Platinum 8373C","family":"Ice Lake, 2.6 GHz, 54 MB L3","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the chip's memory type"},"cloud":"Google Cloud","region":"us-west1, The Dalles, Oregon","runsAs":"Firecracker microVM, nested inside a Google Cloud VM","hidden":["The CPU name, microcode and clock leaves are hidden. The chip is identified from its model number, cache size and 2.6 GHz clock.","The default 2 vCPUs are one physical core with two hyperthreads, so two-thread work scales well below twice one thread.","No DMI or PCI devices, so the memory type can't be read inside; DDR4 follows from the chip."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"metered","perTaskUsd":0.0004926,"per1000TasksUsd":0.4926,"confidence":"medium","tasks":10,"source":"https://api.e2b.app/events/sandboxes/{id}","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.109,"perVcpuHour":0.054,"perHourUsd":0.109,"commonSizeUsd":0.331,"note":"10 GiB Hobby / 20 GiB Pro included disk. Subscription excluded. Per-second billing; no separate creation fee listed.","usdNote":null,"source":"https://e2b.dev/pricing"},"pause":{"supported":true,"split":{"pause":0.2633845460000448,"resume":0.45188361200038346,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]},{"id":"freestyle","name":"Freestyle","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":32},"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.755,"diskGiB":31.201}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":{"vcpu":4,"memoryGiB":8,"diskGiB":32,"source":"https://www.freestyle.sh/docs/guides/run-hexclave-in-a-sandbox"},"ranIn":null,"cpu":null,"hardware":{"chip":"AMD EPYC 7763, likely","family":"Zen 3 (Milan), 2.45 GHz","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the Milan platform's memory"},"cloud":"Freestyle's own network (AS36320)","region":"San Francisco Bay Area","runsAs":"Firecracker microVM, 4 vCPU, 8 GB, no hyperthreads exposed","hidden":["The CPU name, microcode and topology are hidden. The chip is inferred from its 2.45 GHz clock, cache and Zen 3 features.","Memory is added after boot: about 3 GB is free at first, 7.9 GB later.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"estimate","perTaskUsd":0.001782,"per1000TasksUsd":1.782,"confidence":"low","tasks":10,"source":"https://www.freestyle.sh/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.264,"perVcpuHour":0.066,"perHourUsd":0.264,"commonSizeUsd":0.264,"note":"Published overage rates, allowances/subscriptions excluded. $0.02/GB transfer excluded. Billing increment/minimum and creation fees unspecified; one-second estimate.","usdNote":null,"source":"https://www.freestyle.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.2712142259999964,"resume":0.32223399200011044,"n":100}},"reliability":{"attempts":1074,"succeeded":1072,"successRate":0.99814},"failures":[{"kind":"provider-operation","reason":"Preview HTTP 502, no first byte","count":1,"runs":1,"tests":["Preview URL first byte"]},{"kind":"provider-operation","reason":"Native operation timed out. (timeout)","count":1,"runs":1,"tests":["Write + read 10 MB"]}],"setAside":[{"reason":"Our SDK timed out or wrapped the error moving 10 MB, so we can't tell whose side failed","count":2,"runs":2}]},{"id":"tenki","name":"Tenki","allocation":{"requested":{"size":null,"template":null,"compute":null,"provisioned":null,"text":"provider default"},"measured":[{"cpu":2,"memoryGiB":3.797,"diskGiB":4.84}],"measuredText":"2 vCPU, 3.8 GB","equalSpec":false,"verified":null,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":3.796802520751953}},"allocationNote":"Tenki documents a 5 GiB default disk but not a default vCPU or RAM, so those are what we measured.","advertisedDefault":{"vcpu":2,"memoryGiB":3.796802520751953,"diskGiB":5,"source":"https://tenki.cloud/docs/sandbox/cli.md"},"ranIn":null,"cpu":"AMD EPYC 7B13 64-Core Processor","hardware":{"chip":"AMD EPYC 7B13, as reported","family":"Zen 3 (Milan), 2.25 GHz. The 4 vCPU VM also showed EPYC 9575F (Zen 5, Turin) cores","chipBasis":"conflicting","memory":{"type":null,"note":"No memory data inside the VM"},"cloud":"Luxor (AS19538)","region":"Seattle","runsAs":"Cloud Hypervisor microVM, with a container inside","hidden":["On the 4 vCPU VM, two vCPUs report an EPYC 7B13 (Milan) and two an EPYC 9575F (Turin). That fits a snapshot taken on a Milan host and restored on a Turin one, so the real host may be newer than the name says.","CPU features are capped at the Milan set on every vCPU, so no AVX-512 even on Turin.","The clock reads 2.25 GHz, the 7B13's base, which may be carried over from the snapshot."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"estimate","perTaskUsd":0.0008566,"per1000TasksUsd":0.8566,"confidence":"low","tasks":10,"source":"https://tenki.cloud/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.162,"perVcpuHour":0.081,"perHourUsd":0.162,"commonSizeUsd":0.331,"note":"Per-second, no minimum. Separate persistent storage $0.000000076/GiB-second excluded. Credits excluded.","usdNote":null,"source":"https://tenki.cloud/pricing"},"pause":{"supported":true,"split":{"pause":55.56055400800053,"resume":2.2910779680004345,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[{"kind":"provider-operation","reason":"[internal] internal server error","count":2,"runs":2,"tests":["snapshot-cleanup"]}],"setAside":[]},{"id":"boxd","name":"boxd","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":2,"memoryGiB":8},"provisioned":{"cpu":2,"memoryGiB":8,"diskGiB":100},"text":"2 vCPU, 8 GB"},"measured":[{"cpu":2,"memoryGiB":7.777,"diskGiB":97.872}],"measuredText":"2 vCPU, 7.8 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":{"vcpu":2,"memoryGiB":8,"diskGiB":100,"source":"https://boxd.sh/"},"ranIn":"Frankfurt am Main, DE","cpu":"AMD EPYC","hardware":{"chip":"AMD EPYC 9554P, likely","family":"Zen 4 (Genoa), 3.1 GHz","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-4800, the Genoa platform's memory"},"cloud":"Latitude.sh bare metal","region":"Frankfurt, Germany","runsAs":"Firecracker microVM, with Docker installed inside the VM","hidden":["The CPU name is trimmed to \"AMD EPYC\" and the microcode and topology are hidden. The chip is inferred from its 3.1 GHz clock and the Genoa servers Latitude.sh sells; a dual-socket 9554 can't be ruled out.","No DMI, so the memory type is inferred.","AVX-512 is exposed, which can help vectorised work against hosts that hide it.","The disk reports itself as rotational although it is virtual, so tools that tune for hard drives may behave differently."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"metered","perTaskUsd":0.00103,"per1000TasksUsd":1.03,"confidence":"medium","tasks":10,"source":"boxd orgs.usage / orgs.billingActivityPage","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"EUR","perHour":0.174,"perVcpuHour":0.087,"perHourUsd":0.195,"commonSizeUsd":null,"note":"EUR; RAM in use and written disk billed, so allocated resources are an upper bound. Metered per second, settled every 15 minutes (not a rounding increment). FX budget uses conservative 2 USD/EUR, not a market quote.","usdNote":"Converted at the ECB rate of 1.1234 USD per EUR on October 21, 2026.","source":"https://boxd.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.1264708259999752,"resume":0.412353187000379,"n":100}},"reliability":{"attempts":1076,"succeeded":1074,"successRate":0.99814},"failures":[{"kind":"provider-operation","reason":"boxd request failed. (provider)","count":2,"runs":2,"tests":["Write + read 10 MB"]}],"setAside":[]},{"id":"superserve","name":"Superserve","allocation":{"requested":{"size":null,"template":"sandbox-bench-node22-py311-1x1-pinned","compute":{"cpu":1,"memoryGiB":1},"provisioned":{"cpu":1,"memoryGiB":1,"diskGiB":4},"text":"1 vCPU, 1 GB, template sandbox-bench-node22-py311-1x1-pinned"},"measured":[{"cpu":1,"memoryGiB":0.971,"diskGiB":3.859}],"measuredText":"1 vCPU, 994 MB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":1,"memoryGiB":1}},"allocationNote":"Superserve ran a template we built with Node 22 and Python 3.11 preinstalled, at its default 1 vCPU, 1 GB RAM and 4 GB disk. Superserve's own templates have Node (node-22) or Python (python-3.11), not both, and our tests need both.","advertisedDefault":{"vcpu":1,"memoryGiB":1,"diskGiB":4,"source":"https://docs.superserve.ai"},"ranIn":"Washington, US","cpu":"Intel Xeon Processor","hardware":{"chip":"Intel Xeon, Sapphire Rapids","family":"Likely Xeon Platinum 8481C; 2.2 GHz, 105 MB L3","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5, the Sapphire Rapids platform's memory"},"cloud":"Google Cloud","region":"us-east4, Northern Virginia, from its IP range","runsAs":"Firecracker microVM, likely inside a Google Cloud VM","hidden":["The CPU name and microcode are hidden. The chip family is clear from its model number; the exact SKU is not.","The old 4.14 guest kernel can't turn on the chip's AMX units, so AMX workloads can't use them.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.0002687,"per1000TasksUsd":0.2687,"confidence":"high","tasks":10,"source":"https://api.superserve.ai/billing/usage-series (millisecond range) + /activity","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":0.067,"perVcpuHour":0.067,"perHourUsd":0.067,"commonSizeUsd":0.331,"note":"Published vCPU and GiB-hour rates converted to seconds. Credits excluded. One-second rounding assumed; snapshot storage, disk, transfer and any minimum charge excluded. Adapter default superserve/node-22 is 1 vCPU, 1 GiB RAM, 4 GiB disk; resources are template-owned. SDK runtime preparation is included in cold timing.","usdNote":null,"source":"https://www.superserve.ai/pricing"},"pause":{"supported":true,"split":{"pause":0.35829360100021584,"resume":0.5446929919999093,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]},{"id":"upstash","name":"Upstash Box","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":2,"memoryGiB":4},"provisioned":{"cpu":2,"memoryGiB":4,"diskGiB":5},"text":"2 vCPU, 4 GB"},"measured":[{"cpu":2,"memoryGiB":4,"diskGiB":5}],"measuredText":"2 vCPU, 4 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":4}},"allocationNote":null,"advertisedDefault":{"vcpu":2,"memoryGiB":4,"diskGiB":5,"source":"https://upstash.com/pricing/box"},"ranIn":"Ashburn, US","cpu":null,"hardware":{"chip":"AWS Graviton4","family":"Arm Neoverse V2, 2.8 GHz; Graviton4 from the m8g instance type","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-5600, from AWS's Graviton4 spec"},"cloud":"AWS EC2","region":"us-east-1, Ashburn, Virginia","runsAs":"Docker container, on a shared m8g.16xlarge VM; cgroups set its CPU and RAM","hidden":["The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.","nproc and /proc/meminfo show the host's 64 CPUs and 256 GB, while a box may use 2 CPUs and 4 GB (4 and 8 at the medium size).","No DMI tables or clock readings inside, so memory type and clock come from AWS's published spec.","Neighbours share the host kernel and the chip's L3 cache."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.0001652,"per1000TasksUsd":0.1652,"confidence":"high","tasks":10,"source":"https://us-east-1.box.upstash.com/v2/box/{id}","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":0.1,"perVcpuHour":0.05,"perHourUsd":0.1,"commonSizeUsd":0.2,"note":"PAYG ACTIVE CPU billing: Small $0.10, Medium $0.20, Large $0.40 per active box-hour; tier price × wall time is an upper-bound estimate, not metered usage. Disk allocated capacity is an upper bound for written data. 30-day month. No RAM fee listed. Increment/minimum unspecified.","usdNote":null,"source":"https://upstash.com/pricing/box"},"pause":{"supported":true,"split":{"pause":0.7307477690000087,"resume":0.5890358970002271,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]},{"id":"daytona","name":"Daytona","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":1,"memoryGiB":1},"provisioned":{"cpu":1,"memoryGiB":1,"diskGiB":3},"text":"1 vCPU, 1 GB"},"measured":[{"cpu":1,"memoryGiB":1,"diskGiB":3}],"measuredText":"1 vCPU, 1 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":1,"memoryGiB":1}},"allocationNote":null,"advertisedDefault":{"vcpu":1,"memoryGiB":1,"diskGiB":3,"source":"https://www.daytona.io/docs/en/sandboxes/"},"ranIn":null,"cpu":"AMD EPYC 9354P 32-Core Processor","hardware":{"chip":"AMD EPYC 9354P, 9354, 9254 or 9275F","family":"Zen 4 (Genoa); the 9275F is Zen 5 (Turin). Hosts vary","chipBasis":"named by the CPU","memory":{"type":"DDR5","basis":"measured","detail":"Registered DDR5, read from the memory controller on a 9254 host"},"cloud":"Bare-metal servers","region":"Location not visible; control plane on AWS us-east-1","runsAs":"Docker container, shares the host's kernel; cgroups set its CPU and RAM","hidden":["Nothing at the CPU level is hidden: the real CPU name, microcode, server model (Dell PowerEdge R6615 in the probe, a Supermicro AS-2115GT-HNTR in the benchmark) and disks are visible.","nproc and lscpu show every host thread (48 or 64) and /proc/meminfo shows the host's RAM, while the default sandbox may use 1 CPU and 1 GB. Tools that size thread pools or caches from those numbers oversubscribe.","Hosts differ in clock speed, so single-thread results vary from sandbox to sandbox. Neighbours share the host kernel.","Outbound internet is blocked on our account tier, so the sandbox's own network location could not be read."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.0006357,"per1000TasksUsd":0.6357,"confidence":"low","tasks":10,"source":"https://analytics.app.daytona.io/organization/{org}/usage/sandbox","basis":"Billed over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.067,"perVcpuHour":0.067,"perHourUsd":0.067,"commonSizeUsd":0.331,"note":"Hourly published rates converted to seconds. Assumed one-second increment; minimum and creation fee not stated. Snapshot storage excluded.","usdNote":null,"source":"https://www.daytona.io/pricing"},"pause":{"supported":false,"note":"Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process."},"reliability":{"attempts":973,"succeeded":973,"successRate":1},"failures":[],"setAside":[]},{"id":"upstash-ephemeral","name":"Upstash Ephemeral Box","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":2,"memoryGiB":4},"provisioned":{"cpu":2,"memoryGiB":4,"diskGiB":5},"text":"2 vCPU, 4 GB"},"measured":[{"cpu":2,"memoryGiB":4,"diskGiB":5},{"cpu":2,"memoryGiB":4,"diskGiB":10}],"measuredText":"2 vCPU, 4 GB","equalSpec":false,"verified":false,"mismatches":["Restored allocation differs from source"],"pricedAt":null},"allocationNote":null,"advertisedDefault":{"vcpu":2,"memoryGiB":4,"diskGiB":5,"source":"https://upstash.com/pricing/box"},"ranIn":"Ashburn, US","cpu":null,"hardware":{"chip":"AWS Graviton4","family":"Arm Neoverse V2; Graviton4 from the m8gd instance type","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-5600, from AWS's Graviton4 spec"},"cloud":"AWS EC2","region":"us-east-1, Ashburn, Virginia","runsAs":"Container, Ephemeral Box, on a shared AWS m8gd.12xlarge host","hidden":["The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.","Our deeper hardware probe ran on Upstash's regular Box. For Ephemeral Box we have the benchmark's own fingerprints: the AWS host type, the shared host kernel and the Arm architecture.","Memory type and clock come from AWS's published Graviton4 spec, not from inside the box."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"billed","perTaskUsd":0.0001266,"per1000TasksUsd":0.1266,"confidence":"high","tasks":10,"source":"https://us-east-1.box.upstash.com/v2/box/{id}","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":null,"perVcpuHour":null,"perHourUsd":null,"commonSizeUsd":0.2,"note":"PAYG ACTIVE CPU billing: Small $0.10, Medium $0.20, Large $0.40 per active box-hour; tier price × wall time is an upper-bound estimate, not metered usage. Disk allocated capacity is an upper bound for written data. 30-day month. No RAM fee listed. Increment/minimum unspecified.","usdNote":null,"source":"https://upstash.com/pricing/box"},"pause":{"supported":false,"split":null},"reliability":{"attempts":966,"succeeded":957,"successRate":0.99068},"failures":[{"kind":"provider-operation","reason":"Upstash Box exec failed (provider). (provider)","count":12,"runs":2,"tests":["Run a command","Agent tool replay","perf","CPU, one core","CPU, all cores","Memory bandwidth","Disk write","Disk read","4K random write","4K random read","TCP connect"]},{"kind":"provider-operation","reason":"Upstash Box files.write failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=files.write)","count":4,"runs":1,"tests":["Dev server ready","Write + read 1 KB","Agent tool replay","Write + read 10 MB"]},{"kind":"provider-operation","reason":"Upstash Box connect failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=connect)","count":1,"runs":1,"tests":["Reconnect"]}],"setAside":[]},{"id":"boat","name":"Boat","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"documentedDisk":{"value":50,"unit":"GB","scope":"User files; excludes operating system and pre-installed tools","source":"https://docs.boat.dev/machines"}},"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578},{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578},{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578},{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578},{"cpu":4,"memoryGiB":7.751,"diskGiB":68.578}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":{"vcpu":4,"memoryGiB":8,"diskGiB":50,"source":"https://docs.boat.dev/machines"},"ranIn":"Lille, FR","cpu":"AMD Ryzen 9 9950X 16-Core Processor","hardware":{"chip":"AMD Ryzen 9 9950X","family":"Zen 5 (Granite Ridge), 4.3 GHz base, up to 5.7 GHz","chipBasis":"named by the CPU","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5, the only memory the chip takes"},"cloud":"OVHcloud dedicated servers","region":"Northern France","runsAs":"QEMU/KVM virtual machine, on a desktop-class chip","hidden":["A desktop chip with a high boost clock, so single-thread results depend on boost and neighbours.","The cache sizes the guest sees are synthetic and differ from the real chip."],"probed":"Oct 22, 2026"},"costPerTask":{"kind":"estimate","perTaskUsd":0.000495,"per1000TasksUsd":0.495,"confidence":"low","tasks":10,"source":"https://docs.boat.dev/machines","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.036,"perVcpuHour":0.009,"perHourUsd":0.036,"commonSizeUsd":0.036,"note":"Default VM $0.036/hour (4 vCPU, 8 GB, 50 GB). Flat rate for each size; Small, Large and Xlarge are also offered. $20 monthly minimum/packs excluded; billed per second.","usdNote":null,"source":"https://docs.boat.dev/machines"},"pause":{"supported":false,"note":"Boat has no pause: stop and start restarts the machine and ends every running process."},"reliability":{"attempts":916,"succeeded":916,"successRate":1},"failures":[],"setAside":[]}],"quickPicks":[{"need":"Fastest cold start","pick":"E2B 0.50 s","runnerUp":"Superserve 0.53 s"},{"need":"Fastest agent work (tool replay)","pick":"Superserve 13 s","runnerUp":"Daytona 15 s"},{"need":"Fastest 25-sandbox fan-out","pick":"Superserve 0.75 s","runnerUp":"E2B 0.87 s"},{"need":"Fastest reconnect","pick":"Daytona 0.31 s","runnerUp":"Superserve 0.32 s"},{"need":"Cheapest task","pick":"Upstash Ephemeral Box $0.00013 per task, $0.13 per 1,000 (billed)","runnerUp":"Upstash Box $0.00017 per task, $0.17 per 1,000 (billed)"},{"need":"Most reliable","pick":"E2B, Tenki, Superserve, Upstash Box, Daytona, Boat 100.00% (tied)","runnerUp":"boxd 99.81%"}],"metrics":[{"id":"cold","group":"Start-up","label":"Cold start","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.496,"p95":2.027,"ci95":[0.48,0.512],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.53,"p95":0.628,"ci95":[0.521,0.535],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.683,"p95":0.813,"ci95":[0.679,0.687],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.719,"p95":3.035,"ci95":[0.709,0.754],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.863,"p95":3.069,"ci95":[0.81,0.919],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.062,"p95":1.199,"ci95":[1.055,1.071],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":1.166,"p95":1.448,"ci95":[1.125,1.184],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":1.467,"p95":2.758,"ci95":[1.362,1.581],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":2.626,"p95":12.974,"ci95":[2.615,2.705],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"tool-replay","group":"Agent work","label":"Agent tool replay","unit":"s","better":"lower","results":[{"provider":"superserve","median":13.343,"p95":14.308,"ci95":[13.168,13.498],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":14.689,"p95":33.579,"ci95":[12.802,17.717],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":16.338,"p95":18.065,"ci95":[15.611,16.666],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":18.444,"p95":24.262,"ci95":[17.284,19.658],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":22.906,"p95":23.657,"ci95":[22.801,23.103],"attempts":30,"failed":2,"status":"measured","equalSpec":false},{"provider":"freestyle","median":25.577,"p95":44.114,"ci95":[22.425,28.318],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":26.81,"p95":30.805,"ci95":[26.322,27.516],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":32.737,"p95":40.154,"ci95":[32.543,32.914],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":43.401,"p95":50.504,"ci95":[42.819,45.714],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"npm-install","group":"Agent work","label":"npm install","unit":"s","better":"lower","results":[{"provider":"boat","median":0.863,"p95":4.181,"ci95":[0.842,1.844],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":1.008,"p95":1.074,"ci95":[1.001,1.016],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":1.077,"p95":1.182,"ci95":[1.071,1.099],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":1.204,"p95":1.435,"ci95":[1.167,1.31],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.363,"p95":2.081,"ci95":[1.04,1.528],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":1.408,"p95":1.985,"ci95":[1.263,1.769],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.431,"p95":1.781,"ci95":[1.331,1.531],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.619,"p95":3.301,"ci95":[1.551,1.763],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":2.103,"p95":2.805,"ci95":[1.982,2.316],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"pip-install","group":"Agent work","label":"pip install","unit":"s","better":"lower","results":[{"provider":"superserve","median":3.774,"p95":4.628,"ci95":[3.674,4.211],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":3.968,"p95":6.361,"ci95":[3.733,4.326],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":4.147,"p95":4.986,"ci95":[4.003,4.384],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":4.255,"p95":4.931,"ci95":[3.11,4.471],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":4.315,"p95":5.416,"ci95":[4.067,4.574],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":4.494,"p95":5.219,"ci95":[4.39,4.597],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":4.588,"p95":6.195,"ci95":[4.26,4.67],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":4.989,"p95":5.954,"ci95":[4.798,5.069],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":7.454,"p95":13.716,"ci95":[7.389,9.704],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"git-clone","group":"Agent work","label":"git clone","unit":"s","better":"lower","results":[{"provider":"upstash-ephemeral","median":0.614,"p95":0.672,"ci95":[0.606,0.637],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.616,"p95":0.974,"ci95":[0.471,0.871],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.697,"p95":1.254,"ci95":[0.678,0.767],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":1.137,"p95":1.874,"ci95":[1.082,1.184],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.151,"p95":2.607,"ci95":[1.078,1.188],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.161,"p95":2.341,"ci95":[1.028,1.344],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.499,"p95":2.776,"ci95":[1.412,1.755],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.527,"p95":2.042,"ci95":[1.482,1.531],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":2.122,"p95":4.283,"ci95":[2.112,2.131],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"dev-server","group":"Agent work","label":"Dev server ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":1.847,"p95":2.032,"ci95":[1.813,1.859],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.944,"p95":3.2,"ci95":[1.915,1.979],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":2.211,"p95":4.957,"ci95":[2.05,2.54],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":2.316,"p95":3.307,"ci95":[2.164,2.618],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":2.764,"p95":6.801,"ci95":[2.602,3.643],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":3.192,"p95":3.455,"ci95":[3.177,3.233],"attempts":29,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":3.793,"p95":4.785,"ci95":[3.718,3.819],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":3.964,"p95":4.967,"ci95":[3.935,3.968],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":4.092,"p95":5.373,"ci95":[4.002,4.278],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"preview-first-byte","group":"Agent work","label":"Preview URL first byte","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.188,"p95":0.251,"ci95":[0.183,0.195],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.268,"p95":0.532,"ci95":[0.261,0.278],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.28,"p95":0.457,"ci95":[0.258,0.309],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.29,"p95":0.389,"ci95":[0.281,0.3],"attempts":28,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.296,"p95":0.432,"ci95":[0.287,0.307],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.401,"p95":1.461,"ci95":[0.385,0.443],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.477,"p95":1.69,"ci95":[0.452,0.529],"attempts":30,"failed":1,"status":"measured","equalSpec":true},{"provider":"boat","median":0.854,"p95":1.194,"ci95":[0.728,0.987],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.907,"p95":1.504,"ci95":[0.855,0.946],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"edit-loop","group":"Agent work","label":"Edit, rebuild, reload","unit":"s","better":"lower","results":[{"provider":"freestyle","median":0.312,"p95":3.795,"ci95":[0.292,0.342],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.344,"p95":0.591,"ci95":[0.332,0.356],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.344,"p95":0.962,"ci95":[0.325,0.449],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.363,"p95":0.396,"ci95":[0.358,0.37],"attempts":28,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.429,"p95":1.117,"ci95":[0.419,0.442],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.458,"p95":0.779,"ci95":[0.449,0.495],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.518,"p95":1.662,"ci95":[0.506,0.532],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.068,"p95":2.264,"ci95":[1.064,1.07],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":1.14,"p95":1.493,"ci95":[1.127,1.189],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"command","group":"Everyday calls","label":"Run a command","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.137,"p95":0.149,"ci95":[0.135,0.139],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.138,"p95":0.314,"ci95":[0.133,0.173],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.164,"p95":0.258,"ci95":[0.159,0.168],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.169,"p95":0.195,"ci95":[0.164,0.174],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.192,"p95":0.232,"ci95":[0.189,0.193],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.302,"p95":0.307,"ci95":[0.302,0.303],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.311,"p95":0.391,"ci95":[0.309,0.313],"attempts":100,"failed":2,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.323,"p95":0.413,"ci95":[0.319,0.326],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.333,"p95":0.452,"ci95":[0.331,0.338],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"files-1kb","group":"Everyday calls","label":"Write + read 1 KB","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.098,"p95":0.108,"ci95":[0.096,0.1],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.177,"p95":0.532,"ci95":[0.171,0.185],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.209,"p95":0.545,"ci95":[0.206,0.214],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.209,"p95":0.321,"ci95":[0.207,0.209],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.237,"p95":0.26,"ci95":[0.233,0.239],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.298,"p95":0.329,"ci95":[0.295,0.301],"attempts":99,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.357,"p95":1.091,"ci95":[0.354,0.362],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.5,"p95":0.513,"ci95":[0.499,0.502],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.803,"p95":1.001,"ci95":[0.791,0.813],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"files-10mb","group":"Everyday calls","label":"Write + read 10 MB","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.637,"p95":0.891,"ci95":[0.6,0.659],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":1.184,"p95":2.541,"ci95":[1.152,1.275],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.345,"p95":2.431,"ci95":[1.308,1.376],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":2.634,"p95":10.956,"ci95":[2.608,2.661],"attempts":99,"failed":1,"status":"measured","equalSpec":false},{"provider":"freestyle","median":2.864,"p95":11.315,"ci95":[2.724,3.41],"attempts":98,"failed":1,"status":"measured","equalSpec":true},{"provider":"upstash","median":2.885,"p95":7.502,"ci95":[2.812,3.052],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":3.507,"p95":7.274,"ci95":[3.413,3.585],"attempts":100,"failed":2,"status":"measured","equalSpec":false},{"provider":"tenki","median":3.61,"p95":9.163,"ci95":[1.853,4.597],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":7.369,"p95":20.376,"ci95":[6.988,7.687],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"reconnect","group":"Everyday calls","label":"Reconnect","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.313,"p95":0.77,"ci95":[0.286,0.344],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.323,"p95":0.371,"ci95":[0.321,0.326],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.457,"p95":0.683,"ci95":[0.455,0.46],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.499,"p95":0.829,"ci95":[0.484,0.505],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.531,"p95":0.605,"ci95":[0.528,0.533],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.604,"p95":0.758,"ci95":[0.593,0.642],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.671,"p95":2.946,"ci95":[0.661,0.68],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.683,"p95":0.93,"ci95":[0.679,0.693],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.944,"p95":0.991,"ci95":[0.942,0.946],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"destroy","group":"Everyday calls","label":"Destroy","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.111,"p95":0.12,"ci95":[0.111,0.112],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.136,"p95":0.206,"ci95":[0.133,0.14],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.156,"p95":0.167,"ci95":[0.156,0.157],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.183,"p95":0.299,"ci95":[0.179,0.191],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":0.185,"p95":0.245,"ci95":[0.183,0.186],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.191,"p95":0.461,"ci95":[0.179,0.209],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":0.275,"p95":0.424,"ci95":[0.271,0.285],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.958,"p95":3.242,"ci95":[0.899,1.054],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.961,"p95":7.836,"ci95":[0.802,1.235],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"pause-resume","group":"Pause and snapshots","label":"Pause + resume","unit":"s","better":"lower","results":[{"provider":"boxd","median":0.54,"p95":3.198,"ci95":[0.538,0.541],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.641,"p95":2.783,"ci95":[0.6,0.688],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.739,"p95":3.624,"ci95":[0.717,0.836],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.915,"p95":3.687,"ci95":[0.899,0.943],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":1.735,"p95":12.987,"ci95":[1.336,2.772],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":58.057,"p95":81.559,"ci95":[56.15,59.47],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process.","equalSpec":false},{"provider":"upstash-ephemeral","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Upstash Ephemeral Box has no pause and resume","equalSpec":false},{"provider":"boat","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Boat has no pause: stop and start restarts the machine and ends every running process.","equalSpec":true}]},{"id":"snapshot","group":"Pause and snapshots","label":"Take snapshot","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.344,"p95":0.667,"ci95":[0.317,0.391],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.357,"p95":0.404,"ci95":[0.353,0.363],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.393,"p95":1.274,"ci95":[0.329,0.46],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":2.123,"p95":2.13,"ci95":[2.121,2.125],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":2.139,"p95":16.608,"ci95":[2.134,4.199],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":3.068,"p95":3.104,"ci95":[3.066,3.072],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":6.742,"p95":6.973,"ci95":[6.724,6.762],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":10.141,"p95":336.146,"ci95":[7.514,15.486],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported","equalSpec":true}]},{"id":"create-from-snapshot","group":"Pause and snapshots","label":"Start from snapshot","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.555,"p95":0.681,"ci95":[0.541,0.566],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.73,"p95":18.005,"ci95":[0.699,0.895],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.75,"p95":1.166,"ci95":[0.552,0.915],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":1.079,"p95":1.304,"ci95":[0.929,1.111],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.122,"p95":1.284,"ci95":[1.096,1.131],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.826,"p95":2.482,"ci95":[1.72,2.13],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":2.615,"p95":6.827,"ci95":[2.341,3.184],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":4.818,"p95":15.143,"ci95":[2.836,4.845],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported","equalSpec":true}]},{"id":"burst-10","group":"Bursts","label":"10 at once, all ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.647,"p95":0.75,"ci95":[0.569,0.75],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":0.748,"p95":1.083,"ci95":[0.74,1.083],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.039,"p95":1.535,"ci95":[1.034,1.535],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.054,"p95":1.072,"ci95":[0.623,1.072],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.895,"p95":2.603,"ci95":[1.661,2.603],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":1.995,"p95":2.839,"ci95":[1.658,2.839],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":2.299,"p95":2.382,"ci95":[2.088,2.382],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":2.462,"p95":2.643,"ci95":[1.844,2.643],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":5.022,"p95":6.752,"ci95":[2.774,6.752],"attempts":3,"failed":0,"status":"measured","equalSpec":false}]},{"id":"burst-25","group":"Bursts","label":"25 at once, all ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.753,"p95":1.649,"ci95":[0.747,1.649],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.869,"p95":1.161,"ci95":[0.826,1.161],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":1.044,"p95":1.128,"ci95":[0.972,1.128],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.308,"p95":1.875,"ci95":[0.981,1.875],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":3.959,"p95":4.583,"ci95":[3.201,4.583],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":4.636,"p95":9.714,"ci95":[2.885,9.714],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":5.979,"p95":6.338,"ci95":[5.1,6.338],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":15.394,"p95":17.395,"ci95":[9.006,17.395],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"stopped-early: in one of its three bursts of 25, one create timed out on Daytona's side (24 of 25 ready), so only 2 of 3 bursts finished and the size is not published; coming in the next update","equalSpec":false}]},{"id":"cpu-single","group":"Machine speed","label":"CPU, one core","unit":"ops/s","better":"higher","results":[{"provider":"boat","median":18806868.562,"p95":21684871.586,"ci95":[17832135.516,19094245.376],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":18636780.279,"p95":21366810.51,"ci95":[11324913.14,19901714.983],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":18515717.845,"p95":18797796.688,"ci95":[18450597.087,18571708.58],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":13725720.473,"p95":13765771.868,"ci95":[13709300.18,13732476.431],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":13715680.634,"p95":13767875.506,"ci95":[13706265.88,13724213.288],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"boxd","median":12529897.277,"p95":12903004.873,"ci95":[12260795.555,12615614.378],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":10002746.854,"p95":15281191.108,"ci95":[9723696.118,10646967.789],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":9627280.88,"p95":10370353.962,"ci95":[9468758.973,9860129.233],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":9085949.894,"p95":13686739.872,"ci95":[8915114.314,9253975.06],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"cpu-all","group":"Machine speed","label":"CPU, all cores","unit":"ops/s","better":"higher","results":[{"provider":"boat","median":51673262.263,"p95":71527831.622,"ci95":[48624230.283,53718569.248],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":26102954.403,"p95":30821851.248,"ci95":[24872441.529,27129123.819],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash-ephemeral","median":26048807.552,"p95":26476930.139,"ci95":[25971340.338,26119308.436],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":24961217.098,"p95":25777210.703,"ci95":[24580630.622,25385928.128],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":20868639.811,"p95":31785886.202,"ci95":[18111234.256,22071295.759],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":15746005.347,"p95":19241624.837,"ci95":[15572425.061,15932044.985],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":14064935.684,"p95":20202960.354,"ci95":[13852729.417,14898408.682],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":12153511.736,"p95":16663913.788,"ci95":[12050149.106,12233066.986],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":6939456.483,"p95":10295672.724,"ci95":[6776784.702,7034213.496],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"memory","group":"Machine speed","label":"Memory bandwidth","unit":"MB/s","better":"higher","results":[{"provider":"upstash-ephemeral","median":41093.439,"p95":44096.856,"ci95":[40471.757,42736.386],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"tenki","median":38900.837,"p95":52093.643,"ci95":[24291.51,48027.177],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":35283.958,"p95":40131.438,"ci95":[33908.704,36153.265],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":32147.398,"p95":48200.782,"ci95":[27340.708,35814.552],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":31039.442,"p95":57030.223,"ci95":[30299.621,31851.516],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":22407.59,"p95":24676.296,"ci95":[21969.865,22804.802],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":16102.635,"p95":16967.804,"ci95":[15887.713,16278.511],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":14354.682,"p95":20467.766,"ci95":[12996.507,15789.205],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":12347.819,"p95":14197.466,"ci95":[12147.253,12657.542],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"disk-write","group":"Machine speed","label":"Disk write","unit":"MB/s","better":"higher","results":[{"provider":"upstash-ephemeral","median":2377.542,"p95":2933.845,"ci95":[2361.923,2436.347],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":1734.72,"p95":2195.041,"ci95":[1513.004,1913.546],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":1215.408,"p95":2867.014,"ci95":[1084.228,1338.61],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":948.921,"p95":1144.3,"ci95":[885.329,996.488],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":746.526,"p95":834.545,"ci95":[741.22,750.504],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":482.108,"p95":786.695,"ci95":[433.125,522.208],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":414.644,"p95":577.646,"ci95":[266.971,457.436],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":203.258,"p95":244.455,"ci95":[190.883,210.463],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":155.079,"p95":720.116,"ci95":[151.31,160.302],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"disk-read","group":"Machine speed","label":"Disk read","unit":"MB/s","better":"higher","results":[{"provider":"upstash-ephemeral","median":23036.161,"p95":24036.296,"ci95":[22859.73,23256.291],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":20767.906,"p95":22827.012,"ci95":[20362.408,21124.928],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boat","median":20511.448,"p95":28865.008,"ci95":[17864.418,21822.765],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":16184.48,"p95":28838.01,"ci95":[15912.732,16598.344],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":11156.425,"p95":13244.869,"ci95":[8178.485,11619.826],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":10557.55,"p95":21552.834,"ci95":[7806.081,11701.998],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":6543.66,"p95":8719.928,"ci95":[6041.207,7277.965],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":6242.443,"p95":7595.339,"ci95":[6075.814,6520.874],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":3974.642,"p95":8060.873,"ci95":[3852.096,4774.514],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"iops-write","group":"Machine speed","label":"4K random write","unit":"IOPS","better":"higher","results":[{"provider":"upstash-ephemeral","median":43322.627,"p95":46514.115,"ci95":[43016.591,43913.709],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"daytona","median":20227.984,"p95":29622.513,"ci95":[18553.922,20916.745],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":17797.2,"p95":31700.839,"ci95":[17014.816,18155.673],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":8787.638,"p95":14103.293,"ci95":[8323.161,9422.221],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":7207.38,"p95":8832.879,"ci95":[6804.212,7471.494],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":5643.226,"p95":7914.267,"ci95":[4703.988,6448.612],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boat","median":5333.094,"p95":9275.854,"ci95":[4523.973,6234.913],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":439.101,"p95":556.247,"ci95":[379.394,480.702],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":361.302,"p95":433.345,"ci95":[332.591,379.824],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"iops-read","group":"Machine speed","label":"4K random read","unit":"IOPS","better":"higher","results":[{"provider":"boat","median":772620.269,"p95":1063069.947,"ci95":[697471.665,845068.563],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":641378.965,"p95":748054.62,"ci95":[632126.86,654718.062],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash-ephemeral","median":543848.531,"p95":577571.665,"ci95":[536341.903,557065.629],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":505216.902,"p95":527276.943,"ci95":[501561.749,509105.756],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":466975.186,"p95":643001.208,"ci95":[451691.107,479054.466],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":415566.859,"p95":458198.36,"ci95":[398297.276,429584.262],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":342378.516,"p95":774722.968,"ci95":[323914.707,348596.226],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":266989.842,"p95":335738.147,"ci95":[252697.248,283806.513],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":223463.492,"p95":623356.385,"ci95":[221451.372,226612.646],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]}]}}}