Methodology

How we benchmark

Our benchmark suite is being finalized. So that the results mean something when they land, this page states the standard we hold them to up front: the exact setup, what each workload does on every platform, and the honest limits of what a given comparison can and can't tell you. We would rather publish numbers you can reproduce than numbers that flatter us.

Setup

Same handler, same load, measured over the wire

Same handler: the identical worker source runs on DataVec and on each platform under test — no per-platform hand-tuning.
Deployment shape as shipped: each platform is measured in the shape it actually deploys. A runtime that can't terminate TLS or HTTP/2 itself is measured with the edge proxy in front of it, because that hop is intrinsic to the architecture, not something we add.
Isolated client: the load generator runs on separate cores so it never contends with the server under test.
Protocol: HTTP/2 with TLS on every workload.
Load: a standard driver (fortio) at fixed concurrency with keep-alive, reporting full latency percentiles and checking for server saturation.
Reproducible: the workloads, driver, and comparison scripts are committed, so a run on comparable hardware reproduces the figures.

Workloads

The same handler on every platform

Static routing — a tiny response; pure dispatch + framing.
JSON transform — parse a JSON body, sum an array, serialize the result.
JWT verify — verify one token per request; RS256 (RSA-2048) and HS256 (HMAC-SHA-256).
SSE token streaming — stream a chunked text/event-stream response from a ReadableStream (the AI-gateway shape, minus the model).
Static asset — serve a cached body at small and large sizes.
Database query — one indexed SELECT per request over the network, isolating the connection model (pooled session vs. reconnect-per-request vs. embedded store).

Honest limits

What a benchmark does and doesn't show

Compute efficiency vs. geographic latency are different measurements. A same-machine comparison isolates compute efficiency and cost per request; what a user in a specific city feels depends on how close they are to each provider's edge — a separate measurement we will label as such and not conflate.
We choose the conservative framing. Where a fair comparison can be drawn more than one way, we prefer the one that is, if anything, generous to the alternative, and we say which we chose.
We report tails, not just averages. Full p50/p99 latency accompanies throughput, including any workload where a tail favors another platform.
First-connection warm-up is distinct from cold starts. The first request to a just-started instance pays a one-time TLS-handshake and instruction-cache warm-up that amortizes on a live connection; it is not a per-request cold start (of which there are none).
Cost is quoted as a ratio. The per-request cost advantage is a throughput ratio; absolute dollars depend on your hosting price, so we quote the ratio rather than a headline dollar figure.

Results are on the way

Back to Benchmarks View Pricing