China cloud VM benchmark methodology — open tools, one window, raw data

Public, independently measured VM performance data for Mainland China’s clouds effectively does not exist — every “which China cloud is fastest” answer today is a catalog check, a vendor claim, or folklore. The Chinaready engineering team closes that gap with a recurring benchmark: one open-source toolchain, run in a single time window, on same-tier instances of Alibaba Cloud, Tencent Cloud, AWS China, and Azure China. The first measured report lands in November 2026, and waves repeat every six months. This page is the methodology, published before any data — the contract every report will be held to.

The gap

Public measurement stops exactly where it matters most for China: the mainland-region IaaS layer.

• The closest international precedent, Cloud Mercato’s Public Cloud Reference, continuously collects catalog, pricing, and performance data across 70+ providers — but its performance coverage does not extend to the Mainland China regions of the Chinese hyperscalers.
• VPSBenchmarks has run monthly, buy-like-a-customer VPS trials since 2014, including a 24-hour sustained CPU endurance test and dedicated consistency scoring — but it benchmarks VPS plans, not a cross-cloud matrix of mainland-region VMs.
• Existing China performance studies measure the web delivery layer — whether global websites load from Mainland vantage points — not the IaaS underneath them.
• The vendors themselves publish testing guides for exactly the tools a benchmark needs: Volcengine documents Sysbench and iPerf3 procedures, Alibaba Cloud documents fio block-storage testing, and Tencent Cloud publishes a network performance testing guide. The vocabulary is agreed. The cross-cloud numbers are missing.

What we measure, and with what

Every dimension uses a pinned, open-source, industry-standard tool. Nothing exotic, nothing proprietary — deliberately, because the toolchain must be reproducible by any reader.

Dimension	Tool (version pinned per wave)	What we report
CPU	sysbench cpu, single- and multi-thread; UnixBench for longitudinal history	events/s, single-thread and all-thread, with the host CPU model disclosed verbatim from lscpu
Memory	STREAM (official run rules); sysbench memory	sustained bandwidth GB/s (Copy/Scale/Add/Triad) and access latency, reported separately
Block storage	fio: 4k random read/write, 1M sequential read/write; libaio, direct=1, iodepth 32	IOPS, throughput MB/s, p50/p99 latency — disk tier and capacity disclosed (performance scales with both)
Network	iperf3 TCP single vs multi stream, UDP small-packet; sockperf latency-under-load	Gbps, packets/s, and p50/p99 latency with load — idle and loaded latency reported separately
Stability	The same tests, repeated on freshly created instances across time slots	coefficient of variation, min/p50/p95 spread, a 24-hour sustained-CPU endurance run, and evening-peak deltas
Price-performance	Every score divided by the instance’s list-price hour	score per CNY/hour at list price — never negotiated discounts

The rules that keep waves comparable

A single wave’s numbers are anecdotes; a benchmark program is the discipline wrapped around them. Ours:

1. One window, all clouds. Every cloud is measured inside the same 72-hour period, including Mainland China’s evening peak (20:00–23:00 CST), because cross-time performance variance is itself a finding.
2. Tier mapping, disclosed. Two standard tiers are tested (4 vCPU/16 GB and 8 vCPU/32 GB, the 1:4 general-purpose ratio every cloud sells), each cloud’s closest-spec instance family is chosen, and the mapping rationale is published. CPU model, disk tier, and bandwidth-limit model are disclosed per cloud — Chinese clouds commonly run previous-generation or domestic CPUs, and hiding that would be the easiest way to fake fairness.
3. Five rounds, three time slots. Every cell is the median of ≥5 runs across ≥3 time slots, with p95 and coefficient of variation alongside. Best-of numbers are never reported.
4. Fresh instances. Test instances are recreated rather than reused, so residual state cannot flatter a number; noisy-neighbor effects surface as the spread across rebuilds rather than being averaged away.
5. Frozen environment. Same OS image and kernel across clouds, tool versions pinned per wave and printed in the report, fio job files published verbatim.
6. List prices, triple-noted. Currency, exchange rate, and fetch date accompany every price. List prices are public and reproducible; negotiated discounts are neither.
7. Versioned data, changelogged methodology. Each wave freezes a dataset version (v2026-11 style). When a cloud retires an instance family, the successor runs one overlap wave against the old family and the change is logged as a generation change — never silently swapped.

How to reproduce the tests

The toolchain is deliberately ordinary — stock open-source releases, installed from distro packages or upstream source, versions pinned per wave. A minimal reproduction needs two same-tier VMs in the same region (the network tests are peer-to-peer) and a fresh data disk:

# CPU — single-thread and all-thread, 60 s per run
sysbench cpu --threads=1 --time=60 run
sysbench cpu --threads=$(nproc) --time=60 run

# Memory — access latency; STREAM bandwidth is compiled and run
# per its official run rules (array size >= 4x last-level cache)
sysbench memory --memory-block-size=1M --memory-total-size=100G --time=60 run

# Block storage — 4k random and 1M sequential, against a fresh data disk
fio --name=randread  --filename=/data/fio.test --size=20G \
    --ioengine=libaio --direct=1 --bs=4k --rw=randread  --iodepth=32 \
    --runtime=60 --time_based --group_reporting
fio --name=randwrite --filename=/data/fio.test --size=20G \
    --ioengine=libaio --direct=1 --bs=4k --rw=randwrite --iodepth=32 \
    --runtime=60 --time_based --group_reporting
fio --name=seqread   --filename=/data/fio.test --size=20G \
    --ioengine=libaio --direct=1 --bs=1M --rw=read --iodepth=32 \
    --runtime=60 --time_based --group_reporting
fio --name=seqwrite  --filename=/data/fio.test --size=20G \
    --ioengine=libaio --direct=1 --bs=1M --rw=write --iodepth=32 \
    --runtime=60 --time_based --group_reporting

# Network — start the receiver first, then run from the peer
iperf3 -s
iperf3 -c <peer-ip> -t 60              # TCP single stream
iperf3 -c <peer-ip> -P 8 -t 60         # TCP, 8 parallel streams
iperf3 -c <peer-ip> -u -b 0 -l 64 -t 60  # small-packet UDP (pps)

# Latency under load — idle and loaded latency reported separately
sockperf server
sockperf under-load -i <peer-ip> --time 60

The environment discipline around the commands is what makes results comparable: same OS image and kernel across clouds; freshly created instances and a fresh data disk per cell; every run repeated at least five times across at least three time slots including the 20:00–23:00 CST evening peak; lscpu output and every tool version captured alongside the raw JSON. The complete fio job files, STREAM build, and orchestration scripts ship with each report.

Scope discipline: VMs only

The benchmark measures virtual machines — CPU, memory, block storage, and network — and nothing else. That tightness is deliberate: every dimension runs the same open toolchain on the same kind of surface with the same disclosure rules, so there is no per-vendor asterisk anywhere in the data. Scope grows only when a new dimension would genuinely change a cloud decision, and any expansion is announced in the methodology changelog one wave before it appears in a report. For the network path into China rather than compute inside it, the live China network diagnostics probe continuously from the other direction.

Why this benchmark matters — and how to read it

Why these tiers. The 4 vCPU / 16 GB tier is the most-sold general-purpose shape on every cloud in scope — the ratio most real workloads actually buy — and 8 vCPU / 32 GB checks how each platform scales within the same family. General-purpose rather than compute- or memory-optimized, because general-purpose is the default landing zone for the workloads readers of this research run. Every report publishes the full instance-mapping table with its selection rationale, so the tier choice is auditable rather than asserted.

How to read the numbers. Read medians with their coefficient of variation, never a single best run — the CV is the stability and noisy-neighbor signal, and a fast median with a wide CV is a different product than a fast median with a tight one. Treat the evening-peak delta as a real cost for anything user-facing in China, not as noise. For procurement decisions, read the price-performance column — score per CNY/hour at list price — rather than absolute performance alone.

How not to read them. These are synthetic micro-benchmarks: they characterize infrastructure, not your application, and the shortlist they produce should be validated against your own workload. A 72-hour window is an observation, not an SLA — cloud platforms continuously re-balance hardware and generations, which is why the cadence is a fresh wave every six months with dataset versions preserved for longitudinal comparison. And this is not a “which cloud is best” ranking: managed services, consoles, support rails, and filing or compliance tracks sit outside the measurement. Platform choice is a multi-layer decision; compute performance is one layer of it.

What every report contains

The report format is fixed and identical across waves: a versioned dataset stamp and the exact test window; the instance-mapping table with selection rationale; CPU model, disk tier, and bandwidth-model disclosure per cloud; median (p95) tables with coefficient of variation for every cell; evening-peak deltas; a price-performance table at list price; field notes where anomalies are recorded rather than deleted; reproduction commands; and the independence statement. No number ships without its conditions.

The first wave covers Alibaba Cloud, Tencent Cloud, AWS China, and Azure China Mainland regions in a single 72-hour window in November 2026, with the report following in the same month. Challenges to any number are welcome — the raw data and scripts ship with every release precisely so they can be argued with.

Disclaimers

This program is independent research, funded by Chinaready, using list prices only, and accepts no vendor money. Chinaready is not affiliated with, sponsored by, or endorsed by any cloud provider named in these reports; all vendor names and trademarks belong to their respective owners and are used for identification only. Measurements are observations from a specific, disclosed time window and configuration — they are not performance guarantees, SLAs, or certifications, and they do not constitute procurement or architectural advice; readers should validate findings against their own workloads. The research program operates independently of Chinaready’s commercial services, and no dataset is altered for commercial reasons. Methodology changes are announced in the changelog one wave in advance, and historical dataset versions remain available for comparison.