Cockroach Labs Cloud Report 2022: AMD outperforms Intel
Cockroach Labs recently released its annual Cloud Report which assesses the performance of AWS, Microsoft Azure and Google Cloud for common OLTP workloads. Unlike in the past, this year’s report does not point to a better overall vendor, but concludes that AMD instances outperform Intel. ARM instances were not covered by the tests.
According to the report, all three major cloud providers offer similar options at competitive prices for mission-critical applications and workloads. Achieving over 3000 benchmark runs on 56 different instance types and 107 discrete configurations, the price-performance leaders were AMD-based instances running Milan processors. Keith McClellan, Director of Partner Solutions Engineering at Cockroach Labs and lead author of the Cloud Report, writes:
It’s still like the late 1990s, with AMD beating Intel on pure performance…I think the last time this happened AMD was still selling chips called “Athlon”!
The lack of coverage for ARM instances and newer instance types leaves unanswered questions for use cases where, for example, Graviton instances on AWS might be better suited. McClellan acknowledges this:
ARM will be making a comeback in the 2023 Cloud Report – unfortunately we didn’t support ARM binaries for CockroachDB ready in time for this year’s report.
Not all reviews agree on the benefits of AMD instances for database workloads. The benchmark “AWS CPU Economics Comparison for MySQL (ARM vs Intel vs AMD)” conducted by Percona, concludes that Graviton is cheaper in most cases and that Intel generally performs better than AMD on AWS.
According to Cockroach Labs, small instances perform better than large ones. In OLTP and CPU benchmarks, the report shows a performance advantage per vCPU compared to running on smaller instances of a particular instance type, regardless of CPU platform, cloud, or server. instance type.
Processors and CPU benchmarks weren’t the only focus. The report highlights the importance of storage and transfer costs and how they have a significant impact on the total cost of operation:
Even for relatively small amounts of persistent block storage, the cost of running a particular workload is influenced much more by the cost of storage than by the cost of the instance. For persistent workloads, it is extremely important to optimize cost calculations based on this consideration.
The benchmark points out that it’s generally not worth paying for high-performance storage. For the same CPU, cloud providers offer different instance classes, with different vCPU to RAM ratios. In the report, the authors conclude:
Our testing suggests that while you can save some money by choosing instance types with a lower vCPU to RAM ratio, you’ll likely get more consistent performance from instance types with more available memory. The impact of this decision is more apparent with larger and more complex workloads. In our testing, we found the sweet spot to be a 1:4 vCPU:RAM ratio.
Compared to the previous edition, the Cloud 2022 report added tests for instance types of different sizes, cross-region latency tests, storage tests with fsync, and variable OLTP workload complexity tests . As the framework with all resources and configurations is available on GitHub, cloud architects can perform further benchmarks with newer processors and different storage options.
Access to the full report is free but requires registration.