-
Developer Tools,
Oracle Cloud Infrastructure
May 16, 2017
Block Volume Performance Analysis
Hi everyone, I’m Lee Gates and I am focused on all things performance in the Bare Metal Cloud Services product management team.
My colleague Max Verun posted about the exciting update to our BMCS Block Volume Service. Today, we will describe common sizing, performance, and new features from a performance perspective. Max's overview touched on the performance enabled with this update. Oracle’s Block Volume Storage Service meets or exceeds the demanding requirements presented by latency sensitive enterprise applications, difficult to forecast High Performance Computing problems, and modern data guzzling applications.
Demanding enterprise applications need consistent predictable performance, and that is exactly what we are delivering. In case you haven’t reviewed Max’s blog (link above) here is a summary of our block volume service:
Before running any tests, protect your data by making a backup of your data and operating system environment to prevent any data loss.
WARNING: Do not run FIO tests directly against a device that is already in use, such as /dev/sdX. If it is in use as a formatted disk and there is data on it, running FIO with a write workload (readwrite, randrw, write, trimwrite) will overwrite the data on the disk, and cause data corruption. Run FIO only on unformatted raw devices that are not in use.
| Metric or Feature | Block Volume Service Characteristics |
|---|---|
| Flexibility |
Configurable: 50 GB to 2 TB (1 GB increments) All NVMe-based. 16 TB soon. |
| Performance: IOPS/Volume | 60 IOPS/GB - up to 25K IOPS |
| Performance: Throughput/Volume | 480 KBPS/GB - up to 320 MBPS |
| Performance: Latency/Volume | Sub-millisecond latencies |
| Performance: Per-instance Limits |
32 attachments/instance, up to 64 TB Up to 200K or more IOPS, near line rate throughout |
| Cost per GB/month | 5 cents! Simple model, 1 option! |
Now let’s dig into the numbers and performance testing of our offering, and you will see we are delivering unmatched performance, with linear scale from a throughput and IOPS perspective with consistency. First, performance, new BMCS block volumes now deliver 25,000 IOPS, meeting our goal of unmatched performance at an unmatched price of 0.05/GB-month. There is more to unmatched performance than simply an IOP’s number. We are delivering IOPS that scale linearly, 60 IOPS per GB. Throughput is also critical to today’s demanding applications, and our service provides 480 KBS per GB, with linear scale. We are also designed to deliver repeatable, consistent performance, as this is what enterprise applications demand. All of these tests are run consecutively 5 times to achieve steady state results. Our consistency allows our customers to deliver a superior user experience that they can depend on.
Now, we'll look at how the new volumes perform in detail and present a summary of application performance we expect to see for new use cases with similar data traffic profiles or simply sizing for workload migration. Let me describe the environments that we will test:
- 50GB - Our smallest size you can provision, 3,000 IOPS @ 4K, a great fit for many applications.
- 1TB - The workhorse of the storage industry, 25,000 IOPS @ 4K. Most applications may have a small working set, but keep a lot of supporting data and applications on the same volume.
- Host maximum Ashburn: 400,000 IOPS @ 4K
Let’s begin with a basic overview of the expected performance we plan to observe:
|
Block Volumes
|
IOPS:GB |
Network Throughput |
Latency |
Optimized |
|---|---|---|---|---|
| Created after May 15, 2017 |
60:1 up to |
480KBS:GB |
< 1 millisecond |
4K Block size |
We’ll start measuring with 1TB to match the requirements for many configurations requiring large capacity. For these tests, we'll use a wide range block sizes and the most common read and write patterns. These tests and charts were generated with the Gartner Cloud Harmony test suite. For 1TB , we see the bandwidth limit for the larger block size test at 320MBS. For most environments, 4K, 8K, or 16K are common depending on your application workload. In the charts below the X axis describes the block size tested, ranging from 4KB to 1MB, the Y axis describes the number of IOPS delivered and the Z axis describes the read/write mix tested, ranging from 100% read to 100% write. We like this test specifically because it shows all of the configurations applications present to block storage infrastructure.
|
Test Detail
|
Observed Performance: 1TB 25,000 IOPS |
|---|---|
|
Block Volume Capacity: 1TB
Reproduction Steps
|
|
Note: For 1TB & 2TB the results are almost identical, so only 1TB is included here.
Let's study a smaller 50GB volume, great for starting out when you're sizing for capacity and don't need a large block volume.
| Test Details | Observed Performance: 50GB volume, 3,000 IOPS |
|---|---|
|
Block Volume Capacity: 50GB
Reproduction Steps
|
|
For our 50GB volume, the bandwidth limit is clear for the larger block size tests, and the maximum of 3,000 of 4K IOPS delivered. For small volumes, 4K block size is very common.
I had a chance to run the a test in our newest region in Ashburn. Here's how twenty volumes perform when concurrently attached to the same dense instance but each volume with independent FIO tests. We see an incredible average of 2.1GBPS, and 400,000 IOPS to the host for the 50/50 read/write pattern.
| Test Details | Observed Performance: 1TB x 20 volumes, 400,000 IOPS |
|---|---|
| Block Volume Capacity: 1TB x 20 10GB working set Direct I/O Host Shape: Dense Region: Ashburn Reproduction Steps
|
|
Not sure about these results? Don’t just take our word for it, all of these tests were run using Gartner Cloud Harmony test suite. If you wanted to get these results yourself, it would normally cost approximately $88 to run. Right now we are offering all new customers a $300 credit to our cloud, so validating this performance costs you nothing!
To summarize, the block storage service is delivering IOPS and throughput that scales linearly to a peak of 25K IOPS per volume. We are delivering this performance in a consistent manner will little variability. We are providing our customers with the ability to validate our numbers and our competitor’s offerings with an "apples to apples” comparison test. It is a great day to be a block volume service customer, don’t wait take advantage of this service.
I would love to hear your feedback on your most challenging high availability and performance sensitive environments. We think the performance enabled with this update is the perfect match for mission critical workloads with difficult performance requirements, existing databases, and applications. We want to ensure your success, if you want more information on our performance methodology, have questions on specific workloads or need help achieving similar results, please reach out to me at lee.gates [-at-] oracle.com.