X

News, tips, partners, and perspectives for the Oracle Linux operating system and upstream Linux kernel work

Learn How Your OpenStack Cloud Can Perform Better

Dilip Modi
Product Manager

Authors: Matt Lord, Mikael Ronström, Octave Orgeron, Dilip Modi
 

OpenStack is evolving and improving in several areas to address the scalability required for production cloud deployments. For example:

1) Neutron and Nova improvements and tuning have reduced the delays when building and tearing down VLANs.

2) Horizontal auto scaling -- or turning on/off  more VMs – By using  Ceilometer to monitor workloads in VMs you are able to trigger  Heat which can  automatically expand or contract the VM count.

3) More choices of clustering technology for the OpenStack database backend.

In this blog, we will discuss the choices of clustering technology to improve the scalability and performance of OpenStack cloud deployments.  Next month we’ll cover tuning recommendations for improving Network Scalability.

Several OpenStack services require highly performant read/write transactions into databases supporting those services.  As the size of the deployment increases, performance demands on the database can increase rapidly.  To support production workloads, OpenStack database nodes must be able to scale up or down automatically without downtime or performance hits incurred by restructuring on-disk data.

Galera/InnoDB is the widely used database backend for OpenStack, largely because Galera adds redundancy with near zero additional complexity. However, for large OpenStack deployments, Galera's scaling properties introduce tradeoffs between performance and consistency, resulting in higher total cost of ownership for the cloud operator and reduced control plane performance for cloud users.

To address these scaling issues, Oracle OpenStack employs MySQL Cluster with NDB storage engine for the database backend.  MySQL Cluster was originally built at Ericsson for the high availability and highly consistent requirements of telecom network workloads, where there was a strong focus on write scaling.  It's designed around a distributed, multi-master ACID compliant architecture with no single point of failure.  MySQL Cluster uses automatic sharding to scale out read and write operations.

A few technical hurdles had to be overcome to deploy MySQL Cluster/NDB for OpenStack:

  • Table row length is limited to 14k in NDB vs 65k in InnoDB.
  • Hard coding of SQL statements such as "mysql_engine=InnoDB" used in the SQLAlchemy and Alembic migration scripts
  • MySQL Cluster (NDB) uses a strict adherence to ordering of SQL operations for foreign key constraints and indexes to ensure consistency.
  • No support for Savepoints or Rollbacks in MySQL Cluster (NDB)
  • No support for nested transactions in MySQL Cluster (NDB)

The Oracle OpenStack team has been working on addressing these challenges using innovative methods to reduce the impact on the OpenStack projects upstream. Our approach:

  • Extend the functionality of oslo.db to automatically handle the table schema changes, the mysql engine table setting, and the disablement of unsupported features. This is activated via a single configuration setting called "mysql_enable_ndb", which will be available to any OpenStack service to consume.
  • Improve the SQLAlchemy and Alembic scripts that contain tables that need the columns adjusted to fit within the row limit of MySQL Cluster (NDB). This is accomplished using the oslo.db function to wrap around the selected column’s string data type and pass arguments that are used to dynamically change the column size or type when using MySQL Cluster (NDB).
  • Correct SQLAlchemy and Alembic scripts to handle the ordering of dropping and recreating foreign key constraints when making table or index changes.
  • Disable the use of nested transactions when using MySQL Cluster (NDB).
  • Enhance OpenStack services to help ensure they are using the oslo.db framework when doing database creations, upgrades, and migrations.

The chart below compares MySQL Active/Active Clustering and Galera clustering. MySQL Active/Active Clustering provides many advantages including High Availability for OpenStack services and the performance and scalability required as the OpenStack deployment size increases. With support for MySQL Active/Active Clustering, Oracle OpenStack is the best choice for an enterprise-class OpenStack cloud solution.

 

 

   Other OpenStack:

  A/A Galera Clustering

Oracle OpenStack:

 A/A MySQL Clustering 

Design Objectives

HA Only

HA, Performance,  Scalability

Write scaling on the order of

30 thousand TPS

20 million TPS

Max data nodes

n/a

48

Max MySQL nodes

9

approx 200

Two-level replication

Yes

Yes

Multi-master within shard

Yes †

Yes

Automatic, transparent
sharding capability

No

Yes
(up to 24 shards)

All MySQL server nodes
guaranteed to see most
recent state for all queries

No

Yes

Predictable latency

No

Yes

Online DDL

No

Yes

NoSQL APIs

No

Yes

Load balancing

No

Yes

Low latency in-memory

No

Yes

Self healing capabilities

No

Yes

Fully synchronous

Virtually synchronous

Yes

† Galera multi-master incurs performance penalties and potentially aborted commits.

For more information on Oracle OpenStack, visit
https://www.oracle.com/linux/openstack/index.html

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.