The MySQL NDB Cluster Development team is happy to announce MySQL NDB Cluster 8.4 (LTS) is available for download. This release includes new features made available under Innovation Releases – 8.1, 8.2 and 8.3 – and new features released under 8.4 which are described below. MySQL NDB Cluster binary and source packages are available to download from MySQL NDB Cluster Community Downloads or MySQL NDB Cluster Commercial Downloads.
Security
TLS support in MySQL NDB Cluster, providing authentication and encryption.
The continued emphasis on improving the security of MySQL NDB Cluster produced additional improvements in this area. TLS enables authentication and data-in-motion encryption between all node types: management, data and API/SQL nodes. You can read more in Online MySQL Cluster Documentation and go deeper in the Introducing NDB TLS blog post.
Replication Performance
Multi-threaded-applier (MTA)
MySQL NDB Cluster asynchronous replication records and applies a binary log containing a serial sequence of transactions. Each transaction contains row changes, applied in parallel to table shards in a replica cluster’s data nodes. MTA allows independent and therefore, parallelizable transactions to be identified so that they can be applied in parallel by separate worker threads on the replica. This extra dimension of parallelism can increase peak replication throughput.
Replica Updates Filter
In some asynchronous replication topologies, replicated row changes are not recorded in a replica’s binary log. Filtering out of such row changes previously occurred in binlogging MySQL Servers attached to a replica cluster. The Replica Updates Filter functionality moves this filtering to the MySQL NDB Cluster data nodes, saving wasted resources by forwarding these changes to MySQL Servers for filtering.
Replica Update Filter is accomplished by setting –log-replica-updates=OFF while at the same time filtering such pushes to NDB, avoiding them from being written to the binlog.
For example, the efficiency increases 4x in a four site configurationfor the following metrics:
- Network traffic from NDB to MySQL
- Receive thread CPU
- NDB binlog thread CPU
NDB Plugin
NDB Binlog Injector memory management refactoring
The MySQL Server connects to NDB using the ndbcluster plugin which implements the functionality to modify and query data (DML), perform schema changes (DDL) and write changes in NDB to the binary log.
The work done at the NDB binlog injector memory management is internal only but should provide a noticeable improvement by saving on memory release and re-allocation over successive epochs.
General Robustness
Overload controls improvements for event streaming / change data capture and schema synchronization.
Several improvements to subscriber management, asynchronous replication, and schema synchronization makes NDB more robust to support deployments over different environments, such as private/public cloud native environments.
Observability / Operability
MultiTransporters Observability Improvement
We have introduced a new ndbinfo table, specifically the transporter_details table with information per transporter instance. This table is similar to the transporters table but provides information about individual transporters rather than in the aggregate.
SendBuffer usage monitoring
Exposing information about current send buffer memory usage, whether an overload limit is being crossed, how memory allocated to the send buffer is shared among different transporters, and the historical maximum usage of send buffer memory is quite important to determine the proper configuration of the send buffer. This enables better fine tuning of overall memory usage, avoiding waste or misconfigured setting of the send buffer, which can affect stability.
Considering the above, the ndbinfo.transporter_details table has been extended with four new columns, as follows:
- sendbuffer_used_bytes
- The number of bytes of signal data stored is pending sending using this transporter.
- sendbuffer_max_used_bytes
- The historical maximum number of bytes of signal data stored is pending sending using this transporter. Reset when the transporter connects.
- sendbuffer_alloc_bytes
- The number of send bytes is currently allocated to the store pending sending bytes for this transporter. Send buffer memory is allocated in large blocks, which may be sparsely used.
- sendbuffer_max_alloc_bytes
- The historical maximum number of bytes of send buffer was allocated to store pending sending bytes for this transporter.
Improved visibility over API Heartbeats
APIs monitor data nodes using heartbeat signals and may autonomously disconnect from data nodes they consider to have failed. This decision is now logged to improve system observability.
Log STOP REPLICA and START REPLICA commands
Under troubleshooting and debugging scenarios, knowing when the replica is stopped or started, makes the analysis simpler.
Having the proper logs populated with the correct information, like when START REPLICA and STOP REPLICA are executed, helps in figuring out many field issues.
NdbFS activity and performance monitoring
Modern on-prem and cloud environments have a wide range of OS and file system behaviors. MySQL NDB Cluster is adapted to these environments by improving its robustness to non-standard file system semantics, observability and logging.
ClusterJ
The setLimits()method can now be chained to deletePersistentAll() to limit the number of items to delete.
MySQL Cluster Manager (MCM)
MySQL Cluster Manager simplifies MySQL NDB Cluster administration, allowing operations like creating a cluster, backing up/restoring, changing NDB configuration, rolling out software updates, and more.
MCM Security has been improved in line with MySQL NDB Cluster. MCM has been enhanced to enable the configuration of the new MySQL NDB Cluster TLS security features. In addition, MCM can also use TLS within the MCM cluster itself.
Apart from this, several additions were also developed, like the possibility to start and start processes individually: “start process <nodeid-list>|<nodetype>“ command.
NDB Kubernetes Operator
Support NDB TDE in NDB K8s Operator
MySQL NDB Cluster brought Transparent Data Encryption as a security improvement.
NDB K8s Operator now supports MySQL NDB TDE feature as well.
Installation
Operator Lifecycle Manager (OLM) : The NDB K8s Operator has now been certified as a Red Hat OpenShift certified operator, making it available in the curated catalog of OpenShift operators, as well as being available from OperatorHub.io.
The OpenShift certification process involved several validation procedures to ensure that the NDB K8s Operator meet the required standards.
Documentation
MySQL NDB Cluster documentation has been enhanced to describe how tables and indexes use memory, including how to monitor and estimate memory footprint. For more information, see the section in the online documentation entitled The ndbinfo memory_per_fragment Table.