Power Management with Exadata X11M: Maximizing Performance and Efficiency

Oracle Exadata Database Machine (Exadata) is an engineered system that integrates hardware and software with advanced database-aware algorithms to run Oracle Databases optimally for AI, analytics, and mission-critical OLTP workloads. With Exadata X11M, the platform continues to provide customers with extreme scale and performance and the choice and flexibility to deploy wherever they need – on-premises, hybrid cloud with Cloud@Customer, and Exadata Database Service in all major public clouds (OCI, Microsoft Azure, Google Cloud, and Amazon AWS). Exadata X11M adopts the latest AMD two-socket 96-core processor and the latest AMD one-socket 32-core processor. Each 96-core processor draws 400 Watts of power for a total of 800 Watts. Each 32-core processor draws 210 Watts of power. That is a lot of power draw when multiple servers are combined in one rack of Exadata.

In the previous sustainability blog, we covered the Exadata features that help reduce energy consumption and environmental impact while maintaining high performance for all workloads. Exadata X11M adopts a new generation of hardware and system software innovations capable of further reducing energy consumption. This blog covers all the latest Exadata X11M power-saving capabilities we provide customers to achieve their sustainability goals further.

Motivation

It is more important than ever for our customers to address environmental concerns and sustainability. Customers must comply with complicated regulatory frameworks, minimize their impact on the environment, and save on operating expenses. To meet the sustainability goals of today, Exadata X11M helps customers lower energy consumption and, thereby, operating expenses. With rising energy prices, consuming more energy also comes with environmental impact. In today's world, environmental sustainability is a challenge, and people are concerned with ways to meet environmental and sustainability goals. Many governments have introduced stricter regulations and guidelines to curb power consumption.

Exadata X11M comprises multiple database and storage servers and network switches that provide unprecedented performance. With this extreme performance, one might wonder about the power consumption of Exadata infrastructure. Customers have varied requirements to run workloads, and Exadata X11M power optimization capabilities can reduce power for different workload types.

Flexibility to select appropriate database servers

In partnership with processor vendors, Oracle proactively brings solutions to control power consumption around its infrastructure. Exadata X11M is no exception and has introduced capping power consumption on database servers. Exadata X11M introduces a flexible option, a one-socket 32-core CPU database server - known as the X11M-Z Database Server. Customers can choose to run appropriate workloads on the one-socket database server and save on power. The X11M-Z database server consumes 210 Watts of power, 590 Watts less than a 2-socket database server (chart below). Customers with low compute workloads must evaluate the X11M-Z database server to see if that satisfies their workload requirement.

Figure 2: Power savings of an X11M-Z database server compared to a 2-socket X11M database server

Intelligently turn off unneeded CPU cores

Exadata X11M packs 192 cores in a standard database server. Many times, this many cores are not required to run the workload. A significant number of customers license the database software by number of cores rather than an unlimited license agreement (ULA) or perpetual unlimited license agreement (PULA). Oracle has long provided the ability to constrain the number of cores to comply with the licensing using Capacity-On-Demand. Capacity-On-Demand lets customers set the number of active cores on a database server to the licenses procured. In the prior generations of Exadata, enabling lower number of cores did not provide any power savings to the Customer. Exadata X11M hardware, combined with the latest Exadata System Software reduces the power consumed by the database server CPUs when the active cores are set to 128 or lower. When set in such a manner, Exadata intelligently and automatically disables 64 cores (32 per CPU) which results in approximately 80 Watts of energy savings per server.

One can set the lower core count using the following command:

                             DBMCLI> ALTER DBSERVER PendingCoreCount[1] = {target core count};

Figure 1: Intelligently turn off unneeded CPU cores

Intelligently cap power consumption

Another approach to curb energy consumption is to set the power usage cap on the database servers. This enables customers to run the CPUs at a specified wattage reducing power consumption. Each CPU in a 2-socket database server consumes 400 Watts of power, for a total of 800 Watts for two processors. The target Watts specified for the database server can range between 400 Watts and 800 Watts and be increased or decreased dynamically in 40 Watts intervals. Once set, this power setting option remains in effect until the setting is changed.

                               DBMCLI> ALTER DBSERVER CPUTargetWatts = {target CPU usage limit in Watts};

Setting the power cap is easy; let us look at the impact. The performance of a database server is directly proportional to the power it uses. When the CPU Watts of a database server are capped, the server’s performance linearly reflects that cap. For example, a reduction of 80 Watts on a database server (10% of the total 800 Watts) will also reduce performance by 10%.

Figure 3: Performance changes linearly when CPU Watts is increased or decreased.

Intelligently optimize power for low usage periods

Sometimes, based on business requirements, workloads take a breather and do not require the power of the entire machine; for example, many businesses slow down on weeknights and weekends. These periods are good candidates for curbing power consumption. By intelligently enabling a low power mode during these “off-peak” periods, power consumption is automatically reduced. If workload increases unexpectedly during low power period, Exadata automatically exits low power mode to ensure the workload gets the full performance of the CPUs.  Low power mode reduces the power used by database server by 50 Watts.

To set a low-power mode schedule to activate during "off-peak" periods, use the following command:

                                  DBMCLI> ALTER DBSERVER lowPowerModeSchedule   (startTimestamp={}, durationMinutes={1-1440}, frequency={daily|weekly}), …);

For more command details and examples, refer to the documentation here

To come out of the low-power settings when workloads need more power, use the following command:

                                     DBMCLI> ALTER DBSERVER lowPowerModeUntil = { timestamp | NEVER | “”};

• manually switch on the low power mode until a specified time
• NEVER option turns off low-power mode permanently
• “” forces immediate exit of low-power mode until next scheduled interval

Finally, Exadata automatically exits the low-power mode schedule when one of the following metrics is observed for a 5-minute sample.

• CPU utilization exceeds 30%
• I/O utilization exceeds 30%
• Throughput on the client network or RDMA Network Fabric exceeds 200 MB/s.

Exadata will automatically resume low power mode at the next scheduled time.

Conclusion

Exadata X11M is a powerful database machine designed with extreme performance and sustainability in mind. Exadata X11M provides the ability to reduce energy consumption by using the flexible X11M-Z database server, fewer cores, setting target power consumption, and setting low power usage periods, all of which help customers lessen their environmental impact while running high-performance workloads. The performance of Exadata also enables more workloads with more data and more users to be run on fewer systems reducing the number of database and storage servers needed, thereby increasing data center efficiency. Exadata maximizes Oracle Database performance, empowering organizations to accelerate innovation and drive towards a more sustainable future through reduced energy consumption.