The What, Why, and When of Oracle Exadata

Juan Loaiza, senior vice president of systems technology at Oracle, provides an inside look at the latest Oracle Exadata machines.

By Tom Haunert

March/April 2011

Oracle’s Exadata Database Machine X2-8, a system for high-volume transaction processing, is the newest member of the Oracle Exadata product family, which also includes Exadata Database Machine X2-2. Tom Haunert, Oracle Magazine editor in chief, talked with Juan Loaiza, senior vice president of systems technology at Oracle, about how Oracle’s engineered systems have evolved since the introduction of Oracle Exadata in 2008. The following is an excerpt from that interview. Download the full podcast at

Oracle Magazine: What do engineered systems in general and Oracle Exadata Database Machine in particular mean for DBAs and system administrators?

Loaiza: When you deploy a new system, a lot of time has to be spent assembling the system, debugging it, and getting it ready to run. Then you have to go through a phase of performance optimization where you figure out what the bottlenecks are. And then you have to go through failure mode testing, where a system is tested across every different kind of failure.

One really nice thing about an engineered system like Oracle Exadata is that it comes assembled, debugged, and ready to run. We’ve tuned all the components—everything from firmware to OS [operating system] drivers to Oracle Database configuration settings—so that you can achieve the full performance potential on Day One. And we’ve done all the failure mode testing—pulling network cables and power cables, pulling disks out, crashing nodes, and crashing databases. So deployment is a whole lot faster and easier and a lot less risky.

One of the interesting things about Oracle Exadata is that every system is identical. All customers are running the exact same configuration, from the exact same model of disk drives all the way to the servers and the DRAM [dynamic random access memory]. When a customer encounters an issue that is unique to their configuration, it typically takes a long time to solve. Oracle Exadata removes the unique issues. Furthermore, the customer Oracle Exadata system is identical to the systems that we run at Oracle.

Another interesting thing is that Oracle Exadata has the largest user community of any complete platform that Oracle Database runs on. That means there’s a lot of platform expertise in the community and also within Oracle. Oracle support is very accustomed to getting calls about Oracle Exadata systems; they’ve become experts on Oracle Exadata systems, and they have the exact systems in their support labs.

And, because it all comes from Oracle—from the hardware all the way to the software—you get full end-to-end support, so there’s no finger-pointing between vendors, with the customer stuck in the middle.

So from all those perspectives, there are a lot of benefits for DBAs and system administrators because there are a lot fewer hassles—during deployment and operations—and there’s a lot more support from the community. What it really means is less tuning and troubleshooting of low-level issues and a lot more getting important work done.

Oracle Magazine: What should DBAs and system administrators consider when choosing between Exadata Database Machine X2-2 and Exadata Database Machine X2-8?

Loaiza: To a large extent, it’s based on preference. The X2-8 is based on large SMP [symmetric multiprocessing] servers that have eight sockets and 64 cores each, and the X2-2 is based on small database servers with two sockets each. We recommend the X2-8 servers for large OLTP [online transaction processing] or very large Oracle Exadata configurations in general. Customers that currently run their Oracle systems on very large SMPs are going to really like the X2-8. It brings lots of cores, a very large memory configuration, and fewer systems to manage.

On the other hand, customers that are comfortable with small servers will be very happy with the X2-2. One benefit of the X2-2 is that it scales down to smaller workloads than the X2-8. Since the X2-8 only has two servers per rack, you can’t really reduce the size and maintain a highly available configuration. With an X2-2, we can fit eight servers per rack along with all the storage, so we can scale that down to a half rack or a quarter rack. So organizations with smaller workloads will tend to use an X2-2.

Oracle Magazine: How can DBAs and system administrators predict, in advance of moving to Oracle Exadata, the types of improvements and performance that they’re going to see?

Loaiza: There are a number of ways to predict the improvements. One is to measure the bandwidth of the current storage subsystem and compare it to the bandwidth of Oracle Exadata; this is especially useful for bandwidth-intensive applications like data warehousing. Another tool that many customers have been using successfully is Oracle Real Application Testing. It can capture a real production workload, including all the SQL statements and all the timing, and replay that workload on a different platform. It’s particularly useful in OLTP because customers generally have a hard time reproducing complex workloads. So we can capture a complex OLTP workload running on any platform that Oracle Database runs on, and then replay that workload on an Oracle Exadata platform and see what benefit the customer will get.

For data warehousing, we often capture the customer queries or reports that take the longest to run. We’ll work with a customer to migrate their data to an Oracle Exadata system and then take the top dozen to hundred slowest queries and run those queries on Oracle Exadata and measure the performance improvements.

Oracle Magazine: Speaking of performance, the predicted performance when running on Oracle Exadata is extreme and the performance numbers are impressive, but how is Oracle Exadata able to achieve these incredible numbers with largely industry-standard technologies?

Loaiza: There is a huge amount of hardware power in Oracle Exadata. We’re using the latest Intel processors in a grid architecture. We have 22 separate servers in an Oracle Exadata rack, more than 200 processing cores, and terabytes of memory and flash.

In addition, we have three really special capabilities in Oracle Exadata. One is our intelligent storage architecture—the ability to push database operations directly into storage. Another is our compression capability. We are often able to achieve 10 times compression of data in Oracle Exadata, which is far more than we can achieve on any other platform. The third special capability is the way we’ve integrated flash very tightly into Oracle Exadata, both in the software and hardware. We are using PCI flash cards instead of flash disks because they run much faster. We have a feature called Smart Flash Cache that automatically moves data in and out of flash to achieve really good performance without requiring manual placement of data in flash.

Oracle Magazine: InfiniBand is another key technology in Oracle Exadata. What is it, how is it engineered into Oracle Exadata, and what performance benefits does it bring?

Loaiza: The latest trend in networking is unified networking. Unified networks use a single network for both server-to-server communication and server-to-storage communication. In Oracle Exadata, we have a unified network based on InfiniBand, which is the fastest network technology on the market, running at 40 gigabits per second. We use that unified network to communicate both between the database servers and between the database and storage. InfiniBand also gives us zero data loss transmission, and RDMA [remote direct memory access] to greatly reduce the amount of CPU utilized to transfer data across the network. The performance, reliability, cost, and capabilities of InfiniBand are really exceptional. Also, InfiniBand is really simple to manage. It looks very much like an Ethernet network from an administrative point of view.

Oracle Magazine: How has feedback from DBAs and other administrators driven the evolution of Oracle Exadata?

Loaiza: We have made a number of changes in Oracle Exadata based on user feedback. One thing that we got a lot of feedback on was the default security of Oracle Exadata. Initially we shipped Oracle Exadata with a default OS configuration that had lots of capabilities installed by default. This sometimes led to security vulnerabilities in software packages the customer was not even using.

In our later releases, we stripped down the OS to just what we needed to run Oracle Exadata and the database, and then we allowed customers to add just the additional packages that they want to their environment.

Another piece of feedback was that users wanted an integrated call home capability. So we’ve deployed an Automatic Service Request capability that logs a trouble ticket directly with Oracle when there is a hardware issue.

Oracle Magazine: What have you seen in the adoption of Oracle Exadata in its first couple years of release, and what do you see in the future for Oracle Exadata adoption?

Loaiza: We’ve seen great adoption of Oracle Exadata from Day One. There’s been a lot of interest in the whole concept of an engineered system—a database machine—that is specifically designed and optimized around running a database. There is a lot of interest in the special capabilities of Oracle Exadata and the benefits they bring to businesses. That’s what’s driven the adoption of Oracle Exadata so far, and that’s what we think will continue to drive it into the future.

Oracle Exadata has huge benefits in cost, performance, management, and availability. So the question that a lot of customers are starting to ask is not “Why Oracle Exadata?”; it’s “Why not Oracle Exadata?” or “When Oracle Exadata?” Over the next five or so years, we expect a majority of large Oracle Database customers to switch to Oracle Exadata. The benefits are that compelling, and the interest is that great.

Next Steps

 LEARN more about Oracle Exadata Database Machine

 LISTEN to the podcast


Photography by Tim Scharner, Unsplash