By John Soat
In an enterprise technology world featuring too much hot air and not enough world-class technology, SAP has done an excellent job spreading the myth that its five-year-old HANA database delivers game-changing performance for its customers’ businesses. But SAP’s problem is that HANA’s performance simply can’t match the marketing hyperbole SAP is spreading around.
For those of you whose companies are running SAP’s applications, the objective reality is that most of those applications can perform much better on Oracle Database than on any other database platform, including HANA. We’ll demonstrate that disparity, and the controversy around it, using the results from SAP’s own standard applications performance benchmarks.
Meanwhile, SAP would be happy for customers to believe one myth after another as it tries to send customers on a forced march to HANA. Here are SAP’s two main myths:
Myth #1: SAP’s HANA is an in-memory database, so it must be blazingly fast running all of SAP’s business applications.
Fact: HANA is an in-memory column store database. It is optimized for running analytics applications, not operational (transaction processing) applications.
Evidence: In the last 20 years, Oracle has published more than 250 results for SAP standard application benchmarks on Oracle Database. These benchmarks cover a wide range of SAP operational and analytics applications, including sales, distribution, production, and others.
Unfortunately, SAP won’t allow such a head-to-head comparison to be made to HANA because SAP has not published a single benchmark result for any of its transaction processing applications running on HANA. Why not? Customers should ask SAP: What are you trying to hide?
The results of Oracle’s recent BW-EML benchmark demonstrate that Oracle Database In-Memory runs this benchmark twice as fast as HANA does.”
Myth #2: OK, so SAP won’t reveal HANA’s performance running operational applications. But HANA is still way faster than Oracle at running analytics applications.
Fact: The new in-memory column store technology integrated into Oracle Database 12c runs analytics applications much faster than HANA does.
Evidence: SAP formulated a new benchmark, BW-EML, in 2012 to evaluate the performance of databases running its Business Warehouse (BW) analytics application. The benchmark is intended to measure typical demands made on the SAP Business Warehouse database from users generating real-time reports and ad hoc queries.
This is the only SAP standard application benchmark for which SAP publishes results with HANA. And the results of Oracle’s recent BW-EML benchmark demonstrate that Oracle Database 12c In-Memory runs this benchmark twice as fast as HANA does. Period.
More than two-thirds of SAP’s midsize and large customers run their applications on Oracle databases.”
For example, Oracle’s latest world record result, run earlier this year using Oracle Database 12c, involves the SAP Standard Application Sales and Distribution (SD) Benchmark. The results, certified by SAP (see Certification 2015050), demonstrated 30,800 SD benchmark users, an average dialog response time of 0.96 seconds, and more than 3 million fully processed order line items per hour.
SAP maintains that Oracle manipulated the benchmark and refuses to certify Oracle’s results (more on that below). Could it be because SAP now has “a horse in the race”—that is, its own competing HANA database? asks Maria Colgan, master product manager for Oracle Database In-Memory.
SAP Spreads FUD
Embarrassed by the BW-EML benchmark results Oracle published, SAP responded by spreading additional fear, uncertainty, and doubt about Oracle’s database technology. In the interest of transparency, we lay out that FUD—and Oracle’s rebuttals—below.
FUD factor #1: Oracle manipulated its BW-EML benchmark by using a “custom setup” involving database functions known as “triggers” and “materialized views” that can lead to hard-to-spot data inconsistencies and aren’t supported in real-world production environments.
Not true. “Both of those features have been in Oracle Database since early on,” Colgan says. “And customers have been using triggers and materialized views in databases for years without any data inconsistency.”
Nevertheless, to disprove SAP’s accusation Oracle reran the benchmark without using triggers—resulting in no loss of performance—and has submitted those results to SAP for certification. (SAP has yet to respond.)
FUD factor #2: Oracle misrepresented the true number of processors it used in the test by running it on its Oracle Exadata Database Machine, which incorporates storage servers that can be used for database processing.
More dissembling. “In this benchmark, all data was in memory, in the compute servers,” Colgan says. “The storage servers were not used in the benchmark at all—and SAP knows that.”
FUD factor #3: An Oracle Exadata Database machine is much more expensive than a commodity Dell or Lenovo server that databases often run on.
“Sure, it is,” Colgan says. But for many enterprise workloads, the performance advantages provided by Oracle Exadata machines—optimized hardware-software systems preconfigured with compute, storage, and networking hardware—more than make up for the difference.
SAP customers “are a little bit terrified” of the forced march to HANA.”– Maria Colgan, Master Product Manager, Oracle Database In-Memory
FUD factor #4: SAP has more than 7,000 HANA customers, while Oracle has zero customers running SAP applications on Oracle Database In-Memory.
More than two-thirds of SAP’s midsize and large customers run their applications on Oracle databases. Oracle Database In-Memory, introduced in 2014, is one of Oracle’s most popular technology options. SAP certified Oracle Database In-Memory last June for use with its applications, and many SAP applications customers now have it in testing. Several production Oracle Database In-Memory customers spoke at Oracle OpenWorld 2015 in October, and a presentation about a major wireless carrier’s use of Oracle Database In-Memory explained why the company chose it over HANA to run its SAP Business Objects business intelligence reporting tool.
Colgan agrees that many SAP customers have bought HANA, but she questions how many actually are using it, given the fact that SAP throws in HANA as part of deep-discount programs for its enterprise applications. Those discounts enticed customers to acquire HANA but “didn’t require them to implement it,” she says.
FUD factor #5: Oracle Database In-Memory is more complex than SAP HANA.
Complexity is the crux of the debate. This is probably the point to pay the closest attention to.
Oracle Database In-Memory works in conjunction with the relational engine in Oracle Database 12c. It’s a dual-format approach that optimizes both analytics and mixed-workload transaction processing, delivering outstanding performance for transactions while simultaneously supporting real-time analytics, business intelligence, and reporting, all without requiring any changes to applications that are compatible with Oracle Database 12c.
In its response to Oracle’s SAP BW-EML benchmark running on Oracle Database In-Memory, SAP called out the effort required to make this mixed-workload environment work: “To analyze the data faster, database administrators must manually duplicate tables from disk into memory. Then administrators must selectively drop analytical indexes, so they do not impact transactional application performance.”
That’s true enough, Colgan says, but the effort isn’t nearly as labor-intensive for customers as SAP makes it out to be. “It’s a one-time thing,” she says. “It’s not an ongoing effort that’s going to make their lives difficult.”
Nervous SAP Customers
SAP started life as a business applications vendor, pioneering enterprise resource planning (ERP) capabilities that manage financial, procurement, manufacturing, and other company processes. SAP’s applications were “platform agnostic,” intended to run on a variety of databases—mostly Oracle’s, as it turned out.
Now SAP is betting its future on HANA. The company introduced its first HANA-native application suite, SAP S/4HANA, earlier this year. SAP executives proselytize their new technology with a certitude that has some people nervous. “The customers are a little bit terrified,” Colgan says.
That’s because SAP maintains that S/4HANA is the future, and “SAP on Oracle will only be supported until 2025,” according to John Appleby, SAP blogger and GM of SAP consultancy Bluefin Solutions, forcing most customers to move off their existing databases. “What they’re saying to customers is that you will all eventually move to this S4 platform and you will all eventually have to move to SAP HANA,” Colgan says.
Such a forced replatforming is a bitter pill for customers to swallow, Colgan points out, mostly because “it’s a huge, risky endeavor, and a painful one.” Implementing new applications or a new database takes the average customer six months, Colgan says—much longer for large, global deployments.
Along with the investment in new hardware and software, the forced march involves a steep learning curve for a company’s IT staff. “I’d need to reskill all of my DBA team in order to be able to adopt and manage this new technology,” she says.
Those pain points help explain why less than half of more than a thousand SAP customers surveyed this year by ASUG (the Americas’ SAP Users’ Group) have purchased HANA. And of those who haven’t, nearly 60 percent say it was “due to lack of a business case that justifies [the] investment” in HANA.
It points to why Oracle Database 12c remains a more viable option for SAP customers going forward. Says Colgan: “Having Oracle Database In-Memory as a go-faster switch underneath your application without requiring any upgrades or changes to the app is far more palatable for customers.”
SAP spreads fear, uncertainty, and doubt
|SAP FUD: Oracle manipulated the benchmark of an SAP analytics application running on Oracle Database 12c In-Memory by using a “custom setup” with database triggers and materialized views that aren’t supported in real-world production environments.||Oracle Rebuttal: Both features, which have been included in Oracle Database for years, are used regularly by customers. There’s nothing “custom” about them.|
|SAP FUD: Oracle’s “custom” trigger-and-materialized-views setup can lead to hard-to-spot data inconsistencies.||Oracle Rebuttal: A large percentage of Oracle’s customers take advantage of triggers in a variety of different systems without introducing any data inconsistency.|
|SAP FUD: Oracle misrepresented the true number of processors it used in the benchmark by running the benchmark on its Oracle Exadata Database Machine, which incorporates storage servers that can be used for database processing.||Oracle Rebuttal: All data used in Oracle’s benchmark was processed in memory, in the compute servers. No data was processed in the storage servers. None.|
|SAP FUD: Oracle Exadata Database Machine is much more expensive than a commodity Dell or Lenovo server, which are often used to run databases.||Oracle Rebuttal: The performance advantages provided by Oracle Exadata Database Machine—optimized hardware/software systems that are preconfigured with compute, storage, and networking hardware—more than make up for the difference.|
LEARN more about the benefits of Oracle Database In Memory.
READ more about why people run SAP on Oracle Database.