Tuesday Nov 12, 2013
Wednesday Sep 04, 2013
Oracle Data Miner (Extension of SQL Developer 4.0) Integrate Oracle R Enterprise Mining Algorithms into workflow using the SQL Query node
By chberger on Sep 04, 2013
I posted a new white paper authored by Denny Wong, Principal Member of Technical Staff, User Interfaces and Components, Oracle Data Mining Technologies. You can access the white paper here and the companion files here. Here is an excerpt:
Oracle Data Miner (Extension of SQL Developer 4.0)
Integrate Oracle R Enterprise Mining Algorithms into workflow using the SQL Query node
Oracle R Enterprise (ORE), a component of the Oracle Advanced Analytics Option, makes the open source R statistical programming language and environment ready for the enterprise and big data. Designed for problems involving large amounts of data, Oracle R Enterprise integrates R with the Oracle Database. R users can develop, refine and deploy R scripts that leverage the parallelism and scalability of the database to perform predictive analytics and data analysis.
Oracle Data Miner (ODMr) offers a comprehensive set of in-database algorithms for performing a variety of mining tasks, such as classification, regression, anomaly detection, feature extraction, clustering, and market basket analysis. One of the important capabilities of the new SQL Query node in Data Miner 4.0 is a simplified interface for integrating R scripts registered with the database. This provides the support necessary for R Developers to provide useful mining scripts for use by data analysts. This synergy provides many additional benefits as noted below.
· R developers can further extend ODMr mining capabilities by incorporating the extensive R mining algorithms from the open source CRAN packages or leveraging any user developed custom R algorithms via SQL interfaces provided by ORE.
· Since this SQL Query node can be part of a workflow process, R scripts can leverage functionalities provided by other workflow nodes which can simplify the overall effort of integrating R capabilities within the database.
· R mining capabilities can be included in the workflow deployment scripts produced by the new sql script generation feature. So the ability of deploy R functionality within the context of an Data Miner workflow is easily accomplished.
· Data and processing are secured and controlled by the Oracle Database. This alleviates a lot of risk that are incurred by other providers, when users have to export data out of the database in order to perform advanced analytics.
Oracle Advanced Analytics saves analysts, developers, database administrators and management the headache of trying to integrate R and database analytics. Instead, users can quickly gain the benefit of new R analytics and spend their time and effort on developing business solutions instead of building homegrown analytical platforms.
Monday Jul 15, 2013
Oracle Data Miner GUI, part of SQL Developer 4.0 Early Adopter 1 is now available for download on OTN
By chberger on Jul 15, 2013
The NEW Oracle Data Miner GUI, part of SQL Developer 4.0 Early Adopter 1 is now available for download on OTN. See link to SQL Developer 4.0 EA1.
The Oracle Data Miner 4.0 New Features are applicable to Oracle Database 11g Release 2 and Oracle Database Release 12c: See Oracle Data Miner Extension to SQL Developer 4.0 Release Notes for EA1 for additional information
· Workflow SQL Script Deployment
o Generates SQL scripts to support full deployment of workflow contents
· SQL Query Node
o Integrate SQL queries to transform data
or provide a new data source
o Supports the running of R Language
Scripts and viewing of R generated data and graphics
· Graph Node
o Generate Line, Scatter, Bar, Histogram
and Box Plots
· Model Build Node Improvements
o Node level data usage specification applied to underlying models
o Node level text specifications to govern text transformations
o Displays heuristic rules responsible for excluding predictor columns
o Ability to control the amount of Classification and Regression test results generated
· View Data
o Ability to drill in to view custom objects and nested tables
These new Oracle Data Miner GUI capabilities expose Oracle Database 12c and Oracle Advanced Analytics/Data Mining Release 1 features:
· Predictive Query Nodes
o Predictive results without the need to build models using Analytical Queries
o Refined predictions based on data
· Clustering Node New Algorithm
o Added Expectation Maximization algorithm
· Feature Extraction Node New Algorithms
o Added Singular Value Decomposition and Principal Component Analysis algorithms
· Text Mining Enhancements
o Text transformations integrated as part of Model's Automatic Data Preparation
o Ability to import Build Text node specifications into a Model Build node
· Prediction Result Explanations
o Scoring details that explain predictive result
· Generalized Linear Model New Algorithm Settings
o New algorithm settings provide feature selection and generation
Wednesday May 08, 2013
By chberger on May 08, 2013
Periodically, I've recorded a demonstration and/or presentation on Oracle Advanced Analytics and Data Mining and have posted them on YouTube. Here are links to some of more recent YouTube postings--sort of an
Oracle Advanced Analytics and Data Mining at the Movies experience.
- New - Oracle Business Intelligence Enterprise Edition (OBIEE) SampleAppls Demo featuring integration with Oracle Advanced Analytics/Data Mining
- New - Oracle Big Data Analytics Demo mining remote sensor data from HVACs for better customer service
- In-Database Data Mining for Retail Market Basket Analysis Using Oracle Advanced Analytics
- In-Database Data Mining Using Oracle Advanced Analytics for Classification using Insurance Use Case
- Fraud and Anomaly Detection using Oracle Advanced Anlaytics Part 1 Concepts
- Fraud and Anomaly Detection using Oracle Advanced Analytics Part 2 Demo
- Overview Presentation and Demonstration of Oracle Advanced Analytics Database Option
So.... grab your popcorn and a comfortable chair. Hope you enjoy!
Friday Feb 22, 2013
By chberger on Feb 22, 2013
I wanted to highlight a wonderful new resource provided by our partner Vlamis Software. Extremely easy! Fill out the form, wait a few minutes for the Amazon Cloud instance to start up and them BAM! You can login and start using the Oracle Advanced Analytics Oracle Data Miner work flow GUI. Demo data and online Oracle by Example Learning Tutorials are also provided to ensure your data mining test drive is a positive one, Enjoy!!
We have partnered with Amazon Web Services to provide to you, free of charge, the opportunity to work, hands-on, with the latest of Oracle's Business Intelligence offerings. By signing up to one of the labs below, Amazon's Elastic Cloud Computer (EC2) environment will generate a complete server for you to work with.
These hands on labs are working with the actual Oracle software running on the Amazon Web Services EC2 environment. They each take approximately 2 hours to work through and will give you hands-on experience with the software and a tour of the features. Your EC2 environment will be available for you for 5 hours, at which time it will self-terminate. If, after registration, you need additional time or need further instructions, simply reply to the registration email and we would be glad to help you.
This test drive walks through some basic exercises in doing predictive analytics within an Oracle 11g Database instance using the Oracle Data Miner extension for Oracle SQL Developer. You use a drag-and-drop "workflow" interface to build a data mining model that predicts the likelihood of purchase for a set of prospects. Oracle Data Mining is ideal for automatically finding patterns, understanding relationships, and making predictions in large data sets.
Tuesday Jan 01, 2013
By chberger on Jan 01, 2013
Turkcell İletişim Hizmetleri A.S. Successfully Combats Communications Fraud with Advanced In-Database Analytics
[Original link available on oracle.com http://www.oracle.com/us/corporate/customers/customersearch/turkcell-1-exadata-ss-1887967.html]
- Oracle Customer: Turkcell İletişim Hizmetleri A.Ş.
- Location: Istanbul, Turkey
- Industry: Communications
- Employees: 3,583
- Annual Revenue: Over $5 Billion
Turkcell İletişim Hizmetleri A.Ş. is a leading provider of mobile communications in Turkey with more than 34 million subscribers. Established in 1994, Turkcell created the first global system for a mobile communications (GSM) network in Turkey. It was the first Turkish company listed on the New York Stock Exchange.
Communications fraud, or the use of telecommunications products or services without intention to pay, is a major issue for the organization. The practice is fostered by prepaid card usage, which is growing rapidly. Anonymous network-branded prepaid cards are a tempting vehicle for money launderers, particularly since these cards can be used as cash vehicles—for example, to withdraw cash at ATMs. It is estimated that prepaid card fraud represents an average loss of US$5 per US$10,000 in transactions. For a communications company with billions of transactions, this could result in millions of dollars lost through fraud every year.
Consequently, Turkcell wanted to combat communications fraud and money laundering by introducing advanced analytical solutions to monitor key parameters of prepaid card usage and issue alerts or block fraudulent activity. This type of fraud prevention would require extremely fast analysis of the company’s one petabyte of uncompressed customer data to identify patterns and relationships, build predictive models, and apply those models to even larger data volumes to make accurate fraud predictions.
To achieve this, Turkcell deployed Oracle Exadata Database Machine X2-2 HC Full Rack, so that data analysts can build predictive antifraud models inside the Oracle Database and deploy them into Oracle Exadata for scoring, using Oracle Data Mining, a component of Oracle Advanced Analytics, leveraging Oracle Database11g technology. This enabled the company to create predictive antifraud models faster than with any other machine, as models can be built using search and query language (SQL) inside the database, and Oracle Exadata can access raw data without summarized tables, thereby achieving extremely fast analyses.
A word from Turkcell İletişim Hizmetleri A.Ş.
“Turkcell manages 100 terabytes of compressed data—or one petabyte of uncompressed raw data—on Oracle Exadata. With Oracle Data Mining, a component of the Oracle Advanced Analytics Option, we can analyze large volumes of customer data and call-data records easier and faster than with any other tool and rapidly detect and combat fraudulent phone use.” – Hasan Tonguç Yılmaz, Manager, Turkcell İletişim Hizmetleri A.Ş.
- Combat communications fraud and money laundering by introducing advanced analytical solutions to monitor prepaid card usage and alert or block suspicious activity
- Monitor numerous parameters for up to 10 billion daily call-data records and value-added service logs, including the number of accounts and cards per customer, number of card loads per day, number of account loads over time, and number of account loads on a subscriber identity module card at the same location
- Enable extremely fast sifting through huge data volumes to identify patterns and relationships, build predictive antifraud models, and apply those models to even larger data volumes to make accurate fraud predictions
- Detect fraud patterns as soon as possible and enable quick response to minimize the negative financial impact
Oracle Product and Services
- Used Oracle Exadata Database Machine X2-2 HC Full Rack to create predictive antifraud models more quickly than with previous solutions by accessing raw data without summarized tables and providing unmatched query speed, which optimizes and shortens the project design phases for creating predictive antifraud models
- Leveraged SQL for the preparation and transformation of one petabyte of uncompressed raw communications data, using Oracle Data Mining, a feature of Oracle Advanced Analytics to increase the performance of predictive antifraud models
- Deployed Oracle Data Mining models on Oracle Exadata to identify actionable information in less time than traditional methods—which would require moving large volumes of customer data to a third-party analytics software—and achieve an average gain of four hours and more, taking into consideration the absence of any system crash (as occurred in the previous environment) during data import
- Achieved extreme data analysis speed with in-database analytics performed inside Oracle Exadata, through a row-wise information search—including day, time, and duration of calls, as well as number of credit recharges on the same day or at the same location—and query language functions that enabled analysts to detect fraud patterns almost immediately
- Implemented a future-proof solution that could support rapidly growing data volumes that tend to double each year with Oracle Exadata’s massively scalable data warehouse performance
“We selected Oracle because in-database mining to support antifraud efforts will be a major focus for Turkcell in the future. With Oracle Exadata Database Machine and the analytics capabilities of Oracle Advanced Analytics, we can complete antifraud analysis for large amounts of call-data records in just a few hours. Further, we can scale the solution as needed to support rapid communications data growth,” said Hasan Tonguç Yılmaz, datawarehouse/data mining developer, Turkcell Teknoloji Araştırma ve Geliştirme A.Ş.
Oracle Partner: Turkcell Teknoloji Araştırma ve Geliştirme A.Ş.
All development and test processes were performed by Turkcell Teknoloji. The company also made significant contributions to the configuration of numerous technical analyses which are carried out regularly by Turkcell İletişim Hizmetleri's antifraud specialists.
- Turkcell İletişim Hizmetleri Uses Engineered System to Analyze 10 Billion Daily, Call-Data Records and Service Logs and to Generate 100,000 Monthly Reports
- Turkcell Deploys Oracle Data Integrator to Drive Efficiency
- Turkcell Accelerates Reporting Tenfold, Saves on Storage and Energy Costs with Consolidated Oracle Exadata Platform
- Turkcell Superonline Transforms Its Order Management and Service Fulfillment with Oracle Communications Solutions
- Technologist of the Year
- Turkcell is an Exemplary Oracle Cross Stack Customer
- Turkcell Gets Three 10X Improvements with Oracle
- Oracle Exadata Changes the Rules of the Game for Turkcell
- Customers Discuss Benefits of Oracle Exadata
- Turkcell Technology Uses Oracle Complex Event Processing for Extreme Scale Mobile Networks
- Turkcell Technology Research & Development Inc. Achieves Substantial Savings with Fault Prevention
- Turkcell iletisim Hizmetleri A.S. Processes Mobile Network Data of 33 Million Subscribers in Real Time
- Kcell Boosts Business Intelligence with Data Warehouse Solution
- Turkcell Gets the Benefits of Oracle Exadata
- Turkcell Eliminates Manual Updates with Oracle IDM
Tuesday May 29, 2012
By chberger on May 29, 2012
I've created and recorded another YouTube-like presentation and "live" demos of Oracle Advanced Analytics Option, this time focusing on Fraud and Anomaly Detection using Oracle Data Mining. [Note: It is a large MP4 file that will open and play in place. The sound quality is weak so you may need to turn up the volume.]
Data is your most valuable asset. It represents the entire history of your organization and its interactions with your customers. Predictive analytics leverages data to discover patterns, relationships and to help you even make informed predictions. Oracle Data Mining (ODM) automatically discovers relationships hidden in data. Predictive models and insights discovered with ODM address business problems such as: predicting customer behavior, detecting fraud, analyzing market baskets, profiling and loyalty. Oracle Data Mining, part of the Oracle Advanced Analytics (OAA) Option to the Oracle Database EE, embeds 12 high performance data mining algorithms in the SQL kernel of the Oracle Database. This eliminates data movement, delivers scalability and maintains security.
But, how do you find these very important needles or possibly fraudulent transactions and huge haystacks of data? Oracle Data Mining’s 1 Class Support Vector Machine algorithm is specifically designed to identify rare or anomalous records. Oracle Data Mining's 1-Class SVM anomaly detection algorithm trains on what it believes to be considered “normal” records, build a descriptive and predictive model which can then be used to flags records that, on a multi-dimensional basis, appear to not fit in--or be different. Combined with clustering techniques to sort transactions into more homogeneous sub-populations for more focused anomaly detection analysis and Oracle Business Intelligence, Enterprise Applications and/or real-time environments to "deploy" fraud detection, Oracle Data Mining delivers a powerful advanced analytical platform for solving important problems. With OAA/ODM you can find suspicious expense report submissions, flag non-compliant tax submissions, fight fraud in healthcare claims and save huge amounts of money in fraudulent claims and abuse.
This presentation and several brief demos will show Oracle Data Mining's fraud and anomaly detection capabilities.
Thursday May 10, 2012
Oracle Virtual SQL Developer Days DB May 15th - Session #3: 1Hr. Predictive Analytics and Data Mining Made Easy!
By chberger on May 10, 2012
Oracle Data Mining's SQL Developer based ODM'r GUI + ODM is being featured in this upcoming Virtual SQL Developer Day online event next Tuesday, May 15th. Several thousand people have already registered and registration is still growing. We recorded and uploaded presentations/demos and then anyone can view them "on demand", but at the specified date/time per the SQL DD event agenda. Anyone can also download a complete 11gR2 Database w/ SQL Developer 3.1 & Oracle Data Miner GUI extension VM installation for the Hands-on Labs and follow our 4 ODM Oracle by Examples e-training. We moderators monitor the online chat and answer questions.
Session #3: 1Hr. Predictive Analytics and Data Mining Made Easy!We're also included in the June 7th physical event in NYC and future virtual and physical events. Great event(s) and great "viz" for OAA/ODM.
Oracle Data Mining, a component of the Oracle Advanced Analytics database option, embeds powerful data mining algorithms in the SQL kernel of the Oracle Database for problems such as customer churn, predicting customer behavior, up-sell and cross-sell, detecting fraud, market basket analysis (e.g. beer & diapers), customer profiling and customer loyalty. Oracle Data Miner, SQL Developer 3.1 extension, provides data analysts a “workflow” paradigm to build analytical methodologies to explore data and build, evaluate and apply data mining models—all while keeping the data inside the Oracle Database. This workshop will teach the student the basics of getting started using Oracle Data Mining.
By chberger on May 10, 2012
Two Oracle Data Mining Virtual Classes are now scheduled. Register for a course in 2 easy steps.
Step 1: Select your Live Virtual Class options
|Live Virtual Class
Course ID: D76362GC10
Course Title: Oracle Database 11g: Data Mining Techniques NEW
Duration: 2 Days
Price: US$ 1,300 Dollars
Step 2: Select the date and location of your Live Virtual Class
Please select a location below then click on the Add to Cart button
|Location||Duration||Class Date||Class Start Time||Class End Time||Course Materials||Instruction Language||Seats||Audience||Employees|
|2 Days||09-Aug-2012||04:00 AM EDT||12:00 PM EDT||English||English||Available||Public|
|2 Days||18-Oct-2012||04:00 AM EDT||12:00 PM EDT||English||English||Available||Public|
Wednesday Apr 04, 2012
By chberger on Apr 04, 2012
Ever want to just sit and watch a YouTube-like presentation and "live" demos of Oracle Advanced Analytics/Oracle Data Mining? Then click here! (plays large MP4 file in a browser)
This 1+ hour long session focuses primarily on the Oracle Data Mining component of the Oracle Advanced Analytics Option and is tied to the Oracle SQL Developer Days virtual and onsite events. I cover:
- Big Data + Big Data Analytics
- Competing on analytics & value proposition
- What is data mining?
- Typical use cases
- Oracle Data Mining high performance in-database SQL based data mining functions
- Exadata "smart scan" scoring
- Oracle Data Miner GUI (an Extension that ships with SQL Developer)
- Oracle Business Intelligence EE + Oracle Data Mining results/predictions in dashboards
- Applications "powered by Oracle Data Mining for factory installed predictive analytics methodologies
- Oracle R Enterprise
Please contact firstname.lastname@example.org should you have any questions. Hope you enjoy!
Charlie Berger, Sr. Director of Product Management, Oracle Data Mining & Advanced Analytics, Oracle Corporation
Friday Mar 23, 2012
By chberger on Mar 23, 2012
A NEW 2-Day Instructor Led Course on Oracle Data Mining has been developed for customers and anyone wanting to learn more about data mining, predictive analytics and knowledge discovery inside the Oracle Database. To register interest in attending the class, click here and submit your preferred format.
- Explain basic data mining concepts and describe the benefits of predictive analysis
- Understand primary data mining tasks, and describe the key steps of a data mining process
- Use the Oracle Data Miner to build,evaluate, and apply multiple data mining models
- Use Oracle Data Mining's predictions and insights to address many kinds of business problems, including: Predict individual behavior, Predict values, Find co-occurring events
- Learn how to deploy data mining results for real-time access by end-users
Five reasons why you should attend this 2 day Oracle Data Mining Oracle University course. With Oracle Data Mining, a component of the Oracle Advanced Analytics Option, you will learn to gain insight and foresight to:
- Go beyond simple BI and dashboards about the past. This course will teach you about "data mining" and "predictive analytics", analytical techniques that can provide huge competitive advantage
- Take advantage of your data and investment in Oracle technology
- Leverage all the data in your data warehouse, customer data, service data, sales data, customer comments and other unstructured data, point of sale (POS) data, to build and deploy predictive models throughout the enterprise.
- Learn how to explore and understand your data and find patterns and relationships that were previously hidden
- Focus on solving strategic challenges to the business, for example, targeting "best customers" with the right offer, identifying product bundles, detecting anomalies and potential fraud, finding natural customer segments and gaining customer insight.
UDDATED for Oracle Database 12c & SQLDEV 4.0: Evaluating Oracle Data Mining Has Never Been Easier - Evaluation "Kit" Available
By chberger on Mar 23, 2012
UPDATED (March 2014) for ORACLE DATABASE 12c & SQL DEVELOPER 4.0 (with ORACLE DATA MINER 4.0) Extension
The Oracle Advanced Analytics Option turns the database into an enterprise-wide analytical platform that can quickly deliver enterprise-wide predictive analytics and actionable insights. Oracle Advanced Analytics empowers data and business analysts to extract knowledge, discover new insights and make predictions—working directly with large data volumes in the Oracle Database. Oracle Advanced Analytics, an Option of Oracle Database Enterprise Edition, offers a combination of powerful in-database algorithms and integration with open source R algorithms accessible via SQL and R languages and provides a range of GUI (Oracle Data Miner) and IDE (R client, RStudio, etc.) options targeting business users, data analysts, application developers and data scientists.
Now you can quickly and easily get set up to starting using Oracle Data Mining, the SQL API & GUI component of the Oracle Advanced Analytics Database Option for evaluation purposes. Just go to the Oracle Technology Network (OTN) and follow these simple steps.
Oracle Data Mining Evaluation "Kit" Instructions
- Anyone can download and install the Oracle Database for free for evaluation purposes. Read OTN web site http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html for details.
- Oracle Database 12c is the latest release and contains many new features. See Oracle Advanced Analytics 12c Dcoumentation's New Features and this recent Oracle Data Mining Blog posting.
- For Oracle Database Release 11g, 184.108.40.206.0 DB is the minimum, 220.127.116.11 is better and naturally 18.104.22.168 is best if you are a current customer and on active support.
- Either 32-bit or 64-bit is fine. 4GB of RAM or more works fine for SQL Developer and the Oracle Data Miner GUI extension.
- Downloading the database and then installing it should take just about an hour or so at most, depending on your network and computer.
- For more instructions on setting up Oracle Data Mining see: http://www.oracle.com/technetwork/database/options/odm/dataminerworkflow-168677.html
- When you install the Oracle Database, the Oracle Data Mining Examples including sample data is available as part of the total Database installation. See link.
Step 2: Install SQL Developer 4.0 (the Oracle Data Miner GUI Extension installs automatically but additional post installation Set Up in required. See Setting Up Oracle Data Miner )
- Setting Up Oracle Data Miner 4.0 This tutorial covers the process of setting up Oracle Data Miner for use within Oracle SQL Developer 4.0.
- Using Oracle Data Miner 4.0 This tutorial covers the use of Oracle Data Miner 4.0 to perform data mining against Oracle Database 12c. In this lesson, you examine and solve a data mining business problem by using the Oracle Data Miner graphical user interface (GUI). The Oracle Data Miner GUI is included as an extension of Oracle SQL Developer, version 4.0.
- Using Feature Selection and Generation with GLM This tutorial covers the use of Oracle Data Miner 4.0 to leverage enhancements to the Oracle implementation of Generalized Liner Models (GLM) for Oracle Database 12c. These enhancements include support for Feature Selection and Generation.
- Text Mining with an EM Clustering Model This tutorial covers the use of Oracle Data Miner 4.0 to leverage new text mining enhancements while applying a clustering model. In this lesson, you learn how to use the Expectation Maximization (EM) algorithm in a clustering model.
- Using Predictive Queries With Oracle Data Miner 4.0 This tutorial covers the use of Predictive Queries against mining data by using Oracle Data Miner 4.0.
- Using the SQL Query Node in a Data Miner Workflow This tutorial covers the use of the new SQL Query Node in an Oracle Data Miner 4.0 workflow.
That’s it! Easy, fun and the fastest way to get started evaluating Oracle Advanced Analytics/Oracle Data Mining. Enjoy!
Note: There are also four (4) additional Oracle Data Miner 3.2 Tutorials that are similar that may be helpful to review.
- Setting Up Oracle Data Miner 11g Release 2 This tutorial covers the process of setting up Oracle Data Miner 11g Release 2 for use within Oracle SQL Developer 3.0.
- Using Oracle Data Miner 11g Release 2 This tutorial covers the use of Oracle Data Miner to perform data mining against Oracle Database 11g Release 2. In this lesson, you examine and solve a data mining business problem by using the Oracle Data Miner graphical user interface (GUI).
- Star Schema Mining Using Oracle Data Miner This tutorial covers the use of Oracle Data Miner to perform star schema mining against Oracle Database 11g Release 2.
- Text Mining Using Oracle Data Miner This tutorial covers the use of Oracle Data Miner to perform text mining against Oracle Database 11g Release 2.
Wednesday Feb 08, 2012
Monday Sep 19, 2011
By chberger on Sep 19, 2011
Example Predictive Analytics Applications (partial list)
- Oracle Communications & Retail Industry Models —factory installed data mining for specific industries
- Oracle Spend Classification
- Oracle Fusion Human Capital Management (HCM) Predictive Workforce
- Oracle Fusion Customer Relationship Management (CRM) Sales Prediction
- Oracle Adaptive Access Manager real-time Security
- Oracle Complex Event Processing integrated with ODM models
- Predictive Incident Monitoring Service for Oracle Database customers
Pretty cool stuff if you or your customers are interested in analytics. Here's the link to the ppt slides.
Tuesday Aug 09, 2011
By chberger on Aug 09, 2011
America's Cup: Oracle Data Mining supports crew and BMW ORACLE Racing
BMW ORACLE Racing won the 33rd America’s Cup yacht race in February 2010, beating the Swiss team, Alinghi, decisively in the first two races of the best-of-three contest.
BMW ORACLE Racing’s victory in the America’s Cup challenge was a lesson in sailing skill, as one of the world’s most experienced crews reached speeds as fast as 30 knots. But if you listen to the crew in their postrace interviews, you’ll notice that what they talk about is technology.
'The story of this race is in the technology,' says Ian Burns, design coordinator for BMW ORACLE Racing.
Learning by Data
'One of the problems we faced at the outset was that we needed really high accuracy in our data because we didn’t have two boats,' says Burns. 'Generally, most teams have two boats, and they sail them side by side. Change one thing on one boat, and it’s fairly easy to see the effect of a change with your own eyes.'
With only one boat, BMW ORACLE Racing’s performance analysis had to be done numerically by comparing data sets. To get the information needed, says Burns, the team had to increase the amount of data collected by nearly 40 times what they had done in the past.
The USA holds 250 sensors to collect raw data: pressure sensors on the wing; angle sensors on the adjustable trailing edge of the wing sail to monitor the effectiveness of each adjustment, allowing the crew to ascertain the amount of lift it’s generating; and fiber-optic strain sensors on the mast and wing to allow maximum thrust without overbending them.
But collecting data was only the beginning. BMW ORACLE Racing also had to manage that data, analyze it, and present useful results. The team turned to Oracle Data Mining in Oracle Database 11g.
Peter Stengard, a principal software engineer for Oracle Data Mining and an amateur sailor, became the liaison between the database technology team and BMW ORACLE Racing. 'Ian Burns contacted us and explained that they were interested in better understanding the performance-driving parameters of their new boat,' says Stengard. 'They were measuring an incredible number of parameters across the trimaran, collected 10 times per second, so there were vast amounts of data available for analysis. An hour of sailing generates 90 million data points.'
After each day of sailing the boat, Burns and his team would meet to review and share raw data with crewmembers or boat-building vendors using a Web application built with Oracle Application Express. 'Someone in the meeting would say, 'Wouldn’t it be great if we could look at some new combination of numbers?’ and we could quickly build an Oracle Application Express application and share the information during the same meeting,' says Burns. Later, the data would be streamed to Oracle’s Austin Data Center, where Stengard and his team would go to work on deeper analysis.
Because BMW ORACLE Racing was already collecting its data in an Oracle database, Stengard and his team didn’t have to do any extract, transform, and load (ETL) processes or data conversion. 'We could just start tackling the analytics problem right away,' says Stengard. 'We used Oracle Data Mining, which is in Oracle Database. It gives us many advanced data mining algorithms to work with, so we have freedom in how we approach any specific task.'
Using the algorithms in Oracle Data Mining, Stengard could help Burns and his team learn new things about how their boat was working in its environment. 'We would look, for example, at mast rotations—which rotation works best for certain wind conditions,' says Stengard. 'There were often complex relationships within the data that could be used to model the effect on the target—in this case something called velocity made good, or VMG. Finding these relationships is what the racing team was interested in.'
Stengard and his team could also look at data over time and with an attribute selection algorithm to determine which sensors provided the most-useful information for their analysis. 'We could identify sensors that didn’t seem to be providing the predictive power they were looking for so they could change the sensor location or add sensors to another part of the boat,' Stengard says.
Burns agrees that without the data mining, they couldn’t have made the boat run as fast. 'The design of the boat was important, but once you’ve got it designed, the whole race is down to how the guys can use it,' he says. 'With Oracle database technology, we could compare our performance from the first day of sailing to the very last day of sailing, with incremental improvements the whole way through. With data mining we could check data against the things we saw, and we could find things that weren’t otherwise easily observable and findable.'
Flying by Data
The greatest challenge of this America’s Cup, according to Burns, was managing the wing sail, which had been built on an unprecedented scale. 'It is truly a massive piece of architecture,' Burns says. 'It’s 20 stories high; it barely fits under the Golden Gate Bridge. It’s an amazing thing to see.'
The wing sail is made of an aeronautical fabric stretched over a carbon fiber frame, giving it the three-dimensional shape of a regular airplane wing. Like an airplane wing, it has a fixed leading edge and an adjustable trailing edge, which allows the crew to change the shape of the sail during the course of a race.
'The crew of the USA was the best group of sailors in the world, but they were used to working with sails,' says Burns, 'Then we put them under a wing. Our chief designer, Mike Drummond, told them an airline pilot doesn’t look out the window when he’s flying the plane; he looks at his instruments, and you guys have to do the same thing.'
A second ship, known as the performance tender, accompanied the USA on the water. The tender served in part as a floating datacenter and was connected to the USA by wireless LAN.
'The USA generates almost 4,000 variables 10 times a second,' says Burns. 'Sometimes the analysis requires a very complicated combination of 10, 20, or 30 variables fitted through a time-based algorithm to give us predictions on what will happen in the next few seconds, or minutes, or even hours in terms of weather analysis.'
Like the deeper analysis that Stengard does back at the Austin Data Center, this real-time data management and near-real-time data analysis was done in Oracle Database 11g. 'We could download the data to servers on the tender ship, do some quick analysis, and feed it right back to the USA,' says Burns.
'We started to do better when the guys began using the instruments,' Burns says. 'Then we started to make small adjustments against the predictions and started to get improvements, and every day we were making gains.'
Those gains were incremental and data driven, and they accumulated over years—until the USA could sail at three times the wind speed. Ian Burns is still amazed by the spectacle.
'It’s an awesome thing to watch,' he says. 'Even with all we have learned, I don’t think we have met the performance limits of that beautiful wing.'
Read more about Oracle Data Mining
Hear a podcast interview with Ian Burns
Download Oracle Database 11g Release 2
Story republished from: www.oracle.com/technology/oramag/oracle/10-may/o30racing.html
by Jeff Erickson Share 11:41 PM Sat 24 Apr 2010 GMT
Everything about Oracle Data Mining, a component of the Oracle Advanced Analytics Option - News, Technical Information, Opinions, Tips & Tricks. All in One Place
- Deploy Data Miner Apply Node SQL as RESTful Web Service for Real-Time Scoring
- How to generate training and test dataset using SQL Query node in Data Miner
- dunnhumby Accelerates Complex Segmentation Queries from Weeks to Minutes—Gains Competitive Advantage
- How to generate Scatterplot Matrices using R script in Data Miner
- How to export data from the Explore Node using Data Miner and SQL Developer
- Oracle BIWA Summit 2014 January 14-16, 2014 at Oracle HQ in Redwood Shores, CA
- Come See and Test Drive Oracle Advanced Analytics at the BIWA Summit'14, Jan 14-16, 2014
- Oracle Big Data Learning Library
- Oracle Data Miner (Extension of SQL Developer 4.0) Integrate Oracle R Enterprise Mining Algorithms into workflow using the SQL Query node
- Oracle Data Miner GUI, part of SQL Developer 4.0 Early Adopter 1 is now available for download on OTN