Tuesday Jan 14, 2014

How to export data from the Explore Node using Data Miner and SQL Developer

Blog posting by Denny Wong, Principal Member of Technical Staff, User Interfaces and Components, Oracle Data Mining Development

The Explorer node generates descriptive statistical data and histogram data for all input table columns.  These statistical and histogram data may help user to analyze the input data to determine if any action (e.g. transformation) is needed before using it for data mining purpose.  An analyst may want to export this data to a file for offline analysis (e.g. Excel) or reporting purpose.  The Explorer node generates this data to a database table specified in the Output tab of the Property Inspector.  In this case, the data is generated to a table named “OUTPUT_1_2”.


To export the table to a file, we can use the SQL Developer Export wizard. Go to the Connections tab in the Navigator Window, search for the table “OUTPUT_1_2” within the proper connection, then bring up the pop-up menu off the table. Click on the Export menu to launch the Export Wizard.


In the wizard, uncheck the “Export DDL” and select the “Export Data” option since we are only interested in the data itself. In the Format option, select “excel” in this example (a dozen of output formats are supported) and specify the output file name. Upon wizard finish, an excel file is generated.


Let’s open the file to examine what is in it. As expected, it contains all statistical data for all input columns. The histogram data is listed as the last column (HISTOGRAMS), and it has this ODMRSYS.ODMR_HISTOGRAMS structure.


For example, let’s take a closer look at the histogram data for the BUY_INSURANCE column:

ODMRSYS.ODMR_HISTOGRAMS(ODMRSYS.ODMR_HISTOGRAM_POINT('"BUY_INSURANCE"',''No'',NULL,NULL,73.1),ODMRSYS.ODMR_HISTOGRAM_POINT('"BUY_INSURANCE"',''Yes'',NULL,NULL,26.9))

This column contains an ODMRSYS.ODMR_HISTOGRAMS object which is an array of ODMRSYS.ODMR_HISTOGRAM_POINT structure. We can describe the structure to see what is in it.


The ODMRSYS.ODMR_HISTOGRAM_POINT contains five attributes, which represent the histogram data. The ATTRIBUTE_NAME contains the attribute name (e.g. BUY_INSURANCE), the ATTRIBUTE_VALUE contains the attribute values (e.g. No, Yes), the GROUPING_ATTRIBUTE_NAME and GROUPING_ ATTRIBUTE_VALUE are not used (these fields are used when the Group By option is specified), and the ATTRIBUTE_PERCENT contains the percents (e.g. 73.1, 26.9) for the attribute values respectively.


As you can see the ODMRSYS.ODMR_HISTOGRAMS complex output format may be difficult to read and it may require some processing before the data can be used. Alternatively, we can “unnest” the histogram data to transactional data format before exporting it. This way we don’t have to deal with the complex array structure, thus the data is more consumable. To do that, we can write a simple SQL query to “unnest” the data and use the new SQL Query node (Extract histogram data) to run this query (see below). We then use a Create Table node (Explorer output table) to persist the “unnested” histogram data along with the statistical data.

1. Create a SQL Query node

Create a SQL Query node and connect the “Explore Data” node to it. You may rename the SQL Query node to “Extract histogram data” to make it clear it is used to “unnest” the histogram data.

2. Specify a SQL query to “unnest” histogram data

Double click the “Extract histogram data” node to bring up the editor, enter the following SELECT statement in the editor:

SELECT
    "Explore Data_N$10002"."ATTR",
    "Explore Data_N$10002"."AVG",
    "Explore Data_N$10002"."DATA_TYPE",
    "Explore Data_N$10002"."DISTINCT_CNT",
    "Explore Data_N$10002"."DISTINCT_PERCENT",
    "Explore Data_N$10002"."MAX",
    "Explore Data_N$10002"."MEDIAN_VAL",
    "Explore Data_N$10002"."MIN",
    "Explore Data_N$10002"."MODE_VALUE",
    "Explore Data_N$10002"."NULL_PERCENT",
    "Explore Data_N$10002"."STD",
    "Explore Data_N$10002"."VAR",
    h.ATTRIBUTE_VALUE,
    h.ATTRIBUTE_PERCENT
FROM
    "Explore Data_N$10002", TABLE("Explore Data_N$10002"."HISTOGRAMS") h

Click OK to close the editor. This query is used to extract out the ATTRIBUTE_VALUE and ATTRIBUTE_PERCENT fields from the ODMRSYS.ODMR_HISTOGRAMS nested object.

Note: you may select only columns that contain the statistics you are interested in.  The "Explore Data_N$10002" is a generated unique name reference to the Explorer node, you may have a slightly different name ending with some other unique number. 

The query produces the following output.  The last two columns are the histogram data in transactional format.

3. Create a Create Table node to persist the “unnested” histogram data

Create a Create Table node and connect the “Extract histogram data” node to it. You may rename the Create Table node to “Explorer output table” to make it clear it is used to persist the “unnested” histogram data.


4. Export “unnested” histogram data to Excel file

Run the “Explorer output table” node to persist the “unnested” histogram data to a table. The name of the output table (OUTPUT_3_4) can be found in the Property Inspector below.


Next, we can use the SQL Developer Export wizard as described above to export the table to an Excel file. As you can see the histogram data are now in transactional format; they are more readable and can readily be consumed.


Tuesday Dec 31, 2013

Oracle BIWA Summit 2014 January 14-16, 2014 at Oracle HQ in Redwood Shores, CA


Oracle Business Intelligence, Warehousing & Analytics Summit - Redwood City

Oracle is a proud sponsor of the Business Intelligence, Warehousing & Analytics (BIWA) Summit happening January 14 – 16 at the Oracle Conference Center in Redwood City. The Oracle BIWA Summit brings together Oracle ACE experts, customers who are currently using or planning to use Oracle BI, Warehousing and Analytics products and technologies, partners and Oracle Product Managers, Support Personnel and Development Managers. Join us on Tuesday, January 14 at 5 p.m. to hear featured speaker Balaji Yelamanchili, Senior Vice President Analytics and Performance Management Products, for his keynote: Oracle Business Intelligence -- Innovate Faster. Visit the BIWA site http://www.biwasummit.com/ for more information today.

 Among the approximately 50 technical presentations, featured talks a Hands on Labs, I'll be delivering a presentation on Oracle Advanced Analytics and a Hands on Lab on using the OAA/Oracle Data Miner GUI.  

 AA-1010 BEST PRACTICES FOR IN-DATABASE ANALYTICS

Session ID: AA-1010

Presenter: Charlie Berger, Oracle

Abstract:

In the era of Big Data, enterprises are acquiring increasing volumes and varieties of data from a rapidly growing range of internet, mobile, sensor and other real-time and near real-time sources.  The driving force behind this trend toward Big Data analysis is the ability to use this data for “actionable intelligence” -- to predict patterns and behaviors and to deliver essential information when and where it is needed. Oracle Database uniquely offers a powerful platform to perform this predictive analytics and location analysis with in-database data mining, statistical processing and SQL Analytics.  Oracle Advanced Analytics embeds powerful data mining algorithms and adds enterprise scale open source R to solve problems such as predicting customer behavior, anticipating churn, detecting fraud, market basket analysis and discovering customer segments.  Oracle Data Miner GUI , a new SQL Developer 4.0 Extension, enables business analysts to quickly analyze data and visualize data, build, evaluate and apply predictive models and deploy via SQL scripts sophisticated predictive analytics methodologies—all while keeping the data inside the Oracle Database.  Come learn best practices and customer examples for exploiting Oracle’s scalable, performant and secure in-database analytics capabilities to extract more value and actionable intelligence from your data.

HOL-AA-1008 LEARN TO USE ORACLE ADVANCED ANALYTICS FOR PREDICTIVE ANALYTICS SOLUTIONS

Session ID: HOL-AA-1008

Presenter: Charles Berger, Oracle & Karl Rexer, Rexer Analytics

Abstract:

Big Data;  Bigger Insights!  Oracle Data Mining Release 12c, a component of the Oracle Advanced Analytics database Option, embeds powerful data mining algorithms in the SQL kernel of the Oracle Database for problems such as predicting customer behavior, anticipating churn, identifying up-sell and cross-sell, detecting anomalies and potential fraud, market basket analysis, customer profiling, text mining and retail market basket analysis.  Oracle Data Miner GUI , a new SQL Developer 4.0 Extension, enables business analysts to quickly analyze data and visualize data, build, evaluate and apply predictive models and develop sophisticated predictive analytics methodologies—all while keeping the data inside Oracle Database.  Come see how easily you can discover big insights from your Oracle data and generate SQL scripts for deployment and automation and deploying results into Oracle Business Intelligence (OBIEE) dashboards. 

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Monday Dec 09, 2013

Come See and Test Drive Oracle Advanced Analytics at the BIWA Summit'14, Jan 14-16, 2014

The BIWA Summit '14 January 14-16 at Oracle HQ Conference Center Detailed Agenda is now published.   

Please share with your others by Tweeting, Blogging, Facebook, LinkedIn, Email, etc.!

The BIWA Summit is known for novel and interesting use cases of Oracle Big Data, Exadata, Advanced Analytics/Data Mining, OBIEE, Spatial, Endeca and more!    Opportunities to get hands on experience with products in the Hands on Labs, great customer case studies and talks by Oracle Technical Professionals and Partners.  Meet with technical experts.  Click HERE to read detailed abstracts and speaker profiles. 

Use the SPECIAL DISCOUNT code ORACLE12C and registration is only $199 for the 2.5 day technically focused Oracle user group event.

Charlie  (Oracle Employee Advisor to Oracle BIWA Special Interest User Group)

----


<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Tuesday Nov 12, 2013

Oracle Big Data Learning Library

Click on LEARN BY PRODUCT to view all learning resources.

Oracle Big Data Essentials

Attend this Oracle University Course!

Using Oracle NoSQL Database

Attend this Oracle University class!

Oracle and Big Data on OTN

See the latest resource on OTN.

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Wednesday Sep 04, 2013

Oracle Data Miner (Extension of SQL Developer 4.0) Integrate Oracle R Enterprise Mining Algorithms into workflow using the SQL Query node

I posted a new white paper authored by Denny Wong, Principal Member of Technical Staff, User Interfaces and Components, Oracle Data Mining Technologies.  You can access the white paper here and the companion files here.  Here is an excerpt:

Oracle Data Miner (Extension of SQL Developer 4.0) 

Integrate Oracle R Enterprise Mining Algorithms into workflow using the SQL Query node

Oracle R Enterprise (ORE), a component of the Oracle Advanced Analytics Option, makes the open source R statistical programming language and environment ready for the enterprise and big data. Designed for problems involving large amounts of data, Oracle R Enterprise integrates R with the Oracle Database. R users can develop, refine and deploy R scripts that leverage the parallelism and scalability of the database to perform predictive analytics and data analysis.

Oracle Data Miner (ODMr) offers a comprehensive set of in-database algorithms for performing a variety of mining tasks, such as classification, regression, anomaly detection, feature extraction, clustering, and market basket analysis. One of the important capabilities of the new SQL Query node in Data Miner 4.0 is a simplified interface for integrating R scripts registered with the database. This provides the support necessary for R Developers to provide useful mining scripts for use by data analysts. This synergy provides many additional benefits as noted below.

· R developers can further extend ODMr mining capabilities by incorporating the extensive R mining algorithms from the open source CRAN packages or leveraging any user developed custom R algorithms via SQL interfaces provided by ORE.

· Since this SQL Query node can be part of a workflow process, R scripts can leverage functionalities provided by other workflow nodes which can simplify the overall effort of integrating R capabilities within the database.

· R mining capabilities can be included in the workflow deployment scripts produced by the new sql script generation feature. So the ability of deploy R functionality within the context of an Data Miner workflow is easily accomplished.

· Data and processing are secured and controlled by the Oracle Database. This alleviates a lot of risk that are incurred by other providers, when users have to export data out of the database in order to perform advanced analytics.

Oracle Advanced Analytics saves analysts, developers, database administrators and management the headache of trying to integrate R and database analytics. Instead, users can quickly gain the benefit of new R analytics and spend their time and effort on developing business solutions instead of building homegrown analytical platforms.

This paper should be very useful to R developers wishing to better understand how to leverage imbedding R Scripts for use by Data Analysts.  Analysts will also find the paper useful to see how R features can be surfaced for their use in Data Miner. The specific use case covered demonstrates how to use the SQL Query node to integrate R glm and rpart regression model build, test, and score operations into the workflow along with nodes that perform data preparation and residual plot graphing. However, the integration process described here can easily be adapted to integrate other R operations like statistical data analysis and advanced graphing to expand ODMr functionalities.

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Monday Jul 15, 2013

Oracle Data Miner GUI, part of SQL Developer 4.0 Early Adopter 1 is now available for download on OTN

The NEW Oracle Data Miner GUI, part of SQL Developer 4.0 Early Adopter 1 is now available for download on OTN.  See link to SQL Developer 4.0 EA1.   


The Oracle Data Miner 4.0 New Features are applicable to Oracle Database 11g Release 2 and Oracle Database Release 12c:  See Oracle Data Miner Extension to SQL Developer 4.0 Release Notes for EA1 for additional information  

· Workflow SQL Script Deployment

o Generates SQL scripts to support full deployment of workflow contents

· SQL Query Node

o Integrate SQL queries to transform data or provide a new data source

o Supports the running of R Language Scripts and viewing of R generated data and graphics


· Graph Node

o Generate Line, Scatter, Bar, Histogram and Box Plots



· Model Build Node Improvements

o Node level data usage specification applied to underlying models

o Node level text specifications to govern text transformations

o Displays heuristic rules responsible for excluding predictor columns

o Ability to control the amount of Classification and Regression test results generated

· View Data

o Ability to drill in to view custom objects and nested tables

These new Oracle Data Miner GUI capabilities expose Oracle Database 12c and Oracle Advanced Analytics/Data Mining Release 1 features:

· Predictive Query Nodes

o Predictive results without the need to build models using Analytical Queries

o Refined predictions based on data partitions

· Clustering Node New Algorithm

o Added Expectation Maximization algorithm

· Feature Extraction Node New Algorithms

o Added Singular Value Decomposition and Principal Component Analysis algorithms

· Text Mining Enhancements

o Text transformations integrated as part of Model's Automatic Data Preparation

o Ability to import Build Text node specifications into a Model Build node

· Prediction Result Explanations

o Scoring details that explain predictive result

· Generalized Linear Model New Algorithm Settings

o New algorithm settings provide feature selection and generation

See OAA on OTN pages http://www.oracle.com/technetwork/database/options/advanced-analytics/index.html for more information on Oracle Advanced Analytics.

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Wednesday May 08, 2013

Oracle Advanced Analytics and Data Mining at the Movies on YouTube - Updated August 3, 2015

Updated August 3, 2015

Periodically, I've recorded a demonstration and/or presentation on Oracle Advanced Analytics and Data Mining and have posted them on YouTube.

Here are links to some of more recent YouTube postings--sort of an Oracle Advanced Analytics and Data Mining at the Movies experience.

  1. New Big Data Analyics using Oracle Advanced Analytics12c and Big Data SQL  - Watch on YouTube
  2. New Oracle Academy Webcast:  Ask the Oracle Experts Fraud &  Anomaly Detection using Oracle Advanced Analytics 12c & Big Data SQL - Watch YouTube
  3. New - Oracle Academy Webcast:  Ask the Oracle Experts Big Data Analytics with Oracle Advanced Analytics - Watch YouTube
  4. Oracle Data Miner and Oracle R Enterprise Integration via SQL Query node - Watch Demo
  5. Oracle Data Miner 4.0 (SQL Developer 4.0 Extension) New Features - Watch Demo
  6. Oracle Business Intelligence Enterprise Edition (OBIEE) SampleAppls Demo featuring integration with Oracle Advanced Analytics/Data Mining
  7. Oracle Big Data Analytics Demo mining remote sensor data from HVACs for better customer service 
  8. In-Database Data Mining for Retail Market Basket Analysis Using Oracle Advanced Analytics
  9. In-Database Data Mining Using Oracle Advanced Analytics for Classification using Insurance Use Case
  10. Fraud and Anomaly Detection using Oracle Advanced Analytics Part 1 Concepts
  11. Fraud and Anomaly Detection using Oracle Advanced Analytics Part 2 Demo
  12. Overview Presentation and Demonstration of Oracle Advanced Analytics Database Option

So.... grab your popcorn and a comfortable chair.  Hope you enjoy!

Charlie 

Oracle Advanced Analytics at the Movies

Thursday Mar 21, 2013

Recorded Webcast: Best Practices using Oracle Advanced Analytics with Oracle Exadata

Best Practices using Oracle Advanced Analytics with Oracle Exadata

 On Demand
Launch Presentation


Join us to learn how Oracle Advanced Analytics extends the Oracle database into a comprehensive advanced analytics platform through two major components, Oracle R Enterprise and Oracle Data Mining. Using these tools with Oracle Exadata Database Machine will allow organizations to perform at their peak and find real business value within their data.

You need to visit this Oracle Exadata Webcast Main page first and submit your registration information.  Then you’ll receive an email so you can view the Webcast.  This is external so you can share with anyone can download the presentation as well.  FYI.  Charlie

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Wednesday Mar 13, 2013

Oracle OpenWorld Call for Proposals now OPEN Submit your Oracle Advanced Analytics/Data Mining/ORE talks today!!

Calling All Oracle OpenWorld Oracle Advanced Analytics, Data Mining and R Experts


The Call for Proposals is open. Have something interesting to present to the world’s largest gathering of Oracle technologists and business leaders? Making breakthrough innovations with Java or MySQL? We want to hear from you, and so do the attendees at this year’s Oracle OpenWorld, JavaOne, and MySQL Connect conferences. Submit your proposal now for a chance to share your expertise at one of the most important technology and business conferences of the year.

CHOOSE...

Select one of Oracle’s premiere conferences

SHARE...

Submit your proposal for sharing your most innovative ideas and experiences

JOIN...

Connect with the elite community of Oracle OpenWorld, JavaOne, and MySQL Connect session leaders in 2013

We recommend you take the time to review the General Information, Content Program Policies, and Tips and Guidelines pages before you begin. We look forward to your submissions!


Submit Papers

Please submit your papers by clicking on the link below and then select the event for which you are submitting.

Submit Now!

General Information

Conferences location: San Francisco, California, USA


Dates

  • Oracle OpenWorld: Sunday, September 22, 2013–Thursday, September 26, 2013
  • JavaOne: Sunday, September 22, 2013–Thursday, September 26, 2013
  • MySQL Connect: Saturday, September 21–Monday, September 23, 2013

Key 2013 deadlines

Deliverables

Due Dates

Call for Proposals–Open

Wednesday, March 13

Call for Proposals–Closed

Friday, April 12, 11:59 p.m. PDT

Notifications for accepted and declined submissions sent

Mid-June

For Oracle OpenWorld, Oracle employee submitters will need to contact the appropriate Oracle track leads before submitting. To view a list of track leads, click here

Contact us:

Friday Feb 22, 2013

Take a FREE Test Drive with Oracle Advanced Analytics/Data Mining on the Amazon Cloud

I wanted to highlight a wonderful new resource provided by our partner Vlamis Software.  Extremely easy!  Fill out the form, wait a few minutes for the Amazon Cloud instance to start up and them BAM!  You can login and start using the Oracle Advanced Analytics Oracle Data Miner work flow GUI.  Demo data and online Oracle by Example Learning Tutorials are also provided to ensure your data mining test drive is a positive one,  Enjoy!! 

Test Drive -- Powered by Amazon AWS

We have partnered with Amazon Web Services to provide to you, free of charge, the opportunity to work, hands-on, with the latest of Oracle's Business Intelligence offerings. By signing up to one of the labs below, Amazon's Elastic Cloud Computer (EC2) environment will generate a complete server for you to work with.

These hands on labs are working with the actual Oracle software running on the Amazon Web Services EC2 environment. They each take approximately 2 hours to work through and will give you hands-on experience with the software and a tour of the features. Your EC2 environment will be available for you for 5 hours, at which time it will self-terminate. If, after registration, you need additional time or need further instructions, simply reply to the registration email and we would be glad to help you.

Data Mining

This test drive walks through some basic exercises in doing predictive analytics within an Oracle 11g Database instance using the Oracle Data Miner extension for Oracle SQL Developer. You use a drag-and-drop "workflow" interface to build a data mining model that predicts the likelihood of purchase for a set of prospects. Oracle Data Mining is ideal for automatically finding patterns, understanding relationships, and making predictions in large data sets.

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Monday Jan 28, 2013

BIWA Summit 2013 Presentations Now Available for Viewing

If you missed the BIWA Summit 2013, you can still look through the presentations from the event. 

Go to the Schedule at http://www.biwasummit.com/schedule  and download the presentations using the links for each session.  You can forward this to customers, prospects and others within Oracle.  All is external.

The Oracle BIWA Summit, organized by the leading Oracle Special Interest Group (SIG) for Business Intelligence, Data Warehousing and Analytics professionals, was be held on Jan 9,10 2013, at The Oracle HQ Sofitel Hotel, in Redwood City, CA. The Oracle BIWA Summit brings together Oracle ACE experts, customers who are currently using or planning to use Oracle BI, Warehousing and Analytics products and technologies, partners and Oracle Product Managers, Support Personnel and Development Managers. Everything and everyone that you will need to be successful in your Oracle “BIWA” implementations was at the Oracle BIWA Summit, Jan 9-10, 2013.

The next BIWA Summit will be at the HQ Conference Center, Jan 14-16, 2014.  Mark your calendars.

Tuesday Jan 01, 2013

Turkcell Combats Pre-Paid Calling Card Fraud Using In-Database Oracle Advanced Analytics

Turkcell İletişim Hizmetleri A.S. Successfully Combats Communications Fraud with Advanced In-Database Analytics

[Original link available on oracle.com http://www.oracle.com/us/corporate/customers/customersearch/turkcell-1-exadata-ss-1887967.html]

Turkcell İletişim Hizmetleri A.Ş. is a leading provider of mobile communications in Turkey with more than 34 million subscribers. Established in 1994, Turkcell created the first global system for a mobile communications (GSM) network in Turkey. It was the first Turkish company listed on the New York Stock Exchange.

Communications fraud, or the  use of telecommunications products or services without intention to pay, is a major issue for the organization. The practice is fostered by prepaid card usage, which is growing rapidly. Anonymous network-branded prepaid cards are a tempting vehicle for money launderers, particularly since these cards can be used as cash vehicles—for example, to withdraw cash at ATMs. It is estimated that prepaid card fraud represents an average loss of US$5 per US$10,000 in transactions. For a communications company with billions of transactions, this could result in millions of dollars lost through fraud every year.

Consequently, Turkcell wanted to combat communications fraud and money laundering by introducing advanced analytical solutions to monitor key parameters of prepaid card usage and issue alerts or block fraudulent activity. This type of fraud prevention would require extremely fast analysis of the company’s one petabyte of uncompressed customer data to identify patterns and relationships, build predictive models, and apply those models to even larger data volumes to make accurate fraud predictions.

To achieve this, Turkcell deployed Oracle Exadata Database Machine X2-2 HC Full Rack, so that data analysts can build predictive antifraud models inside the Oracle Database and deploy them into Oracle Exadata for scoring, using Oracle Data Mining, a component of Oracle Advanced Analytics, leveraging Oracle Database11g technology. This enabled the company to create predictive antifraud models faster than with any other machine, as models can be built using search and query language (SQL) inside the database, and Oracle Exadata can access raw data without summarized tables, thereby achieving extremely fast analyses.

Challenges

A word from Turkcell İletişim Hizmetleri A.Ş.

“Turkcell manages 100 terabytes of compressed data—or one petabyte of uncompressed raw data—on Oracle Exadata. With Oracle Data Mining, a component of the Oracle Advanced Analytics Option, we can analyze large volumes of customer data and call-data records easier and faster than with any other tool and rapidly detect and combat fraudulent phone use.” – Hasan Tonguç Yılmaz, Manager, Turkcell İletişim Hizmetleri A.Ş.

  • Combat communications fraud and money laundering by introducing advanced analytical solutions to monitor prepaid card usage and alert or block suspicious activity
  • Monitor numerous parameters for up to 10 billion daily call-data records and value-added service logs, including the number of accounts and cards per customer, number of card loads per day, number of account loads over time, and number of account loads on a subscriber identity module card at the same location
  • Enable extremely fast sifting through huge data volumes to identify patterns and relationships, build predictive antifraud models, and apply those models to even larger data volumes to make accurate fraud predictions
  • Detect fraud patterns as soon as possible and enable quick response to minimize the negative financial impact

Solutions

Oracle Product and Services

  • Used Oracle Exadata Database Machine X2-2 HC Full Rack to create predictive antifraud models more quickly than with previous solutions by accessing raw data without summarized tables and providing unmatched query speed, which optimizes and shortens the project design phases for creating predictive antifraud models
  • Leveraged SQL for the preparation and transformation of one petabyte of uncompressed raw communications data, using Oracle Data Mining, a feature of Oracle Advanced Analytics to increase the performance of predictive antifraud models
  • Deployed Oracle Data Mining models on Oracle Exadata to identify actionable information in less time than traditional methods—which would require moving large volumes of customer data to a third-party analytics software—and achieve an average gain of four hours and more, taking into consideration the absence of any system crash (as occurred in the previous environment) during data import
  • Achieved extreme data analysis speed with in-database analytics performed inside Oracle Exadata, through a row-wise information search—including day, time, and duration of calls, as well as number of credit recharges on the same day or at the same location—and query language functions that enabled analysts to detect fraud patterns almost immediately
  • Implemented a future-proof solution that could support rapidly growing data volumes that tend to double each year with Oracle Exadata’s massively scalable data warehouse performance

Why Oracle

“We selected Oracle because in-database mining to support antifraud efforts will be a major focus for Turkcell in the future. With Oracle Exadata Database Machine and the analytics capabilities of Oracle Advanced Analytics, we can complete antifraud analysis for large amounts of call-data records in just a few hours. Further, we can scale the solution as needed to support rapid communications data growth,” said Hasan Tonguç Yılmaz, datawarehouse/data mining developer, Turkcell Teknoloji Araştırma ve Geliştirme A.Ş.

Partner

Oracle Partner: Turkcell Teknoloji Araştırma ve Geliştirme A.Ş.

All development and test processes were performed by Turkcell Teknoloji. The company also made significant contributions to the configuration of numerous technical analyses which are carried out regularly by Turkcell İletişim Hizmetleri's antifraud specialists.

Resources

<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Friday Jun 08, 2012

New Oracle Advanced Analytics presentation

I recently updated my presentation on Oracle's new Advanced Analytics Option which bundles Oracle Data Mining with Oracle R Enterprise for maximum depth and breadth of data mining, statistics and advanced analytic functions from Oracle.  See New Oracle Advanced Analytics presentation.  

Tuesday May 29, 2012

Fraud and Anomaly Detection using Oracle Data Mining YouTube-like Video

I've created and recorded another YouTube-like presentation and "live" demos of Oracle Advanced Analytics Option, this time focusing on Fraud and Anomaly Detection using Oracle Data Mining.  [Note:  It is a large MP4 file that will open and play in place.  The sound quality is weak so you may need to turn up the volume.]

Data is your most valuable asset. It represents the entire history of your organization and its interactions with your customers.  Predictive analytics leverages data to discover patterns, relationships and to help you even make informed predictions.   Oracle Data Mining (ODM) automatically discovers relationships hidden in data.  Predictive models and insights discovered with ODM address business problems such as:  predicting customer behavior, detecting fraud, analyzing market baskets, profiling and loyalty.  Oracle Data Mining, part of the Oracle Advanced Analytics (OAA) Option to the Oracle Database EE, embeds 12 high performance data mining algorithms in the SQL kernel of the Oracle Database. This eliminates data movement, delivers scalability and maintains security. 

But, how do you find these very important needles or possibly fraudulent transactions and huge haystacks of data? Oracle Data Mining’s 1 Class Support Vector Machine algorithm is specifically designed to identify rare or anomalous records.  Oracle Data Mining's 1-Class SVM anomaly detection algorithm trains on what it believes to be considered “normal” records, build a descriptive and predictive model which can then be used to flags records that, on a multi-dimensional basis, appear to not fit in--or be different.  Combined with clustering techniques to sort transactions into more homogeneous sub-populations for more focused anomaly detection analysis and Oracle Business Intelligence, Enterprise Applications and/or real-time environments to "deploy" fraud detection, Oracle Data Mining delivers a powerful advanced analytical platform for solving important problems.  With OAA/ODM you can find suspicious expense report submissions, flag non-compliant tax submissions, fight fraud in healthcare claims and save huge amounts of money in fraudulent claims  and abuse.  

This presentation and several brief demos will show Oracle Data Mining's fraud and anomaly detection capabilities.  


<script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Thursday May 10, 2012

Oracle Virtual SQL Developer Days DB May 15th - Session #3: 1Hr. Predictive Analytics and Data Mining Made Easy!

All,

Oracle Data Mining's SQL Developer based ODM'r GUI + ODM is being featured in this upcoming Virtual SQL Developer Day online event next Tuesday, May 15th.  Several thousand people have already registered and registration is still growing.  We recorded and uploaded presentations/demos and then anyone can view them "on demand", but at the specified date/time per the SQL DD event agenda.  Anyone can also download a complete 11gR2 Database w/ SQL Developer 3.1 & Oracle Data Miner GUI extension VM installation for the Hands-on Labs and follow our 4 ODM Oracle by Examples e-training.  We moderators monitor the online chat and answer questions. 
Session #3: 1Hr. Predictive Analytics and Data Mining Made Easy!
Oracle Data Mining, a component of the Oracle Advanced Analytics database option, embeds powerful data mining algorithms in the SQL kernel of the Oracle Database for problems such as customer churn, predicting customer behavior, up-sell and cross-sell, detecting fraud, market basket analysis (e.g. beer & diapers), customer profiling and customer loyalty. Oracle Data Miner, SQL Developer 3.1 extension, provides data analysts a “workflow” paradigm to build analytical methodologies to explore data and build, evaluate and apply data mining models—all while keeping the data inside the Oracle Database. This workshop will teach the student the basics of getting started using Oracle Data Mining.
We're also included in the June 7th physical event in NYC and future virtual and physical events.  Great event(s) and great "viz" for OAA/ODM.

 Charlie

Oracle Data Mining Virtual Classes Scheduled

Two Oracle Data Mining Virtual Classes are now scheduled.  Register for a course in 2 easy steps.

Step 1: Select your Live Virtual Class options

Select


Live Virtual Class
Course ID: D76362GC10
Course Title: Oracle Database 11g: Data Mining Techniques
NEW
Duration: 2 Days
Price: US$ 1,300 Dollars

Step 2: Select the date and location of your Live Virtual Class

Please select a location below then click on the Add to Cart button

i

Location  Duration Class Date Class Start Time Class End Time Course Materials Instruction Language Seats Audience Employees
Online 2 Days 09-Aug-2012 04:00 AM EDT 12:00 PM EDT English English Available Public Employees
Online 2 Days 18-Oct-2012 04:00 AM EDT 12:00 PM EDT English English Available Public Employees

100% Student Satisfaction: Oracle's 100% Student Satisfaction program applies to those publicly scheduled and publicly available Oracle University Instructor Led Training classes that are identified as part of the 100% Student Satisfaction program on the http://www.oracle.com/education website at the time the class is purchased. Oracle will permit unsatisfied students to retake the class, subject to terms and conditions. Customers are not entitled to a refund. For more information and additional terms, conditions and restrictions that apply, click here

Wednesday Apr 04, 2012

Recorded YouTube-like presentation and "live" demos of Oracle Advanced Analytics/Oracle Data Mining

Ever want to just sit and watch a YouTube-like presentation and "live" demos of Oracle Advanced Analytics/Oracle Data Mining?  Then click here! (plays large MP4 file in a browser)

This 1+ hour long session focuses primarily on the Oracle Data Mining component of the Oracle Advanced Analytics Option and is tied to the Oracle SQL Developer Days virtual and onsite events.   I cover:

  • Big Data + Big Data Analytics
  • Competing on analytics & value proposition
  • What is data mining?
  • Typical use cases
  • Oracle Data Mining high performance in-database SQL based data mining functions
  • Exadata "smart scan" scoring
  • Oracle Data Miner GUI (an Extension that ships with SQL Developer)
  • Oracle Business Intelligence EE + Oracle Data Mining results/predictions in dashboards
  • Applications "powered by Oracle Data Mining for factory installed predictive analytics methodologies
  • Oracle R Enterprise

Please contact charlie.berger@oracle.com should you have any questions.  Hope you enjoy! 

Charlie Berger, Sr. Director of Product Management, Oracle Data Mining & Advanced Analytics, Oracle Corporation

Friday Mar 23, 2012

NEW 2-Day Instructor Led Course on Oracle Data Mining Now Available!

A NEW 2-Day Instructor Led Course on Oracle Data Mining has been developed for customers and anyone wanting to learn more about data mining, predictive analytics and knowledge discovery inside the Oracle Database.  To register interest in attending the class, click here and submit your preferred format.

Course Objectives:

  • Explain basic data mining concepts and describe the benefits of predictive analysis
  • Understand primary data mining tasks, and describe the key steps of a data mining process
  • Use the Oracle Data Miner to build,evaluate, and apply multiple data mining models
  • Use Oracle Data Mining's predictions and insights to address many kinds of business problems, including: Predict individual behavior, Predict values, Find co-occurring events
  • Learn how to deploy data mining results for real-time access by end-users

Five reasons why you should attend this 2 day Oracle Data Mining Oracle University course. With Oracle Data Mining, a component of the Oracle Advanced Analytics Option, you will learn to gain insight and foresight to:

  • Go beyond simple BI and dashboards about the past. This course will teach you about "data mining" and "predictive analytics", analytical techniques that can provide huge competitive advantage
  • Take advantage of your data and investment in Oracle technology
  • Leverage all the data in your data warehouse, customer data, service data, sales data, customer comments and other unstructured data, point of sale (POS) data, to build and deploy predictive models throughout the enterprise.
  • Learn how to explore and understand your data and find patterns and relationships that were previously hidden
  • Focus on solving strategic challenges to the business, for example, targeting "best customers" with the right offer, identifying product bundles, detecting anomalies and potential fraud, finding natural customer segments and gaining customer insight.

UDDATED for Oracle Database 12c & SQLDEV 4.0: Evaluating Oracle Data Mining Has Never Been Easier - Evaluation "Kit" Available

UPDATED (October 2015) for ORACLE DATABASE 12c & SQL DEVELOPER 4.1 (with ORACLE DATA MINER 4.1)  Extension

The Oracle Advanced Analytics Option turns the database into an enterprise-wide analytical platform that can quickly deliver enterprise-wide predictive analytics and actionable insights. Oracle Advanced Analytics empowers data and business analysts to extract knowledge, discover new insights and make predictions—working directly with large data volumes in the Oracle Database. Oracle Advanced Analytics, an Option of Oracle Database Enterprise Edition, offers a combination of powerful in-database algorithms and integration with open source R algorithms accessible via SQL and R languages and provides a range of GUI (Oracle Data Miner) and IDE (R client, RStudio, etc.) options targeting business users, data analysts, application developers and data scientists.

Now you can quickly and easily get set up to starting using Oracle Data Mining, the SQL API & GUI component of the Oracle Advanced Analytics Database Option for evaluation purposes. Just go to the Oracle Technology Network (OTN) and follow these simple steps.

Oracle Data Mining Evaluation "Kit" Instructions

Step 1: Download and Install the Oracle Database 12c

  • Anyone can download and install the Oracle Database for free for evaluation purposes. Read OTN web site http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html for details.
  • Oracle Database 12c is the latest release and contains many new features.  See Oracle Advanced Analytics 12c Dcoumentation's New Features and this recent Oracle Data Mining Blog posting.  NOTE:  A major new feature of the 12c Oracle Database is multi-tenant and the ability to set up multiple Container databases.  However, to keep things simpler, UNCHECK the "create as Container database" option.  This makes your SQLDEV database connections simpler and then you can use the simpler case Oracle Data Miner tutorials.  If you create the Container database(s), your connection details get a bit more complicated.  
  • For  Oracle Database Release 11g, 11.2.0.1.0 DB is the minimum, 11.2.0.2 is better and naturally 11.2.0.3 is best if you are a current customer and on active support.
  • Either 32-bit or 64-bit is fine. 4GB of RAM or more works fine for SQL Developer and the Oracle Data Miner GUI extension.
  • Downloading the database and then installing it should take just about an hour or so at most, depending on your network and computer.
  • For more instructions on setting up Oracle Data Mining see: http://www.oracle.com/technetwork/database/options/odm/dataminerworkflow-168677.html
  • When you install the Oracle Database, the Oracle Data Mining Examples including sample data is available as part of the total Database installation.  See link.  

Step 2: Install SQL Developer 4.1 (the Oracle Data Miner GUI Extension installs automatically but additional post installation Set Up in required.  See Setting Up Oracle Data Miner )

Step 3: Follow the six (6) free step-by-step Oracle-by-Examples Tutorials:

That’s it!  Easy, fun and the fastest way to get started evaluating Oracle Advanced Analytics/Oracle Data Mining.  Enjoy!  

Charlie

<script type="text/freezescript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-46756583-1']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script>

Wednesday Feb 08, 2012

Oracle Announces Availability of Oracle Advanced Analytics for Big Data

Oracle Announces Availability of Oracle Advanced Analytics for Big Data

Oracle Integrates R Statistical Programming Language into Oracle Database 11g

REDWOOD SHORES, Calif. - February 8, 2012

News Facts

  • Oracle today announced the availability of     Oracle Advanced Analytics, a new option for Oracle Database 11g that bundles Oracle R Enterprise together with Oracle Data Mining.
  • Oracle R Enterprise delivers enterprise class performance for users of the R statistical programming language, increasing the scale of data that can be analyzed by orders of magnitude using Oracle Database 11g.
  • R has attracted over two million users since its introduction in 1995, and Oracle R Enterprise dramatically advances capability for R users. Their existing R development skills, tools, and scripts can now also run transparently, and scale against data stored in Oracle Database 11g.
  • Customer testing of Oracle R Enterprise for Big Data analytics on Oracle Exadata has shown up to 100x increase in performance in comparison to their current environment.
  • Oracle Data Mining, now part of Oracle Advanced Analytics, helps enable customers to easily build and deploy predictive analytic applications that help deliver new insights into business performance. Oracle Advanced Analytics, in conjunction with Oracle Big Data Appliance, Oracle Exadata Database Machine and Oracle Exalytics In-Memory Machine, delivers the industry’s most integrated and comprehensive platform for Big Data analytics.

Comprehensive In-Database Platform for Advanced Analytics

  • Oracle Advanced Analytics brings analytic algorithms to data stored in Oracle Database 11g and Oracle Exadata as opposed to the traditional approach of extracting data to laptops or specialized servers.
  • With Oracle Advanced Analytics, customers have a comprehensive platform for real-time analytic applications that deliver insight into key business subjects such as churn prediction, product recommendations, and fraud alerting.
  • By providing direct and controlled access to data stored in Oracle Database 11g, customers can accelerate data analyst productivity while maintaining data security throughout the enterprise.
  • Powered by decades of Oracle Database innovation, Oracle R Enterprise helps enable analysts to run a variety of sophisticated numerical techniques on billion row data sets in a matter of seconds making iterative, speed of thought, and high-quality numerical analysis on Big Data practical.
  • Oracle R Enterprise drastically reduces the time to deploy models by eliminating the need to translate the models to other languages before they can be deployed in production.
  • Oracle R Enterprise integrates the extensive set of Oracle Database data mining algorithms, analytics, and access to Oracle OLAP cubes into the R language for transparent use by R users.
  • Oracle Data Mining provides an extensive set of in-database data mining algorithms that solve a wide range of business problems. These predictive models can be deployed in Oracle Database 11g and use Oracle Exadata Smart Scan to rapidly score huge volumes of data.
  • The tight integration between R, Oracle Database 11g, and Hadoop enables R users to write one R script that can run in three different environments: a laptop running open source R, Hadoop running with Oracle Big Data Connectors, and Oracle Database 11g.
  • Oracle provides single vendor support for the entire Big Data platform spanning the hardware stack, operating system, open source R, Oracle R Enterprise and Oracle Database 11g. To enable easy enterprise-wide Big Data analysis, results from Oracle Advanced Analytics can be viewed from Oracle Business Intelligence Foundation Suite and Oracle Exalytics In-Memory Machine.

Supporting Quotes

  • “Oracle is committed to meeting the challenges of Big Data analytics. By building upon the analytical depth of Oracle SQL, Oracle Data Mining and the R environment, Oracle is delivering a scalable and secure Big Data platform to help our customers solve the toughest analytics problems,” said Andrew Mendelsohn, senior vice president, Oracle Server Technologies.
  • “We work with leading edge customers who rely on us to deliver better BI from their Oracle Databases. The new Oracle R Enterprise functionality allows us to perform deep analytics on Big Data stored in Oracle Databases. By leveraging R and its library of open source contributed CRAN packages combined with the power and scalability of Oracle Database 11g, we can now do that,” said Mark Rittman, co-founder, Rittman Mead.

Supporting Resources

About Oracle

Oracle engineers hardware and software to work together in the cloud and in your data center. For more information about Oracle (NASDAQ: ORCL), visit http://www.oracle.com.

Trademarks

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Contact Info

Eloy Ontiveros
Oracle
+1.650.607.6458

eloy.ontiveros@oracle.com

Joan Levy
Blanc & Otus for Oracle
+1.415.856.5110

jlevy@blancandotus.com

Monday Sep 19, 2011

Building Predictive Analytical Applications using Oracle Data Mining recorded webcast

I did a Building Predictive Analytical Applications using Oracle Data Mining recorded webcast for IOUG earlier this week.  If this interests you, you can either watch the streaming presentation and demo hosted by the Independent Oracle User Group at http://www.ioug.org/ in conjunction with the Oracle Business Intelligence, Wartehousing and Analytics Special Interest Group (BIWA SIG at www.oraclebiwa.org) or the download the 84 MB file by clicking on this link to the Building Predictive Analytical Applications using Oracle Data Mining.wmv file.   It included an overview of data mining, Oracle Data Mining, some demo slides, and then several example applications where we've factory-installed ODM predictive analytics methodologies into the Appls for self-learning and real-time deployment of ODM models. 

Example Predictive Analytics Applications (partial list)
  • Oracle Communications  & Retail Industry Models —factory installed data mining for specific industries
  • Oracle Spend Classification
  • Oracle Fusion Human Capital Management (HCM)  Predictive Workforce
  • Oracle Fusion Customer Relationship Management (CRM) Sales Prediction
  • Oracle Adaptive Access Manager real-time Security
  • Oracle Complex Event Processing integrated with ODM models
  • Predictive Incident Monitoring Service for Oracle Database customers

Pretty cool stuff if you or your customers are interested in analytics.  Here's the link to the ppt slides.    

Tuesday Aug 09, 2011

SAIL-WORLD article - America's Cup: Oracle Data Mining supports crew and BMW ORACLE Racing

Originally printed at http://www.sail-world.com/UK/Americas-Cup:-Oracle-Data-Mining-supports-crew-and-BMW-ORACLE-Racing/68834

America's Cup: Oracle Data Mining supports crew and BMW ORACLE Racing


  <script language="freezescript"> </script>
'USA-17 on her way to winning the 33rd America’s Cup, use of Oracle’s datamining technology and Oracle Database 11g and Oracle Application Express'    BMW Oracle Racing © Photo Gilles Martin-Raget    Click Here to view large photo

BMW ORACLE Racing won the 33rd America’s Cup yacht race in February 2010, beating the Swiss team, Alinghi, decisively in the first two races of the best-of-three contest.

BMW ORACLE Racing’s victory in the America’s Cup challenge was a lesson in sailing skill, as one of the world’s most experienced crews reached speeds as fast as 30 knots. But if you listen to the crew in their postrace interviews, you’ll notice that what they talk about is technology.

The wrist PDA displays worn by five of the USA-17 crew - where they could read actual and predictive data fed back from the onboard systems -  .. .   Click Here to view large photo

'The story of this race is in the technology,' says Ian Burns, design coordinator for BMW ORACLE Racing.

From the drag-resistant materials encasing its hulls to its unprecedented 223-foot wing sail, the BMW ORACLE Racing’s trimaran, named USA, is a one-of-a-kind technological juggernaut. No less impressive are the electronics used to guide the vessel and fine-tune its performance.

Each crewmember is equipped with a PDA on his wrist that has customized data for his job: what the load balance is on a particular rope, for example, or the current aerodynamic performance of the wing sail. The helmsman’s sunglasses display graphical and numeric data to help him fine-tune the boat’s direction while he keeps two hands on the wheel and visually scans the sea, the boat, the crew, the sails, and the wing.

The America’s Cup is a challenge-based competition in which the winning yacht club hosts the next event and, within certain guidelines, makes the rules. For the 33rd America’s Cup, the competing teams could not agree on a set of rules, so the event defaulted to an unrestricted format for boat design and cost.

'All we knew were the length of the boat and the course configuration,' says Burns. The boats were allowed a maximum length of 90 feet, and the course would be 20 miles out windward and 20 miles back. 'Within those parameters,' says Burns, 'you could build as fast a thing as you can think of.'

Learning by Data

The no-holds-barred rules for this race created what Burns calls an 'open playground' for boat designers. The innovative and costly vessels that resulted were one-of-a-kind creations with unpredictable sailing characteristics that would require a steep learning curve and lots of data.

33rd America’s Cup - BMW ORACLE Racing - Training in Valencia - collecting data via 250 sensors, managing it and analysing it were handled on the yacht, on the tender and ashore in Valencia and in the Austin Data Centre, USA. -  BMW Oracle Racing © Photo Gilles Martin-Raget   Click Here to view large photo

'One of the problems we faced at the outset was that we needed really high accuracy in our data because we didn’t have two boats,' says Burns. 'Generally, most teams have two boats, and they sail them side by side. Change one thing on one boat, and it’s fairly easy to see the effect of a change with your own eyes.'

With only one boat, BMW ORACLE Racing’s performance analysis had to be done numerically by comparing data sets. To get the information needed, says Burns, the team had to increase the amount of data collected by nearly 40 times what they had done in the past.

The USA holds 250 sensors to collect raw data: pressure sensors on the wing; angle sensors on the adjustable trailing edge of the wing sail to monitor the effectiveness of each adjustment, allowing the crew to ascertain the amount of lift it’s generating; and fiber-optic strain sensors on the mast and wing to allow maximum thrust without overbending them.

33rd America’s Cup - BMW ORACLE Racing - Day 1 - The difference between the wingsail and softsail is evident - even though the softsail has more area -  BMW Oracle Racing: Guilain Grenier   Click Here to view large photo


But collecting data was only the beginning. BMW ORACLE Racing also had to manage that data, analyze it, and present useful results. The team turned to Oracle Data Mining in Oracle Database 11g.

Peter Stengard, a principal software engineer for Oracle Data Mining and an amateur sailor, became the liaison between the database technology team and BMW ORACLE Racing. 'Ian Burns contacted us and explained that they were interested in better understanding the performance-driving parameters of their new boat,' says Stengard. 'They were measuring an incredible number of parameters across the trimaran, collected 10 times per second, so there were vast amounts of data available for analysis. An hour of sailing generates 90 million data points.'

After each day of sailing the boat, Burns and his team would meet to review and share raw data with crewmembers or boat-building vendors using a Web application built with Oracle Application Express. 'Someone in the meeting would say, 'Wouldn’t it be great if we could look at some new combination of numbers?’ and we could quickly build an Oracle Application Express application and share the information during the same meeting,' says Burns. Later, the data would be streamed to Oracle’s Austin Data Center, where Stengard and his team would go to work on deeper analysis.

BMW Oracle USA-17 powers thru Alinghi - America’s Cup 2010 Race 1 -  BMW Oracle Racing © Photo Gilles Martin-Raget   Click Here to view large photo


Because BMW ORACLE Racing was already collecting its data in an Oracle database, Stengard and his team didn’t have to do any extract, transform, and load (ETL) processes or data conversion. 'We could just start tackling the analytics problem right away,' says Stengard. 'We used Oracle Data Mining, which is in Oracle Database. It gives us many advanced data mining algorithms to work with, so we have freedom in how we approach any specific task.'

Using the algorithms in Oracle Data Mining, Stengard could help Burns and his team learn new things about how their boat was working in its environment. 'We would look, for example, at mast rotations—which rotation works best for certain wind conditions,' says Stengard. 'There were often complex relationships within the data that could be used to model the effect on the target—in this case something called velocity made good, or VMG. Finding these relationships is what the racing team was interested in.'

BMW Oracle Racing Technology team -  Richard Gladwell   Click Here to view large photo

Stengard and his team could also look at data over time and with an attribute selection algorithm to determine which sensors provided the most-useful information for their analysis. 'We could identify sensors that didn’t seem to be providing the predictive power they were looking for so they could change the sensor location or add sensors to another part of the boat,' Stengard says.

Burns agrees that without the data mining, they couldn’t have made the boat run as fast. 'The design of the boat was important, but once you’ve got it designed, the whole race is down to how the guys can use it,' he says. 'With Oracle database technology, we could compare our performance from the first day of sailing to the very last day of sailing, with incremental improvements the whole way through. With data mining we could check data against the things we saw, and we could find things that weren’t otherwise easily observable and findable.'

BMW Oracle Racing made 4000 data measurements 10 times a second -  BMW Oracle Racing: Guilain Grenier   Click Here to view large photo

Flying by Data

The greatest challenge of this America’s Cup, according to Burns, was managing the wing sail, which had been built on an unprecedented scale. 'It is truly a massive piece of architecture,' Burns says. 'It’s 20 stories high; it barely fits under the Golden Gate Bridge. It’s an amazing thing to see.'

The wing sail is made of an aeronautical fabric stretched over a carbon fiber frame, giving it the three-dimensional shape of a regular airplane wing. Like an airplane wing, it has a fixed leading edge and an adjustable trailing edge, which allows the crew to change the shape of the sail during the course of a race.

Oracle wing under maintenance - standing 70 metres high it is the longest wing ever build for a plane or yacht -  Jean Philippe Jobé   Click Here to view large photo


Next Steps

'The crew of the USA was the best group of sailors in the world, but they were used to working with sails,' says Burns, 'Then we put them under a wing. Our chief designer, Mike Drummond, told them an airline pilot doesn’t look out the window when he’s flying the plane; he looks at his instruments, and you guys have to do the same thing.'

A second ship, known as the performance tender, accompanied the USA on the water. The tender served in part as a floating datacenter and was connected to the USA by wireless LAN.

USA-17 about to round the windward mark, Race 1, 33rd America’s Cup. Under performing senors on the boat were moved to provide better information. -  Richard Gladwell  


'The USA generates almost 4,000 variables 10 times a second,' says Burns. 'Sometimes the analysis requires a very complicated combination of 10, 20, or 30 variables fitted through a time-based algorithm to give us predictions on what will happen in the next few seconds, or minutes, or even hours in terms of weather analysis.'

Like the deeper analysis that Stengard does back at the Austin Data Center, this real-time data management and near-real-time data analysis was done in Oracle Database 11g. 'We could download the data to servers on the tender ship, do some quick analysis, and feed it right back to the USA,' says Burns.

'We started to do better when the guys began using the instruments,' Burns says. 'Then we started to make small adjustments against the predictions and started to get improvements, and every day we were making gains.'

Those gains were incremental and data driven, and they accumulated over years—until the USA could sail at three times the wind speed. Ian Burns is still amazed by the spectacle.

'It’s an awesome thing to watch,' he says. 'Even with all we have learned, I don’t think we have met the performance limits of that beautiful wing.'

USA-17 pursues Alinghi 5 - Race 1, 33rd America’s Cup, Valencia. Her crew flew her off the instruments 'a pilot doesn’t fly a plane by looking out the window'. -  BMW Oracle Racing: Guilain Grenier   Click Here to view large photo

Read more about Oracle Data Mining

Hear a podcast interview with Ian Burns

Download Oracle Database 11g Release 2

Story republished from: www.oracle.com/technology/oramag/oracle/10-may/o30racing.html

by Jeff Erickson Share   11:41 PM Sat 24 Apr 2010 GMT


Thursday Jul 14, 2011

Oracle Fusion Human Capital Management Application uses Oracle Data Mining for Workforce Predictive Analytics

Oracle's new Fusion Human Capital Management (HCM) Application now embeds predictive analytic models automatically generated by Oracle Data Mining to enrich dashboards and manager's portals with predictions about the likelihood that an employee with voluntarily leave the organization and a prediction about the employee's likely future performance. Armed with this new information that is based on historical patterns and relationships found by Oracle Data Mining, enterprises can more proactively manage their valuable employee assets and better compete. The integrated Oracle Fusion HCM Application requires the Oracle Data Mining Option to the Oracle Database. With custom predictive models generated using the customer's own data, Oracle Fusion HCM enables managers to better understand the employees, understand the key factors for each individual and even perform "What if?" analysis to see the likely impact on an employee by adjusting a critical HR factor e.g. bonus, vacation time, amount of travel, etc.

Excerpting from the Oracle Fusion HCM website and collateral: "Every day organizations struggle to answer essential questions about their workforce. How much money are we losing by not having the right talent in place and how is that impacting current projects? What skills will we need in the next 5 years that we don’t have today? How will business be impacted by impending retirements and are we prepared? Fragmented systems and bolt-on analytics are only some of the barriers that HR faces today. The consequences include missed opportunities, lost productivity, attrition, and uncontrolled operational costs. To address these challenges, Oracle Fusion Human Capital Management (HCM)puts information at your fingertips, helps you predict future trends, and enables you to turn insight into action. You will eliminate unnecessary costs, increase workforce productivity and retention, and gain a strategic advantage over your competition. Oracle Fusion HCM has been designed from the ground up so that you can work naturally and intuitively with analytics woven right into the fabric of your business processes."

 

This exceprt from the Solution Brief http://www.oracle.com/us/products/applications/fusion/fusion-hcm-know-your-people-356192.pdf describes the Predictive Analytics features and benefits: "Every day organizations struggle to answer essential questions about their workforce. How much money are we losing by not having the right talent in place and how is that impacting current projects? What skills will we need in the next 5 years that we don’t have today? How will business be impacted by impending retirements and are we prepared? Fragmented systems and bolt-on analytics are only some of the barriers that HR faces today. The consequences include missed opportunities, lost productivity, attrition, and uncontrolled operational costs. To address these challenges, Oracle Fusion Human Capital Management (HCM) puts information at your fingertips, helps you predict future trends, and enables you to turn insight into action. You will eliminate unnecessary costs, increase workforce productivity and retention, and gain a strategic advantage over your competition. Oracle Fusion HCM has been designed from the ground up so that you can work naturally and intuitively with analytics woven right into the fabric of your business processes." ....

"Predictive Analysis Imagine if you could look ahead and be prepared for upcoming workforce trends. Most organizations do not have the analytic capability to do predictive human capital analysis, yet the worker information needed to make educated forecasts already exists today. Aging populations, shifting demographics, rising and falling economies, and multi-generational issues can have a significant impact on workforce decisions – for employees, managers and HR professionals. Not being able to accurately predict how all the moving parts fit together, and where you really have potential problems, can make or break an organization. Oracle Fusion HCM gives you the ability to finally see into the future, analyzing worker performance potential, risk of attrition, and enabling what-if analysis on ways to improve your workforce. Additionally, modeling capabilities provide you with extra power to bring together information from sources unthinkable in the past. For example, imagine understanding which recruiting agencies are providing the highest-quality recruits by comparing first year performance ratings with sources of hire. Being able to see potential problems before they occur and take immediate action will increase morale, save money, and boost your competitive edge. Result: You are able to look ahead and be prepared for upcoming workforce trends."

There is a great demo of Oracle Fusion HCM Workforce Predictive Analytics that highlights the Oracle Data Mining.  This is one of the latest examples of Applications "powered by Oracle Data Mining".

 

Employee grid

When you change your paradigm and move the algorithms to the data rather than the traditional approach of extracting the data and moving it to the algorithms for analysis, it CHANGES EVERYTHING. Keep watching for additional Applications powered by Oracle's in-database advanced analytics.

Friday Apr 29, 2011

Explore graph

[Read More]

Tuesday Apr 05, 2011

Recorded Oracle Data Mining Demo and Presentation

[Read More]

Tuesday Mar 29, 2011

How to Deploy an Oracle Data Miner 11gR2 Workflow Model

[Read More]

Sunday Oct 31, 2010

Effective Hypertensive Treatment using Oracle Data Mining in Saudi Arabia

[Read More]
About

Everything about Oracle Data Mining, a component of the Oracle Advanced Analytics Option - News, Technical Information, Opinions, Tips & Tricks. All in One Place

Search

Categories
Archives
« May 2016
SunMonTueWedThuFriSat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    
       
Today