Wednesday Sep 26, 2012

See a Big Data Appliance in the wild

It is a news day today, check out the pictures of the BDA getting wheeled into Enkitec...

Kerry's Blog Post

Always fun to see  happy faces when the new to arrives. Looking forward to more Enkitec news in the coming weeks (starting at Openworld of course).

Big data: An evening in the life of an actual buyer

Here I am, and this is an actual story of one of my evenings, trying to spend money with a company and ultimately failing. I just gave up and bought a service from another vendor, not the incumbent. Here is that story and how I think big data could actually fix this (and potentially prevent some of this from happening). In the end this story should illustrate how big data can benefit me (get me what I want without causing grief) and the company I am trying to buy something from.

Note: Lots of details left out, I have no intention of being the annoyed blogger moaning about a specific company.

What did I want to get?

We watch TV, we have internet and we do have a land line. The land line is from a different vendor then the TV and the internet. I have decided that this makes no sense and I was going to get a bundle (no need to infer who this is, I just picked the generic bundle word as this is what I want to get) of all three services as this seems to save me money. I also want to not talk to people, I just want to click on a website when I feel like it and get it all sorted. I do think that is reality. I want to just do my shopping at 9.30pm while watching silly reruns on TV.

Problem 1 - Bad links

So, I'm an existing customer of the company I want to buy my bundle from. I go to the website, I click on offers. Turns out they are offers for new customers. After grumbling about how good they are, I click on offers for existing customers. Bummer, it goes to offers for new customers, so I click again on the link for offers for existing customers. No cigar... it just does not work.

Big data solutions:

1) Do not show an existing customer the offers for new customers unless they are the same => This is only partially doable without login, but if a customer logs in the application should always know that this is an existing customer. But in general, imagine I do this from my home going through the internet service of this vendor to their domain... an instant filter should move me into the "existing customer route".

2) Flag dead or incorrect links => I've clicked the link for "existing customer offers" at least 3 times in under 5 seconds... Identifying patterns like this is easy in Hadoop and can very quickly make a list of potentially incorrect links. No need for realtime fixing, just the fact that this link can be pro-actively fixed across my entire web domain is a good thing. Preventative maintenance!

Problem 2 - Purchase cannot be completed

Apart from the fact that the browsing pattern to actually get to what I want is poorly designed, my purchase never gets past a specific point. In other words, I put something into my shopping cart and when I want to move on the application either crashes (with me going to an error page) or hangs or goes into something like chat. So I try again, and again and again. I think I tried this entire path (while being logged in!!) at least 10 times over the course of 20 minutes. I also clicked on the feedback button and, frustrated as I was, tried to explain this did not work...

Big Data Solutions:

1) This web site does shopping cart analysis. I got an email next day stating I have things in my shopping cart, just click here to complete my purchase. After the above experience, this just added insult to my pain...

2) What should have happened, is a Hadoop job going over all logged in customers that are on the buy flow. It should flag anyone who is trying (multiple attempts from the same user to do the same thing), analyze the shopping card, the clicks to identify what the customers wants, his feedback provided (note: always own your own website feedback, never just farm this out!!) and in a short turn around time (30 minutes to 2 hours or so) email me with a link to complete my purchase. Not with a link to my shopping cart 12 hours later, but a link to actually achieve what I wanted...

Why should this company go through the big data effort?

I do believe this is relatively easy to do using our Oracle Event Processing and Big Data Appliance solutions combined. It is almost so simple (to my mind) that it makes no sense that this is not in place? But, now I am ranting... Why is this interesting? It is because of $$$$.

After trying really hard, I mean I did this all in the evening, and again in the morning before going to work. I kept on failing, But I really wanted this to work... so an email that said, sorry, we noticed you tried to get a bundle (the log knows what I wanted, where I failed, so easy to generate), here is the link to click and complete your purchase. And here is 2 movies on us as an apology would have kept me as a customer, and got the additional $$$$ per month for the next couple of years. It would also lead to upsell on my phone package etc.

Instead, I went to a completely different company, bought service from them. Lost money for company A, negative sentiment for company A and me telling this story at the water cooler so I'm influencing more people to think negatively about company A. All in all, a loss of easy money, a ding in sentiment and image where a relatively simple solution exists and can be in place on the software I describe routinely in this blog...

For those who are coming to Openworld and maybe see value in solving the above, or are thinking of how to solve this, come visit us in Moscone North - Oracle Red Lounge or in the Engineered Systems Showcase.

Wednesday Sep 12, 2012

Big Data Sessions at Openworld 2012

If you are coming to San Francisco, and you are interested in all the aspects to big data, this Focus On Big Data is a must have document. 

Some (other) highlights:

  • A performance demo of a full rack Big Data Appliance in the engineered systems showcase
  • A set of handson labs on how to go from a NoSQL DB to an effective analytics play on big data
  • Much, much more

See you all in a few weeks in SF!

Wednesday Sep 05, 2012

E-Book on big data (featuring Analysts, Customers and more)

As we are gearing up for Openworld, here is a nice E-book on big data to start paging through. It contains Gartner's take on big data, customer and partner interviews and a lot more good info. Enjoy the read so you come prepared for Openworld!!

Read the E-Book here.

For those coming to Oracle Openworld (or the Americas Cup races around the same time), you can find big data sessions via this URL.


Tuesday Jul 10, 2012

Big Data Connectors White Paper

Another quick link for those interested:

This white paper describes the performance characteristics of Oracle Big Data Connectors.

Monday Jul 09, 2012

Big Data Videos

You can view them all on YouTube using the following links:

  1. Overview for the Boss:
  2. Hadoop:
  3. Acquiring Big Data:
  4. Organizing Big Data:
  5. Analyzing Big Data:

These videos are a great place to start learning about big data, the value it can bring to your organisation and how Oracle can help you start working with big data today.

Wednesday Jun 27, 2012

Live from ODTUG - Big Data and SQL session #2

Sitting in Dominic Delmolino's session at ODTUG (KScope 12). If the session count at conferences is any indication then we will see more and more people start to deploy MapReduce in the database. And yes, that would be with SQL and PL/SQL first and foremost. Both Dominic and our own Bryn Llewellyn are doing MapReduce in the database presentations. 

Since I have seen both, I would advice people to first look through Dominic's session to get a good grasp on what mappers do and what reducers do, then dive into Bryn's for a bunch of PL/SQL example. The thing I like about Dominic's is the last slide (a recursive WITH statement) to do this in SQL...

Now I am hoping that next year we will see tools vendors show off how they work with Hadoop and MapReduce (at least talking about the concepts!!).

Tuesday May 22, 2012

Following up on a comment

Since I was on vacation I missed an interesting question on one of the posts:

I see the interesting pyramid in the middle of your article showing the current status of big data market in the many eyes of the beholders and have made many thoughts on that. In your opinion, do you think "Big Data is our business" is only applicable for those Internet-scale companies, independent software vendors OR it will happen soon for most large-scale corporations in the upcoming future? I will think that "Big Data improves our business" shall be applicable to those large-scale corporate and up to that line only in the mainstream. Would like to hear more comments from your side especially on the use of Big Data for Corporate IT or in the Corporate level.



Posted by Ivan Ng on April 25, 2012 at 04:23 AM PDT #


I think you can argue about this quite a bit, but I do believe that top tier is something we should be able to achieve, even if we are not an IT, Web or Data company. But you make a good point. At its current state, the wording is a bit off and would lead people to believe what you said... Does a manufacturing company really become a data company?

Maybe it should read "Big Data Transforms Our Business" ... which is what I am really after. Improvements are good, creativity should make us leverage data and change the outcomes of business processes. Then I think it is achievable for everyone...

Then a manufacturing company can transform the business, a telecom company can all of sudden personalize every interaction with every customer.

Thanks for the great comment, it certainly helped formulating some of this more crisp and I hope it makes sense.

Monday May 14, 2012

Price comparison for Big Data Appliance

Untitled Document

We are seeing a lot of discussions about pricing and comparable pricing between a "standard Hadoop cluster" and the Oracle Big Data Appliance. This post is aimed at providing a simple apples-to-apples comparison and a clarification of what is, and what is not included in the pricing and packaging of Oracle Big Data Appliance.

Read the updated post here.

Wednesday May 09, 2012

Big Data: BDA Networking for the Data Center

Quick note, just published a nice comprehensive paper on how to network a Big Data Appliance into your data center. You can find the paper on the main BDA OTN page. Direct link here.

Happy networking...

Tuesday Apr 17, 2012

Read how the Avengers (tm) use Oracle Big Data Appliance

Decisive Power with Oracle Big Data Appliance

Raw information—like raw power—must be channeled and transformed to be useful. Every day, S.H.I.E.L.D. captures terabytes of information from a variety of sources—surveillance videos, satellites, sensors, field reports, network traffic—and all of this high-volume, high-velocity, and high-variety data is processed, filtered, transformed, and sorted in Oracle Big Data Appliance.

Read all about the new Marvel "The Avengers" movie, the systems in their data center and their use of big data here.

Tuesday Apr 10, 2012

Self-Study course on ODI Application Adapter for Hadoop

For those of you who want to work with Oracle Data Integrator and its Hadoop capabilities, a good way to start is the newly released self-study course from Oracle University. You can find the course here.

Enjoy, and if you have any feedback, do send this into Oracle University by logging in (so we can unleash our big data analytics on it ;-) ).

Monday Apr 09, 2012

Big Data - Must Bring Creativity

As everyone reads more about big data, I get more and more questions on use cases. “How should we use big data?” is the most common question. “Are there applications available for my vertical?” and “what do others in my vertical do with big data?” are a close second on the list. There are various reports out there by many authors which describe possible or real use cases. A search on the beacon of truth will no doubt get you links to most, but are they really relevant?

Then when I was reading Fortune Magazine (I do read the paper copy – guess I am old fashioned) the graph of disconnected points in my brain finally connected while reading the edition with the Facebook cover article (volume 165, number 4). Interestingly however, the Facebook articla was not the article that got my dots connected and persuaded me to writing this post.

The article that connected the dots was about the re-invention of JCPenney. More precisely about Ron Johnson, JCPenney’s new CEO and his thoughts on how to re-invent a department store business. For those who do not know who Ron Johnson is, it is the person who – together with Steve Jobs – created the Apple Store for you. That would be the physical, brick-and-mortar one, the hugely successful one (more $$$ per square foot than any other store)…

His quote in the article (following quote is of course copyright Fortune Magazine!) says it all:

“Improvement merely lets you hit the numbers. Creativity is what transforms.” Ron Johnson, CEO, JCPenney

Again, I think that quote says it all, it tells you that you will not find the transformational use case for your business in all those articles and papers on the internet. It says that you will have to dig deeper into the actual business you are running, and try to find that one thing, that one idea that will transform your business. I’m going to guess that this idea – if you are reading this blog post – is somehow related to having more data, or doing much more analytics…

With that in mind look at the following picture. It gives you an idea of the implementations in production of big data solutions (Gartner’s Mark Beyer often uses the 5-20-75 scale to explain where technology is):

What this really means is that big data is not yet in the mainstream of technology deployments. It also means that you will not be able to buy an entire solution off the shelve, that instead, you will have to be the first one to implement that crazy successful idea. That is a good thing! It means that there is so much competitive advantage to be had by investing in big data now…

I then caught a glimpse of Oracle Profit (Volume 17, Number 1), which quotes Mark Hurd, Oracle’s President:

“Technology presents the opportunity to Transform Business”

Creativity and technology, now that is something that will really transform a business. And that is why I am writing this post! We – Oracle – can give you the technology platform, we can even give you analytical building blocks (LEGO for your analytics), but you, your business people, are in charge of that big idea.

How do you start with Big Data projects?

I hope the above at least tickled your brain, because the following will want you to believe! But before we change every business, let’s get grounded in what a real start to a big data project is. For many of us, our first big data project is going to be something that needs to prove “it” can be done.

Step 1) Gather the required technology and people to get a real project done. You will need at least (access to) the following:

  • Advanced analytics (statistics, data mining, graph analytics, semantics, spatial etc.)
  • A technology foundation that can handle a lot of data and allows you to analyze that data (it may not be fully at scale). Make sure you understand what technologies can do what for you, and leverage the internet of ideas to jump start your knowledge on these technologies
  • A data set you know, understand, that at least supports the basic idea you are pursuing in your improvement. For example, if you are going to work on social graph driven churn analysis, you will need both your customer data and relevant social data
  • A small number of people to work with and drive the above – you might not have them all ready to go, so find someone who is willing to learn and excited about technology…

Step 2) Find a problem you have today (risk, fraud, CRM etc.), that actually costs you money and improve on that problem using the stuff you gathered in step 1. This is your playground, this is your chance to learn (!) and in the process improve something in your organization. It also will prove this analytics and big data stuff really works and drives business value. Your known problem is something that gives you a quantifiable ROI. I’ve shortened the time to do X, adding $Y in our coffers by spending only $Z  (where Z < Y)… Oh, and make sure you do this in the shortest possible time frame or you are going to be too late to the party.

For getting things done in #2 (and #4 below), you should actually read the Fortune article mentioned above on how Facebook works, and try to understand the mentality and method applied to new products and features. Build, show and improve or rethink but never take “no” for an answer if you believe in something.

Step 3) Build a production environment and do it all on real hardware and software. Potentially re-evaluate some of the technology you had used before (this stuff better work in production). And then do it all again in step 4 when it really matters.

Step 4) Go catch the big fish! Go after the big idea; what is the new business this opens? That fantastic new component in your services. Rally the troops and go do it! Be creative, use the technology and forget the boundaries. Do make sure you find a corporate sponsor to see this through.

Once you arrive in Step 4, think big, build the first proto-type and never, ever hedge your bets. Go for the big idea, focus and don’t try to do many other things or try to spread the risk by not really doing anything new to avoid the risk of failure.

My Conclusion

IMHO, the way to real innovation by leveraging a big data solution is to first follow the money driven by improvement. Use that pilot to learn the technologies (analytics is key!) while solving a real tangible problem, then go all in and do the cool stuff.

PS. This is my motivational speech. I genuinely believe that most of the technology hurdles are gone, we just need to harness the creativity that naturally lives in a business, apply the technology and stick to the big ideas to transform business.

Tuesday Feb 28, 2012

Interesting IDC report on Oracle's Big Data Solutions

I thought I share this link. Very interesting report written by IDC on Oracle's Big Data strategy.

Happy Reading!

Thursday Feb 23, 2012

Announcing the Big Data OTN Forums

The new OTN forums for big data are now live.

Follow this link for the Forum home.

We have two forums, one covering big data as a topic, which includes Oracle Big Data Appliance and the Hadoop ecosystem, a second covering all the components in Oracle Big Data Connectors. The forums are monitored by Oracle and by folks in development, so we hope to provide you with excellent value on all your questions and ideas.


The data warehouse insider is written by the Oracle product management team and sheds lights on all thing data warehousing and big data.


« July 2014