In Pursuit of the Holy Grail - MySQL Cluster and Push Down JOINs

UPDATE - 15TH NOVEMBER
An on-demand replay of the webinar discussed below is available from the following URL:
http://www.mysql.com/news-and-events/on-demand-webinars/display-od-583.html

The development team would highly value feedback on your experience with the Push Down Joins project, and whether any current limitations prevent you from using this functionality.
Please send feedback or questions via the following mailing list:
spj-feedback@sun.com

You can access the binary (Linux-only) and source here:
ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/mysql-5.1.51-ndb-7.1.9-spj-preview/

ORIGINAL BLOG POSTING:
Ok, so the title maybe a little melo-dramatic, but for those who work with databases that distribute tables and rows across multiple nodes, pursuing the holy grail may not be that far from the truth.

We are running a live webinar to discuss on-going developments in the MySQL engineering group that are focused on enhancing JOIN performance.  Maybe not quite the "miraculous power" promised by the holy grail, but pretty exciting nonetheless !

Why is this even necessary, you may ask ?
Well, there are many advantages to distributing data across a cluster of database nodes:
-    very high levels of Read and Write performance, especially when you have relatively simple access patterns to the data.  Telemaque in France reported over 330,000 operations per second across just 4 x data nodes in a recent case study discussing their deployment of MySQL Cluster for call center services   A recent blog posting shows MySQL Cluster delivering 180,000 primary key SELECTs per second and 120,000 updates per second  on just 2 x data nodes.

-    The ability to scale-out on low cost commodity nodes, enabling highly performant database clusters to be assembled from low cost building blocks

-    With replication of data between cluster nodes and automatic failure detection, failover and recovery, the ability to deliver 99.999%+ availability (or just 5 minutes of downtime per year)

This is all good.  Unless you want to run queries that need to join multiple tables and are returning thousands of rows. Performance for complex multi-table JOIN operations will be low as multiple nodes need to be accessed many times to fulfill the query.

Pushing down JOIN operations to the data nodes is intended to address such limitations, therefore broadening the range of applications that can use the MySQL Cluster database, and get the types of benefits discussed above.

In some testing we've run in the labs - we've seen 20-40x performance improvements.  Some on real workloads, some on totally synthetic, unrealistic benchmarks....but you can try it out for yourself

The Push Down JOIN project is currently in development and a pre-release version will be available for you to try with your own queries - and hopefully to provide feedback to our developers.

The webinar on November 4th will provide an insight into this work. You can learn more about what we'll cover from the registration page.
 
Even if you can't make November 4th, it is still worth registering for the event as you will be automatically notified when the replay is available.

If you want to learn how to optimize performance of the current generation of MySQL Cluster, then there is an excellent whitepaper giving you all the latest best practices

Now, for the legal bit:
The existence, content and timing of the development project described here is included for information only and may be changed at Oracle's discretion.

Hope you will be able to join on November 4th (or for the replay) to learn more about this exciting development

Comments:

I’d be inclined to acquiesce with you here. Which is not something I usually do! I really like reading a post that will make people think. Also, thanks for allowing me to comment!

Posted by Adalberto Rider on November 08, 2010 at 08:24 AM GMT #

This is such a great post, and was thinking much the same myself. I will remember this.

Posted by Article Marketing on November 23, 2010 at 03:21 AM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Get the latest updates on products, technology, news, events, webcasts, customers and more.

Twitter


Facebook

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
2
5
6
9
10
11
12
13
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today