Monday Jan 26, 2009

New York, Boston, Providence

Time for another East Coast tour -- not to worry am planning on a central and western tour as well, but we're still working out the details to see if it will happen!

Next week, I'll be attending a Sun Microsystems "Hatchery" event talking to startups in New York so I'll be in NYC Monday and Tuesday (Feb 2/3). Wednesday I'll Amtrak it to Boston where I'll meet the Boston MySQL Meetup and talk about MySQL 5.1's Event Scheduler. I'll then travel to Providence. I don't have any meetings scheduled for Providence but I'll work from a friend's house and come back on a cheap flight to Montreal the Monday after.

Will you be in any of these areas? Do you want to meet up? I have no problems ducking out during the day to meet any developer, mysql user, customer or dba. Just let me know on this blog how to get in touch with you or drop me an email to dups -at- sun -dot- com.

Now the cool part is that I will be trying my very best to be as environmentally friendly as possible on these travels so I have elected to take the 12 hour+ journey to New York City from Montreal by train. As I understand it my former colleague Morgan Tocker did it a while back and am hoping it will be somewhat relaxing and enjoyable (though I understand the border crossing is a pain). On the other hand, I get to walk downtown in Montreal, get on the train all relaxed, work on stuff on the computer in peace for the day, get out at Penn Station and walk to the hotel near the Sun office.

Friday Jan 23, 2009

"Grant All" to a Web DB User?

Please don't. Please, please don't. I don't know the number of times I have seen this, heck I know that certain software installations recommend you allowing a "grant all" to the user which will connect to the db from that software (see WordPress installation guide).

So... please don't.

One of the quotes I live by is one I remember from my university days and uttered by the German playwright Bertolt Brecht (I've never found the actual quote, if someone knows let me know) where he basically says, if you put a telephone on stage, use it.

MySQL grants for a user should only include what that user needs, never more, never less. If a piece of software, or if you as a programmer/dba decide to use "grant all", make sure the software or you actually needs that and remove it once it is not necessary. If not you are setting yourself up for disaster.

So here's what happens typically: You have a web server with a database and the user from the web site connects locally. You are fairly smart so you disable access to the database from the outside world and the user can only access from localhost. So far, so good. Now, there are very few ways you can hack and gain shell access onto a linux server that keeps its packages updated, so you feel fairly confident (you should never assume this btw). Your misplaced confidence leads you to believe you can give "all" access to a db user used by a web site. You have basically said that web user can do whatever they want. They can "drop" your database even.

How can someone do that without shell access or direct access to the DB? Well one way is that they can do it by means of SQL injection.

Wait! You say, "Wait! I know my code, I know my skill, I have taken care of SQL injection." Well, unless you are smarter than almost every coder I know and you do not use any third-party packages or software (like WordPress, or Gallery or whatever) then you are mistaken.

Why am I writing this? Well this week I witnessed the "trifecta": A directory opened for global public read with the SQL password in a plain text file, the grant statement for that user being "all" and a SQL injection possibility (even if minor and unlikely to do any damage) on a single site.

So please, even if accidentally, even if temporarily, be mindful of what grants you give what user.

Paul DuBois has an older but still relevant article on securing a MySQL installation. There is also notes on securing a MySQL Installation in the MySQL manual.

Thursday Jan 22, 2009

PlanetMySQL now in Japanese!

Language support and collation is an issue which affects most web sites with a global audience. Neither PlanetMySQL or are immune to these problems. The problem both these sites face is that neither site was designed with asian languages in mind, the database and collations and connections are all mixed and none of them in utf-8. It means migration is always an issue, a matter of downtime and not necessarily a sure thing.

With PlanetMySQL we really wanted Japanese and in the future Chinese character set support. As you might notice on PlanetMySQL as of late you can now choose to see a Japanese feed which has made our Japanese colleagues incredibly happy!

To be honest, I cheated.

There were a few problems to solve quickly, one was that the MySQL DB tables are not in the UTF-8 collation. Converting the table to UTF-8 was an option but I opted for the quicker solution which I'm sure will come back to haunt me in the future but for now provided the best solution while we look towards a greater redesign and reworking of PlanetMySQL in the future.

A second problem was that SimplePie which we use to do our feed aggregation was not writing in as utf-8. That was a simple switch and Lenz removed all traces of any reference to MagpieRSS which we used before SimplePie.

A third problem was that the pages were being served in the latin1 character set.

To solve these problems in the quickest time possible with no downtime, I added a character set/collation type into our current languages table, switched SimplePie to read and write as UTF-8 and fixed the web pages to display as UTF-8 for any new languages.

Sometime in the future I'll have to bite the bullet and convert those tables to UTF-8, but for now, this solution works and we were able to add this quite quickly over the holidays.

Over the next little while we'll have to expand our language collection, language choices and fix/improve the interface to choosing languages. As with all things, PlanetMySQL is under a renewed movement for renovation. If you have any ideas or suggestions we always like to hear them. I can't promise everything will be addressed, but we will put them on the list to consider.

Wednesday Jan 21, 2009

5 Days of Sun and MySQL

So it's been a year since we were all sitting thunderstruck in the ballroom of the Swan and Dolphin Hotel in Florida as Marten announced that we were being acquired by Sun Microsystems. To be honest, normally with these things people say it feels like "ages ago", in fact for me it feels like it just happened yesterday and it feels like we're only getting started.

One of the coolest things about Sun and the combination of Sun and MySQL is that Sun is a technology oriented company with some incredibly bright people who are there to come up with neat technology stuff during their waking hours. In some ways, that's very much like MySQL. After all, we build technologies that drive other technologies and business.

For the last year people have talked about how MySQL has been affected by Sun. But you know, influence is a two-way street. Former MySQL-ers (now called Sun Dolphins) are spread throughout the organization outside of the Database Group. We have people working on Drizzle elsewhere, others working on volume business, sales, support and other research projects. Also, some of our Sun colleagues were already busy using MySQL when they woke up to discover that MySQL was now Sun.

To celebrate a year of MySQL within Sun, Lenz and I have done five interviews which have been published each day this week. The interviews are focused on Sun "Classic" folks on their thoughts on having MySQL part of Sun over the past year. The interviews have been accompanied by wonderful banners from Aaron White of the MySQL Web Team and are available in the Developer Zone.

We have two more days to go, but I encourage you to have a look at the ones already done and follow along over the next couple of days:

: Vince Carbone talks about DTrace in Solaris and OpenSolaris and his work with MySQL.

Tuesday: Detlef Ulherr and Thorsten Früauf talk about Open HA Cluster and Solaris Cluster and how MySQL fits into the project.

Neelakanth Nadgir talks about MySQL and ZFS and how a file system could revolutionize your use of a database.

Thursday and Friday: Stay Tuned!

Monday Dec 22, 2008

More Interviews: Masood, Stewart and Lars

Somehow these interviews did not pop up on PlanetMySQL this past week so I link them here for your holiday reading. These are a couple interviews by Lenz Grimmer and one by yours truly.

Interview with Lars Heill, Release Engineering Manager
Born in Northern Norway 41 years ago and Lars has lived in Trondheim for the last 22 years. He is a Physicist by education, has a master degree on semiconductor heterojunctures and has earned a PhD on high temperature superconductors. He worked briefly on nuclear power fuel optimization and petroleum related rock mechanics before joining Clustra in the year 2000, which was acquired by Sun Microsystems in 2002. Clustra was a database software vendor that specialized in clustered, high-availability databases that were required by telecoms and service providers.

Interview with Stewart Smith, Drizzle/MySQL Cluster
Stewart Smith, a former member of the MySQL Cluster team recently decided to move on and work as a programmer on the Drizzle project. We wanted to catch with Stewart on both MySQL Cluster on Windows and what's he up to now.

Interview with Masood Mortazavi, MySQL Engineering Manager at Sun
Masood Mortazavi is an Engineering Manager at the Sun Database Group. After the acquisition of MySQL, and along with the rest of Sun's original database technology group, he joined the MySQL organization to form the larger Sun Database Group. In this interview, Masood talks with Lenz about the flexibility and diversity of Sun as a workplace, his life prior to joining Sun and his current assignment to improve the MySQL code contribution process.

Happy Holidays everyone, see you in 2009!

Ideas on Integrating Memcached into MySQL Queries

There's any number of ways to integrate your application with Memcached to take advantage of Memcached's power. Here's a list of some of them (and because I am most familiar with PHP in this case so that's what I've listed, and by no means is it exhaustive):

  1. Using the PECL PHP Memcached libraries you can write direct queries to Memcached with failover to your SQL queries.
  2. Using the memcached UDFs so that you can write SQL queries into MySQL/Using the memcached storage engine
  3. Using MySQL Proxy to interface with Memcached
  4. Using something from a framework such as Zend_Cache from the Zend Framework (which allows you to use more than one caching system btw.)

So, yes there are many ways to integrate Memcached into your PHP application, so might I suggest one more way.

The problem with the first option above is that your code tends to get littered with memcached calls, with the second options you end up having to modify your server. In many environments such as hosted environments this is not that clean. With the fourth option, you now need to use a framework potentially and potentially you may not want that overhead. The third option of using MySQL Proxy is one of my favourites but let's face it, MySQL Proxy is not GA yet, the available version has stability issues and the memcached scripts I've seen/heard about seem to use memcached as a full on query cache (please do correct me if I am wrong).

My belief is that memcached is a caching solution and it should be used by the developer wherever possible to make the application faster by placing/caching only the data that the developer needs. I also personally want my application to run when memcached is turned off and I want the application to be easy to read. In other words, I want a modification to the SQL query that will both work with memcached and MySQL but gives me control over what I want to save to memcached and what I need to expire/replace etc.

My solution which I tested over the last couple of weeks will only work if you already have the ability to modify/extend your database handler. At, for example, we use Zend Framework but for various reasons, including performance, we actually have our own custom database handler object. Most of my personal sites also do the same; I do not intend to move away from MySQL ;)

I do have to admit, my first crack at the syntax was quite clunky but in chatting with Adam Donnison (who will, by the way, be giving a beginners talk on Memcached at the MySQL Users Conference) we came up with the following.

SELECT /\*INTO MEMCACHED namespace=table key=id\*/ x, y, z FROM table WHERE id=1;

In this case the data will be stored into memcached with a key if table_1 and store an array of x, y and z. In my database handler this will easily parse the query and select from the MySQL database if it is not in memcached and on the way out save it into memcached for the next query.

To round out the queries, I also added support for things like

INSERT /\*REPLACE MEMCACHED namespace=table key=id\*/ ....


DELETE /\*EXPIRE MEMCACHED namespace=table key=id\*/ ....

I wanted to see how a real world use of this would work and so I rewrote my session handler for Zend Framework to take this into account and sure enough it works and it works well. Now my code is a lot neater, will always work with MySQL and I can move my memcached code around as I need it.

By rights, my perfect scenario is to now complete a MySQL Proxy script that understands the above and does the above then I could even remove the database handler code that does this all. To be honest though, the performance of this is quite good on my limited tests.

While I have not done it, I would imagine that extending Zend Framework to be able to handle these kind of queries should not be too difficult, nor is it difficult to simply use Zend_Cache into your database handler object and thereby even further enhancing your application's abilities to cache things.

Virtualbox 2.1

In case you weren't aware Virtualbox had a major update to 2.1 last week. I decided to give it a try on my Ubuntu host. One of the things I've complained about when it comes to Virtualbox has been its rather hard to set up Host Interface Networking on both Linux and Windows. Yes, it's not really that complicated with TUN/TAP interfaces or manual bridging, but for a program that's competing with the likes of Parallels and VMWare, the annoyance of Host Interface Networking setup is something I consider a potential "barrier-to-entry" for a normal, non-technical person.

Well, with 2.1 one of the features is support for a new way of doing Host Interface Networking, more like the Mac OSX version of Virtualbox. Very neat, you simply just select the active network interface and Vbox does all the work for you.

Now, there are probably bugs :) I just noticed on my both my CentOS guest and my Ubuntu Server guest I'm getting duplicate ICMP packets being returned. It's a start anyway!

Sunday Dec 21, 2008

Virtualbox and Shared Folders

A note on Virtualbox and Shared Folders. I installed Windows XP as a guest on my Virtualbox OSE on Ubuntu (8.10) and noticed that my Shared Folders to the host, in this case a Fat32 partition that I share with my Windows Vista dual boot came up extremely slow. A couple quick questions asked of the Google oracle, the following suggestion on one forum definitely helped me: map the network shared folder to a drive. Note: this is actually noted in the GUI when you go to add any Shared Folders -- well fairly small fine print at the bottom... I suspect there is some networking code somewhere when you go through the Windows network neighbourhood that makes accessing the shared folders through it extremely slow.

For those who don't use Windows very often, open a CMD window and type the following to map:

net use [Drive Letter]: \\\\vboxsvr\\[Shared folder name]


net use v: \\\\vboxsvr\\shared

Note: I am using the Virtualbox OSE which is currently part of the Ubuntu packages and is currently at version 2.0.4. Your mileage may vary!

Thursday Dec 11, 2008

Interview with Alexander "Salle" Keremidarski

I forgot to post about this but I uploaded a small catch up interview with Salle, the Manager for EMEA Support for MySQL on life and what it means to be in the Support Team of MySQL. Let me introduce you to another long time employee of MySQL :)

In defence of the bug

Lately I've heard from many blog posts and current and former colleagues on this thing we call a "bug". Frankly, I'm not the greatest giant fan of insects but to name a whole plethora of issues after an entire order of the animal kingdom, seems to be a bit harsh. I mean in some parts of the world, bugs are a delicacy. But yes, back to software bugs.

In this day and age, it's almost impossible to find a piece of software without a bug. I encounter a software bug every hour of every waking day and chances are you do to. Now admittedly some bugs are more serious than others, some are hilarious. Game bugs which do weird things and make you laugh for instance, are common and sometimes fun, if distracting.

Wired magazine has a history of what they call the most serious bugs in history, some of which have caused deaths.

Now, I'm not defending the practice of writing buggy code. Far from it. However, let's define buggy code?

echo "Hello World!";

The above is not buggy code right? What if I released that same code in a country which spells "Hello" as "Hallo". Then theoretically I have created a piece of buggy code in the eyes of some people. And this is where we get into trouble: not all bugs affect people the same way, nor are all bugs considered a bug always.

A software "bug" is "an error, flaw, mistake, failure, fault or “undocumented feature” in a computer program that prevents it from behaving as intended" to quote Wikipedia. It is technically a "glitch". It is the same as your brain confusing two memories, it is the same as you mistaking where you last put your car keys. The reality is, how can you possibly trust someone to write perfect non-faulty code when most of us forget where we put our car keys from time to time. Software bugs are typically human caused because we ourselves are imperfect.

But naturally, some bugs are worse than others, and due to my passionate and genocidal hatred: I will call such bugs "cockroaches".

What we as consumers and software developers need to do is to evaluate and test software with as much personal criteria as possible. It is impossible in this day and age to produce software which is 100% bug free, or even to utter the phrase "zero known fatal bugs". Because the truth is that that attitude is naive at best and dangerous at worst.

To say the phrase "zero known fatal bugs" is to give the false impression to anyone that you have produced "perfect" software when the reality is that given a secondary "ant-like" glitch in a related but separate system combined with a benign "bumblebee-like" glitch in yours may spawn a giant malevolent cockroach in a hospital system which causes death. If I ever heard a piece of software or a software company claim "zero known fatal bugs" I would call them irresponsible because they are falling to marketing and misleading hype and potentially their own hubris.

Bugs are here to stay until we create a computer which logically thinks in a manner that can create software which is bug free, at which point I am afraid that the human species might as well be doomed because truly we would be useless.

What you as the consumer need to do with \*any\* piece of software is to test, read and understand what it is you are about to run when you have control over it. I'm sorry to say I don't generally have control over my operating systems and that annoys the crap out of me. However, I do have control over what MySQL version I run, what PHP version I code with, what version of my IDE to use. When I update software, I read the known bugs, I test my application and I see if one of the documented bugs does indeed catch me out. Or worse, whether one of the undocumented, heretofore unknown cockroaches floats through my application space.

Frankly, do not live in the ignorant and naive world of a piece of software with no bugs, do not live in the world where you decide not use a piece of software simply because it might have a bug (which may or may not affect you). If you did that then I suggest, next time you "forget" your car keys to rip out your brain (uninstall) or go in for some neurosurgery (patch it). By refusing to try a piece of software because you \*think\* it might not work before even trying, you may in fact lose out an amazing chance to make your life, your five minutes, the world or the universe a better place.

In defence of the bug? The bug shows our humanity. When we all think alike, look alike, share the same language, then I would think we would lose all bugs. A truly boring place if you ask me. Like the story of Spencer Silver, the inventor who by accident (ie. a "bug" or "glitch") discovered the glue used for post-it notes, bugs are important and here to stay.

Wednesday Dec 10, 2008

Kicking the Ubuntu/Dell Wireless

Seems like with each update to Ubuntu, wireless seems to get better. This is just a quick note for anyone using Ubuntu 8.10 (Intrepid Ibex) on a Dell XPS 1330 with a recent Dell Wireless. I've been having problems with the wireless dropping in and out (every 100 packets or so the wireless connection would reset, what a pain). To solve it (and don't ask me how this specifically works), I upgraded to the proposed new kernel (.10) and the key was to remove the entry from the network manager and create a new connection including wiping out my past WPA password.

Works peachy now! Hope this helps someone out.

Tuesday Dec 09, 2008

MySQL Buzz

One of the many tasks ahead of me is to help improve the many sites that the Community Team manages for the MySQL Community web presence. I won't get into too much here, but we're looking at a whole host of different things to help make our MySQL community more functional, more interactive and more relevant to all users.

Now this may not seem like much but we've added a "MySQL Buzz" page to PlanetMySQL. It's a small little experiment, bringing some data that we currently don't have anywhere together onto one page. It features some data from our Forums, an addition of Google's Search tool with quick links to certain terms across Google's Blogs, News, Video and Web searches and some fun word frequency analysis.

This is by no means an end, just a quick little experiment, your thoughts are welcome and appreciated.

Hope you enjoy the little addition.

Friday Dec 05, 2008

Wanted: Web Developer @ MySQL

So I guess I have no choice now but to stick with Giuseppe and the Community Team as it appears my former job (or something similar to it) has now been posted on the Sun jobs site. So here is my encouragement for someone to apply for this position.

Now here's also the inside scoop. The MySQL Web Developer is a position which combines pure PHP Development, some DBA work and some Sysadmin work so if that appeals to you then great. You'd be working on improving the external sites as it relates to Demand Generation which is mainly and our partner sites. Believe me when I say there is a lot of work and you get to really customize that work to make it fun for you.

In addition, I can say that I've had a lot of fun working with the team. The #web IRC channel is always busy and usually quite fun with subjects ranging from strange storms as is happening as I blog this ("niall: the gods are angry with you dups") or help with whatever you might be working on. There's also the fact that the sites with its millions of visits/day leads to some interesting collaborations with our developer and engineering teams. Check out Adam Donnison on running MySQL 5.1 on production. There's great collaboration between Developer, Community, Support and Web which makes it a great environment to be in.

So who's in the team? Kristina Hadges is the Project Manager and Manager of the team, Markus Popp, a developer in Austria, Aaron White, a developer in Colorado, Niall Brown, a developer in my home town of Montreal and lastly, but certainly not least: Adam Donnison, the person with whom this position would collaborate heavily with and who hails from somewhere near Melbourne in Australia. Watch out he brews a mean cider, which he kindly smuggled in for us at our last company meeting in Florida. He also likes being woken up in the wee hours of the morning (sshhh don't tell him I told you).

Oh yes, and sadly you'll be working with me and the others on the Community Team as well so I have a vested interest in this position being filled with someone with a great attitude, humour and experience.

So if you dream about PHP, have an interest in working for MySQL and seeing/changing things from the inside, like working from home (though if there is a Sun office you can work from there too), are independent, have no problems enjoying a few laughs on IRC and want to make a difference as we work our way through the pitfalls of all that is MySQL and open source databases, then I strongly urge you to apply.

Thursday Dec 04, 2008

Piping blogs

One of the bad things about being online 80% of your waking life is that you end up creating a lot of content all over the place. This means that I have a blog that's personal that my family and friends occasionally check up on but is more a place for me to record my thoughts so that when I get senile in another 5 years I can actually remember what I did. I also have this blog on Sun. Oh yes, and I have a photo blog on my own personal site. Add to that possible twitter feeds, flikr feeds etc. well you get the picture.

For Facebook, where most of my friends keep track of me, you can only import one feed, so which do I choose? Considering I have work and non-work colleagues I figured I should somehow combine the three main "Dups" feeds into one aggregated feed. 

Before you start putting keystroke to an IDE or VI, you can simply use Yahoo Pipes. Pipes allows you to take different feeds and mash it all up. Now this isn't new, it's been around for a year, but I just didn't see the need for it till this past week.

So anyway, you can now link directly to my Piped news feed for all Dups, all the time...

Wednesday Nov 26, 2008

Happy Thanksgiving v. 5.1

If you are in the US and eating your turkey and giving thanks, you can look forward to also giving thanks that version 5.1 of MySQL is now marked Generally Available for production use (GA). Yes after quite the number of releases, a plethora of bugs fixed, it is now there for use by all.

Not that that's stopped us from using it live on our production systems here at, check out this interview with Adam Donnison of's webteam where he details a bit of running 5.1 over the last year. 

Congratulations to all the devs, it's been a long road, time to enjoy the fruits of their labour, along with a bit of cranberry sauce and some stuffing.


This is the blog of Dups... currently I'm one of MySQL's Community Relations Managers for Sun Microsystems, post, contact me, I want to hear from you!


« July 2016