Tuesday Aug 18, 2009

MySQL Labs : server snapshots available for download

Users familiar with the MySQL development process will remember that our developers use a tool called pushbuild, which builds the server code with the latest changes, using several operating systems, and runs the test suite.

MySQL Labs

This tool produces one binary package for each platform where the test runs, and every day there are a few dozen of such packages, waiting to be deleted and replaced by the ones created with the next build.

For long time, several people suggested publishing these binaries for the community. Each time, there was some minor or major impediment, such as getting together different teams and requesting resources from a third one. But the community team persisted and kept banging at the door until, at last, we can announce that the binaries are available for download at labs.mysql.com.

These binaries are temporary. You will find online for a few days, and they will eventually be replaced by newer ones. The exact dates of updates are not known. We build every day, but not all the builds are successful. Builds that fail to compile don't produce any binary, and builds that fail the test suite are not exported to the snapshot pages either.

But the binaries will be available fairly often, allowing the community to do early testing on bug fixes or new features.

A word of caution. These binaries have passed a test suite, but they have not been tested extensively like the monthly releases. Therefore, these binaries are for testing only, not for production. To make things clear, the package names include the word "snapshot" and the build date. And the files are not in the same page as the official releases.

Currently (2009-08-18) there are two branches: the latest 5.1 builds and the 5.1 GIS. For each branch, we plan to keep online at least the latest two builds before purging the oldest ones, but we may increase this number if we see that storage is not a problem.

Further releases will be introduced next month.

Notice that, unlike the normal releases, only a few platforms are included, and there are no dedicated packages (.dmg, .pkg, .deb, .rpm) but only tarballs (tar.gz, or zip for Windows). These packages are supposed to be used for testing only. In every Unix system, you can install them using MySQL Sandbox or, if it suits you, a manual installation.

I would like to thank the web and build teams who have worked together to provide this feature, and especially Daniel Fischer and Markus Popp, who delivered the ultimate goods.

Your feedback is valuable. Please let us know what you did with the snapshots.

Friday Jul 04, 2008

Test stressing OpenSolaris with MySQL

Over the last months I have seen some impressive presentations about Open Solaris, and I wanted to give it a try.

The live CD provided with opensolaris 2008.05 is very easy to install, and so I set it up in a virtual machine.

The environment looks familiar for a seasoned Linux user, and thus I decided to use it as a test bed for my MySQL Sandbox, which includes a test suite that lets you run a complete test with little effort.

Well, little effort for you, maybe, but not for the operating system. The test puts a lot of stress on the operating system, as you can see from this picture.

Running a complete test for a single version (1 single server + 3 replicated, + 3 circular, + 3 multiple, + 1 custom group = 11 servers) is already demanding. But since in my Mac I dare run the test with 7 concurrent versions for a total of 77 instances of MySQL, and I have still memory to spare, I ran a test with 2 versions on 2 virtual machines, one with Ubuntu 8 and one with Open Solaris.

I am sorry to say that opensolaris did not fare well with this test. The Linux virtual box, with 512 MB of RAM, ran 22 MySQL instances without trouble. The opensolaris box, with 1 GB of RAM, ran out of memory after 18 instances.

Nothing to be afraid of, though. From this negative test I got some useful experience. I asked for help in the opensolaris forums, and I got quick and smart advice from an active and vibrant community (thanks folks!). I haven't solved the problem, but I got enough information to file a proper bug report. According to someone in the forum, that's not even a bug, but the expected way of opensolaris of dealing with memory allocation requests, which is allegedly safer than Linux.

I am not in a position to judge who's right, but perhaps someone in the audience is better qualified to provide an objective answer to this problem. For now, I will just lean back in my chair and wait for comments.

UPDATE It looks like it's more a Linux than a Solaris fault. If you set vm.overcommit_memory=2 in sysctl.conf, as suggested in the comments, Linux will fail even earlier than opensolaris.


Giuseppe Maxia


« July 2016