Software is Hard Sometimes …
By Calvin Sun on Mar 31, 2009
Note: this article was originally published on http://blogs.innodb.com on March 31, 2009 by Vasil Dimov.
Some months ago, Google released a patch for InnoDB that boosts performance on multi-core servers. We decided to incorporate the change into the InnoDB Plugin to make everybody happy: users of InnoDB don’t have to apply the patch, and Google no longer has to maintain the patch for new versions of InnoDB. And it makes us at Innobase happy because it improves our product (as you can in this post about InnoDB Plugin release 1.0.3).
However, there are always technical and business issues to address. Given the low-level changes in the patch, was it technically sound? Was the patch stable and as rock solid as is the rest of InnoDB? Although it was written for the built-in InnoDB in MySQL 5.0.37, we needed to adapt it to the InnoDB Plugin. Could we make the patch portable to many platforms? Could we incorporate the patch without legal risk (so it could be licensed under both the GPL and commercial terms)?
Fortunately Google generously donated the patch to us under the BSD license, so there was no concern about intellectual property (so long as we properly acknowledged the contribution, which we do in an Appendix to the manual and in the source code). So, while the folks at Google are known for writing excellent code, we had to thoroughly review and test the patch before it could be incorporated in a release of InnoDB.
The patch improves performance by replacing InnoDB mutex and rw-mutex with atomic memory instructions. The first issue that arose was that the patch assigned the integer value -1 to a pthread_t variable, to refer to a neutral/non-existent thread identifier. This approach worked for Google because they use InnoDB solely on Linux. As it happens, pthread_t is defined as an integer on Linux.
But we had problems when the patch was tested on FreeBSD. We still
needed to reference a non-existent thread, but in some environments
(e.g. some versions of HPUX, Mac OS X) pthread_t is defined as a
structure, not an integer. As we looked at it, the problem became more complex. For this scheme to work, it must be possible to change thread identifiers atomically, using a Compare-And-Swap instruction. Otherwise, there will be subtle, mysterious and nasty failures.
We thought about enabling this patch only on Linux. But that approach was not optimal, because the patch could work perfectly well on other platforms where pthread_t is the same size as the machine word, like FreeBSD and Solaris, for example. We could have simply enumerated the operating system names where the patch would be supported, but this too was far from perfect. The ability to use the atomic instructions depends not only on the operating system, but the GCC version (has to be 4.1 or newer) and the capabilities of the CPU (some CPUs do not support Compare-And-Swap!).
So, we developed a dynamic compile-time check for checking whether the target environment (pthread_t type and size, GCC version, CPU capabilities) supports the use of atomic instructions as intended. This seemed to be the best approach, but then another problem arose!
Dynamic compile-time checks (i.e. autoconf) are part of the MySQL’s
./configure script. Innobase does not control this script, and so we
must live with what is distributed by MySQL. To make things simple for
users, we wanted to avoid asking the users to re-create ./configure with the autotools. So we simply injected the required checks in the Makefile. This seemed to work fine. Only after release of the InnoDB Plugin 1.0.3 did a small crack in this approach arise. Fortunately this problem turned out to be easy to fix and only occurs if one compiles with “make -j” (in order to perform the make in parallel). See Bug#43740 for details and for a fix.
The bottom line is that sometimes more goes on “behind the scenes” than you might expect when it comes to incorporating a third-party contribution into a product. We are grateful for the Google patch, and are glad that we have been able to include it in the InnoDB Plugin in a way that maximizes its portability, while keeping things as simple as possible for users.