.o Files: Just Say No
By mkupfer on Jun 14, 2006
As I mentioned in a previous entry,
the ON sources that are available via
opensolaris.org are a subset of the ON consolidation in the
Solaris product (90% as of build 42, measured in lines of
text). We've organized the ON source so that if part of a
component (library, command, kernel module) is closed, the
entire component is treated as closed. Closed components are in
their own subtree (
usr/closed), which parallels the
open source tree (
Since the OpenSolaris launch, I've gotten a few requests to support partial-source components. This is where the component is mostly open source, but it contains some code that can't be delivered as source for one reason or another. The usual proposal is to modify the OpenSolaris build to support a mix of .o and source files, similar to the way Sun and other vendors delivered their Unix kernels in the 1980s.
The appeal of supporting closed .o files is that once the
infrastructure is in place, the open source for these components
can be easily exposed outside Sun. And there's some precedent
for this practice, such as the closed-source files that were
split out from libc to
libc_i18n.a, and the uuencoded .o files
in the ath driver. But there are problems with this approach,
and those problems are sufficiently large that we have no plans
to implement it.
The most obvious problem with this approach is that it complicates the OpenSolaris build infrastructure. Mechanisms have to be implemented to identify which .o files need to be included with the closed binaries, and to restore them after a "make clean" (but only for external developers, mind you). By itself, this extra complexity wouldn't be sufficient to kill the .o approach, but it means we don't want to spend energy on it unless it's an approach we want to keep.
A second problem is that this approach blurs the separation
between open and closed source. If we no longer have a system
where everything in
usr/src is delivered and nothing in
usr/closed is delivered, we raise the risk that someone will
accidentally copy closed code into an open file and forget to
tag the open file as closed.
A third problem with this approach is that it makes it harder to
work in the open source tree. Imagine you're working on a
component that contains
you want to change a function
myfunc() that's defined in
You can use nm(1) to determine that
without a copy of
baz.c, you can't tell if your changes will
break the code in
baz.c or not.
Worse, suppose you want to change a struct that's defined in
common.h. Maybe it's used for
baz.o, maybe it isn't; there's no
good way to tell. And if it
is used, all sorts of
nastiness, including random memory corruption, is possible.
A fourth, more strategic, problem with this approach is that by reducing the barriers to having closed source in the Solaris product, it reduces the incentive to eliminate (or open up) the closed source.
A fifth and final problem with this approach is that it assumes a delivery model where the master workspace is internal to Sun, and the external workspace is just a mirror that is produced by filtering the main workspace. That's not the model we're after in the long term. Rather, we eventually want the external workspace to be the master.
So what about the precedents that I mentioned earlier? Let's
look at libc and
libc_i18n.a first. The libc
library is such an important component that keeping it closed
would have doomed OpenSolaris from the start. So we first tried
creating a dynamic library (
libc_i18n.so), where we could use
spec files to enumerate and enforce the interface dependencies
between libc_i18n and libc. But there was a
performance hit that we couldn't figure out how to work around.
So we settled for moving the code into a separate tree and doing
some analysis to show that the libc_i18n code is fairly
self-contained. That is, changes to private interfaces inside
libc are unlikely to break the code in libc_i18n.
But this isn't really a satisfactory approach. We adopted it
with a great deal of reluctance, and only because of
The other precedent that I mentioned was the ath driver, which has some uuencoded .o files in the source tree. This approach reflects the regulatory requirements for wireless devices in (at least) the USA. Government certification is required to deploy these devices, and given the flexibility of the Atheros chip set, the Hardware Abstraction Layer (HAL) software is part of what gets certified. Change the HAL binaries, and you invalidate the certification. So even in Sun's internal source tree, the HAL files are kept as uuencoded binaries, not as source. This means that many of the issues with a general .o mechanism don't apply here. There is no special-case makefile magic for "make clean", and external developers get exactly what internal developers get.
In summary, the .o approach requires non-trivial work, it has legal risks, and it treats the external community as second-class citizens. For these reasons, we will not pursue it.
 We could set up the makefiles in
such a way that you could tell if
baz.o depends on something in
common.h, but you wouldn't know exactly what.