Saturday Mar 10, 2007

OGB Elections: Biography of a Candidate

As you may have seen, I am a candidate for the OpenSolaris Governing Board, and as the other candidates are doing, I'm writing a bio to give background on myself. I've previously posted a more personal summary of my background, so I'll focus here on topics I believe are more relevant to being an OGB candidate. If you have no interest in the OGB elections, or in long rambling non-linear self-promoting stories, you can just move ahead to the next entry in your RSS feed reader now.

First the basics that all the candidates are stating: As you may have guessed from the URL this is posted at, Sun Microsystems, Inc. has been my employer for the past 8 years, but I have never built my own kernel. (I have designed, implemented, and integrated my own DTrace provider though.)

While attending the University of California, Berkeley I became involved in the running of Berkeley's Open Computing Facility, a somewhat unusual student group who provided access to Unix workstations and servers in a time when that was hard to come by for a non-engineering student. When I first started volunteering there in 1991, it was the only place most students on campus could get e-mail and Usenet access. Later it became the first place most students could host web pages, and continued to evolve over time. It was run by a completely volunteer student organization, in which I served first as a member of the system administration staff (their Apollo Domain/OS workstations were the first Unix machines I got root on, and the first time I got to have the account name alanc instead of just a class account name like cs60a-ej). I later joined the OCF's Board of Directors, who set the policies for the machines, and served a semester each as Site Manager (head sysadmin) and General Manager (head of the organization).

It was also at the OCF that I first started contributing to Open Source projects. My first major contributions were to the Gopher client/server software for Unix platforms from the University of Michigan, an early precursor to the world wide web. Of course, my most notable open source work has been through my job at Sun, contributing to the X Window System software that forms the foundation of all major Unix and Linux GUIs today. I participated in the old closed-membership industry consortium X.Org Group on behalf of Sun, and contributed to their irregular public code drops of the very closed development tree, and also contributed some of our code to the XFree86 Project, including the support I wrote for IPv6 in X11. When we were looking for ways to revitalize the stagnant X.Org Group at the same time that a group of current and former XFree86 developers were looking to create a more open development model to replace XFree86, I got to be one of the developers in at the very beginning of the transformation of both groups into the X.Org Foundation and it's Xorg software releases. I was elected to the short-lived X.Org Architecture Board, and was the release manager for the X11R6.9 release and one of the leads of the modularization project that resulted in X11R7.0. Through this I've experienced first hand how much more vibrant and productive a fully open development community can be than one in which a small core team holds the leash to all access, and all changes have to be funneled through them. (OpenSolaris isn't quite as bad, as there are several dozen “request sponsors” who can check in code from community members, but we still see requests that take much longer than they should because sponsors simply get too busy and become an unnecessary bottleneck. We've got several external contributors who have clearly proven they deserve commit access, but we haven't gotten the master repositories for anything besides JDS outside the Sun firewall yet so they can exercise that.)

When I took some time off from college, the Unix administration and end-user communication skills I got at the OCF, combined with the connections I had from that community, got me a position as a contractor in Sun's technical support center (then in Mountain View, California) answering Solaris system administration questions. Since I was more familiar then with other forms of Unix, and already had been addicted to Usenet for years, I looked for help in learning the Solaris specific methods to the comp.unix.solaris Usenet newsgroup, and returned the help I got by answering the questions that I could. (And even though I haven't posted as much lately, I just noticed Google still shows me in the top ten all-time posters to comp.unix.solaris, under my address.) It was through comp.unix.solaris that I first participated in the Solaris community, and first came in contact with current OGB members Rich Teer and Casper Dik, and of course, ksh93 for OpenSolaris project leader Roland Mainz.

After joining Sun I also started participating in the Solaris x86 YahooGroups mailing list, sticking with it even during the dark days in which Sun had declared Solaris 9 for x86 “indefinitely delayed.” After the “Secret Six” from that group convinced Sun to restore Solaris x86, one of the projects Sun started to restore relationship with the community was monthly meetings with a “Cabal” of community leaders. When we decided to ship the Xorg server in Solaris, I was asked to come to one of these meetings to talk about our plans, and given my involvement in the community already, was invited to become a permanent part of the group. That group didn't last long, as it ended when the participants, including me, became some of the original members of the OpenSolaris Pilot Program, so I've been part of the OpenSolaris community since the very beginning. Since the public launch of OpenSolaris I've served as a leader of the OpenSolaris X Window System Community and the representative of the X Consolidation to the OpenSolaris Program Team. (The forum software says I've posted 1,190 times to OpenSolaris forums since the launch, but I don't know how many of those are duplicates from cross posts to multiple forums.)

At Sun, I've also served for about 6 years now on the Desktop C-Team, which oversees the various Desktop consolidations (X, GNOME, Mozilla, CDE, etc.) - approving integration of new projects, setting and enforcing various policies, and coordinating joint projects with other consolidations. For the past three years I've been part of the Layered Software Architecture Review Committee, first as an intern, and now as a full member. LSARC is the ARC which reviews software above the core platform - including the desktop environments, web browsers, compilers and developer tools, and various management tools. X itself is usually straddling the boundary between LSARC and the core platform ARC (PSARC) - cases interfacing with the hardware or kernel go to PSARC (including most updates to the X servers themselves), while client libraries that are mostly used by the desktop environments go to LSARC, so that they fit into the larger picture of those desktops. As such, even though I'm part of LSARC, from time to time I submit cases to PSARC and participate in relevant PSARC reviews.

So that's the long boring story of how I got to where am I today (if you've made it this far, thanks for sticking through it, and be glad that I didn't make it even longer and more boring with recounting my experience as editor of the high school newspaper or how bad I was at memorizing my lines in high school Drama Club productions). Now that we've covered the past, I'll save the topic of the future and my positions on various OGB issues for another post...

Sunday Feb 04, 2007

Moving x86 video drivers into the kernel

Since Keith's recent talk on X at, I've been asked a few times and seen discussion elsewhere (such as in the comments on the linked article) what Sun thinks about moving various portions of the Xorg video drivers from the current userspace shared objects into true kernel drivers, as most people assume this means Linux kernel drivers and reducing the portability of Xorg to non-Linux platforms such as Solaris/OpenSolaris and the BSD's.

While I'm of course too far down the food chain to claim to speak officially for Sun, as the lead engineer for the Xorg integration to Solaris and someone who has talked to a number of the other Solaris engineers about it, I'd say the Solaris engineering teams agree having in-kernel drivers for video cards is something we want and need.

On Solaris, we've long had in kernel graphics drivers for all our SPARC frame buffers, since that was a small and controlled set, and writing the driver for it was simply part of the cost of producing the card. On the x86 side though, we had for years the same model as most other x86 Unix-like OS'es - a generic set of kernel modules, and all device-specific logic in the X server. The Solaris x86 kernel modules were vgatext and xsvc. vgatext provides the text console, using standard/ancient VGA interfaces which virtually all PC graphics boards support. xsvc (which was closed source due to legal encumberances, but has recently been replaced with a newly written open source version by the Solaris Xen project team) provides the mappings to the PCI bus that allow Xorg to probe the bus hardware to find devices, and then map in the video card once it's found.

Recently, new kernel modules have started appearing in Solaris - the first two for 3-D rendering acceleration, nvidia's driver and the DRI driver for Intel graphics. Two more are coming soon from the Suspend-and-Resume project, to provide the hooks to save and restore necessary state across system suspends - they'll initially support the ATI Rage XL from the Sun Ultra 20 and ATI ES-1000 in the Ultra 20 M2. (Dave and Jay gave a talk on their implementation at last years' X Developer's Conference.)

Additionally, our security guys have been pushing for more kernel frame buffer drivers to reduce our need for running Xorg as root. They'd also like to find a way to close the now infamous Ring 0 security weakness. (We've invited them to drop in to this week's X.Org Developer's Conference to talk about this - they're planning to show up Thursday morning for the Secure X talk, so we may try to have a breakout session on this topic at that time to discuss this.)

Now, we still can't write drivers ourselves for every x86 video card on the market, which is one of the big reasons we chose to move from Xsun to Xorg on Solaris, so we'd still prefer if a model could be found that allowed sharing drivers between Solaris, Linux, and BSD (which means agreeing on both licensing and interfaces with the kernel), similar to what we already have in DRI. If we can work out a model like that, then I think Sun would support it - either way, we're going to have to have more graphics driver functionality in the kernel, and sharing would be better than going our own way.

[Technorati Tags: , , , , , , ]

Tuesday Nov 28, 2006

My favorite new feature in Nevada build 53...

I wrote a couple of weeks ago about the then-upcoming Solaris Nevada build 53 in a mail to the Solaris x86 YahooGroups mailing list:

For desktop users, nv_53 should be awesome. Gnome 2.16, Firefox 2.0, Gnome System Tools, DRI for i845/i855/i865/i915, and, just integrated today, for the first time ever, the nvidia accelerated drivers/GL as part of the Solaris OS install. Attentive viewers may note that it also doesn't have kdmconfig run at install time anymore due to the replacement of Xsun with Xorg as the install-time X server.

Now that I've updated to nv_53 on my main desktop at work (which was actually a big jump from it's previous Solaris 10 6/06 install, but went smoothly via Live Upgrade) I've found my favorite new feature is one I didn't even know about in advance - a popup menu from the JDS toolbar clock that shows the time in different timezones:

Time Zone Popup Menu screen shot

I work with people from around the world - our desktop team is mostly split between Ireland and China, with outliers in New Zealand, Canada, and Illinois. The Architecture Review Committee I serve on at Sun has members on both coasts of the US, and in India and Isreal at the moment. Trying to keep track of what time it is where is more than I can remember most of the time, so I was constantly going to to check the time in other parts of the world. Now I can add them to my menu for quick reference.

Unfortunately, it was something I discovered by accident and seems to be a little hidden - to get there, right-click on the panel clock and choose Preferences. In the Clock Preferences panel turn on Show time zones button. You should now have a big world/clock icon next to your clock like the one shown above (one that seems too large and out of place for the panel compared to the other icons there). Click on it to bring up the menu and choose Edit Time Zones. Now, you'll have to ignore the actual question shown in the next dialog:

Even though it's asking you to choose your nearest city, it really means “a city in the time zone and jurisdiction you want to see” — it's neither distance to you that matters or distance to the target, but which set of time zone rules and daylight savings change rules are in effect in the target location. For instance, if you want to know the time in Seattle, Washington, you need to choose Los Angeles, which is much farther away than Boise, Idaho or Vancouver, BC, Canada, but unlike Boise, is in the same time zone, and unlike Vancouver, follows the same national time-shifting schedule. It would be nice if it let you edit the name shown, since I normally think of needing to know the time in Beijing, not Shanghai, but it's still much nicer than what I previously did.

[Technorati Tags: , , , ]

Thursday Nov 09, 2006

X Changes in Nevada Builds 50 - 52

I posted our OpenSolaris code drop Tuesday for Nevada Build 52. The last build published was build 49, so this release includes all the changes from builds 50, 51, and 52. Yet out of the 27 changes in those three builds, almost all the questions I get are about just one of them, which can be summarized in 3 little letters:


This build contains the first release of DRI support in Xorg on Solaris. It only supports 3-D hardware acceleration on the Intel i845 through i915 graphics chipsets for this first release, and requires the kernel DRI support included in the Solaris Nevada Build 51 and later kernels, but is a huge first step. Both the kernel and Xorg/Mesa sides are now published in source form on as well, so those who want to see how it was done can take a look, and the truly brave could start thinking about porting other drivers.

If you want pre-built binaries, you'll have to wait another week or two for the Build 52 ISO's to be published as Solaris Express: Community Release. More information on DRI in Solaris can be found in the DRI heads up message from the DRI team. You can browse the kernel side of the sources in the OpenSolaris source browser in these directories (sorry X source isn't on the source browser yet):

One word of warning though - the kernel driver may cause panics on some i855 machines - see bug 6487609 for details.

But wait! As if that wasn't enough, there's still 26 other changes included in this valuable package! And they're not available in stores! For the full list, see the changelog for builds 50-52. Highlights:

6465198 xdpyinfo shows incorrect extensions list on snv11_46
Dynamically loaded extensions like OpenGL have been failing to load in Xsun in the last few Nevada builds - this fixes that.
6423858 [Xorg bug 5898] file creation race condition in Xsession
A small security issue in the default Xsession if you use xdm instead of gdm2 or dtlogin. Patches for Solaris 8, 9, and 10 are coming for this fix.
6475968 Desegregation of X11 External Libraries [PSARC/2006/557]
When we added libXrender, libXdamage, libXfixes, and libXRes to Solaris, the Sun policy called for keeping libraries which we didn't guarantee ABI stability of in a separate path so users didn't “accidentally” depend on them and create applications not covered by the Solaris Binary Compatibility Guarantee. Sun policy has since changed to prefer usability and to not require users to jump through hoops to build software using them, so we've adapted to the new policy by moving these libraries from /usr/openwin/sfw/lib to /usr/X11/lib and putting links to them in /usr/lib.
6477006 keycode 22 reports keysym 0/NoSymbol on Sun Type 6 Unix keyboard
The Unix layouts of Sun's keyboards have a blank key between Help and F1, in the slot Esc uses on the PC layout. (See these diagrams for comparison.) This allows using the same hardware with just different labels on the keys. Unfortunately, this key has always just been assigned a to keycode 0/keysym "NoSymbol", which is pretty useless. Some intrepid users have manually assigned it to a keycode so that it can be assigned to a hotkey in JDS or CDE. To make it easier for others, the blank key now defaults to a keysym name of (wait for it...) “Blank.” Yes, that's right, X11 defines a keysym XK_Blank — and since it doesn't define XK_Any for those who long ago decided it was the mythical key referenced in Press Any Key to Continue, that seemed the best choice. Sadly, only the old serial keyboards send a keycode for this key – the USB ones send nothing up to the X server, so this doesn't help there.
6476476 Xorg modularization: libXfixes
6477401 Xorg modularization: SUNWxorg-client*
We've converted to the Xorg 7.x modular versions of the libraries we and applications we used to get from the X11R6.9 source tree, including libXrandr, libXv, libXvMC, libXxf86misc, libXxf86vm, libxkbfile, xgamma,, xrandr, xvidtune, xvinfo. We've also updated libXfixes, which was previously brought in from the X11R6.8 source to the modular release of libXfixes.

[Technorati Tags: , , , ]

Tuesday Nov 07, 2006

Xserver DTrace probes integrated

It's been a little over a year since I posted the initial open release of the DTrace probes for the X server. I've gotten busy with a number of other things since then but after a little prodding from the JDS team [1], I've finally run them by the Sun Architecture Review Committee (PSARC case id 2006/609), got their thumbs up, completed code review, and integrated into our Solaris builds of Xorg & Xsun (which due to code sharing also means they're in Xnest, Xvfb, and Xprt). If they pass QA testing, they should appear in Nevada Build 53. I've also put them into the X.Org community git master branch, so they should be in Xorg 7.3 when it ships next May. (The autoconf code for them only checks for the dtrace command needed to build, not for the OS, so when MacOS X & FreeBSD users get DTrace, they should be able to build the probes as well.)

I've also updated with the few changes made to the probes since initial release (added an event probe, and changed one of the arguments made available by the request-done probe) and to include a patch against the Xorg 7.2.RC1 tarball for the current state of the probes for those who don't want to wait.

[1] The conversation went something like this (slightly paraphrased):

  • me: These JavaScript DTrace Probes are cool! We should include them in the FireFox in Solaris!
  • jmr: Yes - and what about the XServer dtrace probes ;)
  • me: oh, um, well, yeah, okay...

[Technorati Tags: , , , ]

Friday Oct 06, 2006

XACE merged into Xorg for X11R7.2

The XACE framework for handling security policy extensions has been merged to the Xorg server code base for the upcoming X11R7.2 release. This is the rough equivalent of the hooks that were put into the core Solaris Xsun & Xorg servers to call out to the Xtsol extension module as necessary to implement the security policy.

XACE was originally designed by Eamon Walsh at the NSA for SELinux, and working with us this summer, modified to add the additional hooks needed by Xtsol, so it could serve as a common framework acceptable to both SELinux & Solaris Trusted Extensions. (For instance, in the original design, there were no hooks for auditing, as the SELinux code did not audit X requests as Xtsol does.)

The actual policy extension modules (X-SELINUX & Xtsol) were not ready to merge in time for the 7.2 release, so they are planned for the 7.3 release. (X.Org is currently doing full releases every 6 months, May & November, but individual modules can release at any time they are ready, so just because we missed 7.2 doesn't mean we have to wait until next May to integrate Xtsol to X.Org.)

For those who aren't familiar with the technology, a brief overview may be found in the slides [PDF format] from my talk on Security Extensions in X from this summer's Desktop Developer's Conference. A much more detailed look at the Solaris Trusted Extensions OS as a whole, including the X server and desktop, can be found in the slides from the Trusted Extensions talk by Glenn Faden at last week's Silicon Valley OpenSolaris User Group meeting.

[Technorati Tags: , , , , ]

Friday Aug 25, 2006

X Changes in Nevada Build 47

I've fallen behind lately on posting the OpenSolaris X code drops - but I'm back on schedule today with the posting of Nevada Build 47. The big change this build is 4869280: Update xscreensaver to 5.0 (from our previous version 4.05). We still modify it heavily, though the changes to use the SCF smartcard API directly have been removed, and we now rely on PAM for our smartcard support. Other changes we made doing this upgrade time were noted in the ARC review.

Other changes in X in this build:

6457364 SUNW0xman & SUNW0xpmn prototypes should be autogenerated from main
SUNW0\* packages are generated to send to Sun's localization teams for translation - they are templates for localized packages containing the English text and just the files to be translated. Instead of having to update both the base packages and the templates every time we add a man page now, a perl script generates the l10n templates from the base packages.
6454339 Xorg modularization: libXau 1.0.2 (missing FILTER entries in libX11)
In the old X.Org monolithic build, a couple files from libXau were symlinked into the libX11 source and built into libX11 - a holdover from the days before shared library dependencies worked everywhere. Since the systems all supported by X11R7 handle shared library dependencies correctly, this was replaced in X11R7 modular builds by just having libX11 depend on libXau.
In build 46, libXau was replaced with the X11R7 modular version. Instead of symlinking from our modular build tree into our monolithic tree, our monolith libX11 had the files removed and uses the dependency just as the modular libX11 does. Unfortunately, since libX11 was shipped years before we started using linker scoping to hide symbols like this, the function names in libX11 were long exposed, though versioned as SUNWprivate so that appcert would warn that applications could be broken by relying on them. Preserving binary compatibility was easy though, simply by adding FILTER function entries to the libX11 mapfile, like this:
SUNWprivate {
        XauDisposeAuth          = FUNCTION FILTER;
        XauFileName             = FUNCTION FILTER;
        XauGetBestAuthByAddr    = FUNCTION FILTER;
        XauReadAuth             = FUNCTION FILTER;
Now any function that tries to call those functions in libX11 will get automatically redirected to the correct location in libXau. However, this mapfile change got accidentally missed in the original putback to build 46, so went in this build instead. Because of this, we found that the gnome-panel in Solaris was actually using these functions from libX11 instead of linking to libXau directly as it should have, so while build 47 will restore compatibility for it, the GNOME team is fixing it to link correctly against libXau. (This is the root cause of bug 6461529: gnome-panel crashes when selecting Launch ONLY if logged in using gdm in Nevada build 46.)

The rest are pretty well summarized in their bug reports:

6459557 remote logins to xdm fail since fix for 6398796
6460081 Xorg modularization: libXdmcp 1.0.2
6237253 Xserver man page should include SMF examples
6459143 Need to ship pkgconfig files for modular protocol header packages

[Technorati Tags: , , , ]

Monday Aug 07, 2006

"the open source DTrace, now built into Mac OS X Leopard."


(which reminds me, I still have to put the Xserver dtrace probes into the X.Org git repository - though I guess they'll have to check for more than just $host_os eq "solaris" now in the autoconf script)

Update: Team DTrace speaks.

[Technorati Tags: , , , ]

Sunday Aug 06, 2006

X Changes in Nevada Builds 43-45

I've fallen behind in both posting the code drops for the OpenSolaris X sources and the summaries of the code changes. I'll try to get back on track with the upcoming Nevada build 46, but for now here's some quick highlights of what went into builds 43, 44, and 45. (For the full list of changes, see the OpenSolaris X Community ChangeLogs page.)

Build 43 — Source Drop 20060612

6398796 Solaris-10: Unable to login thru xdm once password is aged
xdm suffered from the same problem as many legacy programs who had simple password checking code replaced with PAM - instead of implementing a full PAM conversation, where PAM could prompt the user for multiple items, or none at all if using non-keyboard-input authentication methods like smartcards or thumbprint scanners, it just continued to always ask for a username and password and pass them to PAM via a hack that simulated a conversation. While this bug could have been fixed by slightly extending that hack, our experience in trying that with xscreensaver and consultation with Sun's PAM gurus convinced us the best way to solve this was a complete rewrite of xdm's PAM code to offer a full conversation, so we didn't end up with multiple layers of hacks that still didn't offer the full PAM functionality and kept needing to be rewhacked for every place it was found a little more of PAM was needed. This rewrite has also been integrated to the X.Org xdm module, where it's planned for inclusion in an upcoming xdm 1.1 release.
6424854 Decomposition of SUNWxwplt [PSARC/2006/302]
In preparation for the upcoming switch of the Solaris x86 install mini-root from Xsun to Xorg, and to the delight of people everywhere who minimize their Solaris systems, the single SUNWxwplt package which previously contained both the Xsun server and the core X client libraries and applications has been split into three packages. Now SUNWxwplt is the core client-side of X, the parts other packages like Java depend on for libX11 and friends, while Xsun is in a new SUNWxsun-server package. The Xsun keytables were split out to a third package to allow us to hand off responsibility for that package to the localization centers in Sun who have been maintaining them in our package for several years now. And finally, Xprt was split out to a fourth package, to allow us to more easily change it from the old Xsun-based implementation to an Xorg-based version in the future.
6437461 Xorg modularization: common extension protocols
A whole bunch of headers for X11 protocol extensions (see the bug for the full list) were removed from our old X11R6 monolithic build tree and replaced with the corresponding X11R7 proto packages. Users shouldn't see anything, but people building will find us another step closer to being able to build the rest of the X11R7 stack.
And a whole batch of bugs from Henry Zhao's work to improve Xorg auto-configuration - he's working to get these upstream to X.Org as well.
  • 6420892 ATI ES1000: resolution too low on Sun 24-inch LCD
  • 6420309 auto-config improve: Need to move VBE DDC fallback probing from server to drivers
  • 6420320 auto-config improve: nv – Need to consider panel size in mode validation
  • 6420311 auto-config improve: Ferrari 4000 starts with blank screen without xorg.conf
  • 6437062 auto-config: radeon – reboot needed for CRT to function when connected later on Ferrari 4000

Build 44

6436994 radeon: negative refresh rates preventing resolution selection in JDS
Also reported as X.Org Bugzilla #6966, the refresh rates reported by Xrandr when using the ATI driver in MergedFB mode were the combined rates of both screens, which made them appear to be negative and thus caused the GNOME "Change Display Resolution" tool to declare them invalid and not let you change the resolution of a MergedFB display.
And a whole pile of fixes from the Solaris Trusted Extensions team for the XTSol X extension...

Build 45

6261914 Removal of STSF & Xst [PSARC 2006/087]
Sun stopped funding the STSF project a couple of years ago, and switched fully to using Xft in the JDS/GNOME desktop for the Solaris 10 release. This code has been causing problems lately for the integration of other projects, such as Project Looking Glass, so we've announced in the Solaris 10 6/06 release notes plans to remove in the future, and removed it now from Solaris Nevada.
6444546 ia_find_display has small memory leak / fails to cache
The bug report pretty much explains it all - fortunately, in most applications, this is called once during XOpenDisplay(), which most applications only call once, so you'ld lose only a handful of bytes per application. (This code is part of the SolarisIA extension for giving a kernel scheduler priority boost to the process with focus - we've made the code available to X.Org, but it's not integrated into any Xorg releases, so no one else has this particular leak either.)
6450019 root cannot unlock screen
A recent fix to the code in our xscreensaver to allow you to unlock a normal user's session with the root password unfortunately did not allow you to unlock a root session with any password. Oops! (But users who understand security don't login to a desktop session as root anyway.)

[Technorati Tags: , , , ]

Thursday Jun 15, 2006

X Changes in Nevada Build 42

Build 42 sources were posted last week in Source Drop 20060530, so it's time once again for the brief summary of what's changed.

6314490 X app dumps core with LC_ALL != C when XtOpenDisplay() is called twice
The bug report provides a pretty complete description - the fix from our i18n team updated the way we close dlopen()'ed locale modules in XCloseDisplay(). Our libX11 locale module handling is quite a bit different than the X.Org versions - hopefully we'll have the sources to that released to OpenSolaris in a few months.
6416842 [CVE-2006-1526] buffer overflow in Render extension in Xsun
We already released Xorg patches for the recent Render security hole for Solaris. The same code is present in the Xsun sources in Solaris 10 and later as well, but currently disabled at build time, so we checked in the fix to make sure that if we ever re-enable the code in Xsun we don't reintroduce the security hole, but aren't going to release patches that fix code that can't be run.
6425531 integer overflows in FreeType
The recent release of FreeType 2.2.1 included a number of fixes for integer overflows, to prevent crashes or memory corruption when processing fonts with invalid sizes for data tables. Unfortunately, it also includes changes which break the builds of many existing programs including GNOME's Pango library and the Xorg X server, so we can't just upgrade to it until all the software that uses it is fixed. Thus, we've pulled out the integer overflow checks and backported just those to our current FreeType 2.1.10 as a temporary fix.
And a huge pile of keytable and XKB layout data fixes from our localization teams:
  • 6310310 Belgian keytable file "Belgian5.kt" is not present in keytables directory
  • 6325002 Norwegian "no" keyboard layout contains some wrong symbols.
  • 6339418 Can't switch to Finnish keyboard layout using Xsun.
  • 6353678 Hungarian keyboard layout for TYPE6 keyboards is missing for Xsun.
  • 6370065 New keyboard layouts does not work with XKB extension on x86 + Xsun.
  • 6370108 Bulgaria6.kt and Russia6.kt files contain no cyrillic symbols.
  • 6370138 Some keyboard layouts in nevada don't work in Xorg with SunTYPE6 keyboards
  • 6370147 Keyboard software for Macedonia needed.
  • 6370441 Cannot login in login window(dtlogin) when keyboard layout is changed from US English to Hebrew
  • 6384899 Slovenian keyboard layout for x86+Xsun and Sparc contains some errors.
  • 6384921 In Icelandic keyboard layout for x86+Xsun and Sparc are missing some symbols.
  • 6386202 Russian Xorg symbols file needs updated as it includes some incorrect key mappings
  • 6386205 Bulgarian Xorg symbols file needs updated as it includes some incorrect key mappings
  • 6389541 Croatia keyboard layout does not work in Xorg.
  • 6421192 Cyrillic based country layouts should toggle latin/cyrillic using Altgr key
  • 6426647 fr_CH and de_CH keyboard layouts have several wrong mappings on x86 (Xorg).
  • 6426648 fr_CH and de_CH keyboard layouts should be available directly form xorgconfig

[Technorati Tags: , , , ]

Wednesday Jun 14, 2006

One Year of OpenSolaris and X

OpenSolaris 1 Year Anniversary

One year ago today, I wrote about how X fit into the OpenSolaris plans and why the portions released then were just the first of the Solaris consolidations to be released. Since then, OpenSolaris has expanded from just the core OS/Networking (ON) consolidation to include the X Window System, Java Desktop System, Installation & Packaging, Storage, and various other consolidations.

It took a little longer than originally planned to start getting our X sources out (the original schedule I posted in September assumed X.Org would be sticking to their scheduled release of 6.9 in October, but that slipped to nearly the end of December), but they were released at the end of March.

As of the latest posted biweekly code drop, we've released the build infrastructure, source changes, and packaging to the Xorg server; the Mesa, Xft2, freetype, and fontconfig libraries; our forked version of xscreensaver; and the header files in the xproto package at the root of the Xorg modular dependency tree. Work on converting our tree to use the Xorg modular sources continues, and every time we convert a package, that's one more bit of the tree for which our sources become published.

Additionally we've published the sources to the Xtsol extension which provides multi-level security in X, and came to us from the Trusted Solaris product, which is being reshaped into the Trusted Extensions add-on package for Solaris. One of my big current projects is reconciling this with the X-ACE & SELinux X work which was done in a branch of the old X.Org monolith tree and never merged in, so that we can get changes merged into the main X.Org source needed to support both projects.

Going forward, while we know we'll probably never match the X.Org release 100% and always have some customisations and fixes we apply to it, our goal is really to concentrate development in a way that most changes flow back into the main X.Org tree, and that our OpenSolaris releases become as much as possible just a thin layer of building and packaging around those. (That's not just my wishful thinking either, but the instructions we've been given by my boss's boss.)

So where are we going with X in Solaris? As you could probably guess from the presentations our team gave at this year's X Developer's Conference two areas currently getting a lot of our attention are improving the autoconfiguration of Xorg (we still ship Solaris with no xorg.conf by default, relying on autoconfiguration, and want to make that work well enough that xorg.conf files become a rarely used item for exception cases) and in getting power management, especially suspend-and-resume, working reliably on Solaris. Changes to Xorg to improve autoconfiguration have started appearing in recent Solaris Nevada builds and the OpenSolaris code drops, and there's a bunch more in testing and development now. Many of these have also been fed back to the X.Org bugzilla as well, where we hope they'll go in the main drivers and benefit all platforms.

Xorg is definitely our future, as we're concentrating new development there and Xsun is moving to more of a maintenance state. DRI for Solaris Xorg is coming from our compatriots in the Solaris driver team in Beijing, and we're hoping to see some nice performance boosts for the ATI and Intel graphics chips there, though they'll still probably lag a bit behind what NVIDIA's accelerated driver can do for their hardware on Solaris. I wish I could give timeframes for when Solaris will get DRI or Xorg support for SPARC graphics or Sun Ray, but unfortunately, my crystal ball doesn't work that well (as my far off the mark dates for our OpenSolaris X releases proved).

So there you have it - the past, present, and future of the X Window System in OpenSolaris one year into this wild ride. Who knows where we'll be after year two...

[Technorati Tags: , , , ]

Tuesday Jun 13, 2006

Changing the default login session in dtlogin

At some point over the past few years, I somehow went from just checking in the new login and splash images for CDE in each Solaris release to becoming the unofficial dtlogin “special ops” person - handling the overhauls of the dtlogin appearance for the Solaris 10 beta, 3/05, 1/06, and now 6/06 releases and a few other side tasks that the main CDE sustaining team didn't have the resources to handle. The latest of these is shipping in Solaris Nevada starting in build 39, and I've just gotten the draft of the release note for the upcoming Solaris Express release including it:

Default Desktop Session in dtlogin

Now, when a user logs into the Solaris Desktop for the first time, Java Desktop System (JDS) is the default desktop environment instead of the Common Desktop Environment (CDE). JDS has also become the default environment for users who chose a desktop environment on an older Solaris release that is no longer present in the Solaris release, such as OpenWindows or GNOME 2.0.

System administrators can modify the dtlogin configuration to override the default choices using the defaultDt and fallbackDt resources.

For more information about defualtDt and fallbackDt resources, see the dtlogin(1M)man page.

So users who already chose CDE or JDS will have those choices respected - this only changes the defaults for new users or those whose current session can't be found.

For comparison, the description I wrote for our Architecture Review Committee (ARC) has a lot more technical detail (some of which is now also captured in the above mentioned dtlogin(1M) man page):

This case introduces two new resources in the dtlogin Xresources file
(/{usr,etc}/dt/config/$LANG/Xresources) and establishes initial default
values for those in the Xresources files shipped by Sun.

It requests a patch release binding.

1) Dtlogin\*defaultDt:

   When a new user logs in for the first time, a dialog box is presented
   asking which of the desktop environments installed on the machine they
   wish to use.   The default list currently consists of CDE & JDS, but
   any additional desktops installed using the altDt support in dtlogin
   will appear as well.

   This resource controls which of those desktops is selected by default
   when the dialog appears, so a new user who doesn't know the difference
   and just clicks the OK button will get this desktop, but those who know
   the difference can still choose as appropriate.

   The value of this resource must be the altDtStart value of one of the
   desktop environments installed on the system.   (This was chosen since
   it is the one value required for all desktop environments that must be
   most stable, since it is the one recorded in the lastsession file in
   the users home directory to store their chosen desktop.   The altDtName 
   is localized, so not appropriate for a global default setting.  The
   other values for altDt\* settings are all optional and may not be present
   for certain desktops.)

2) Dtlogin\*fallbackDt:

   When an existing user logs in, but the lastsession file in their home
   directory refers to a program that is not executable on the current 
   system, dtlogin will execute the program listed here as a fallback session.

   This may be any program, including /usr/dt/bin/sdt_firstlogin to offer
   the users a choice of the sessions available on the system.

The default X resources files shipped in Solaris will contain these additions:

!!  Default desktop choice for new users in initial login dialog
!!  This should be the altDtStart key of a session defined here or in
!!  an Xresources.d file.

Dtlogin\*defaultDt:	/usr/dt/config/Xsession.jds

!!  Fallback desktop choice for users whose lastsession file refers to
!!  a non-existent session choice.  Set to /usr/dt/bin/sdt_firstlogin to 
!!  offer the users a choice of the sessions available on the system.

Dtlogin\*fallbackDt:	/usr/dt/config/Xsession.jds


Imported interfaces:

dtlogin lastsession mechanism		Project Private	ASARC/1995/390
/usr/dt/bin/sdt_firstlogin		Project Private	ASARC/1995/390

Exported interfaces:

Dtlogin\*defaultDt resource		Stable
Dtlogin\*fallbackDt resource		Stable

Default settings of new resources	Volatile

By declaring these Stable, we're promising that these resources will take values as described and do what we say until further notice. It doesn't prevent us from further changing or refining this in the future (though since we're really trying to replace dtlogin with gdm in the future, we're not likely to do much), it just means that if we do, we'll have to either change these in a compatible way or create new resources. By declaring the defaults as Volatile though, we're warning that we could change them at any time, without warning - for now they're the Xsession file currently used by JDS, but we could change to another mechanism of starting JDS or even (though highly unlikely) to another desktop altogether, without having to worry about having made any compatibility guarantee there.

The note about a “Patch release binding” indicates that we think these changes are compatible enough with current systems to allow shipping this change in a patch or update release (and in fact, this change is being backported to a Solaris 10 patch to be included in the third Solaris 10 update release, tentatively scheduled for release near the end of this year). These changes don't break any existing systems, don't change anything for existing users with a valid desktop choice, still give new users a choice of desktops, and can even be easily overridden by sysadmins who still really prefer their new users be suggested CDE instead of the newer, easier-to-use GNOME-based JDS desktop.

A lot of people think that binary compatibility guarantees that Solaris provides stifle innovation because we can never change things - this is an example of why that's not the case. We don't guarantee nothing about the system will ever change - that would be foolish - what we do is specify what things you can count on remaining compatible (not unchanged, just compatible), and what you can't - and how long that compatibility is promised for. We've never made any promises in the past about what choice would be shown as the default in the session choice dialog, and the Volatile stability for this reflects that we still aren't - but because that's not someting that affects the managability of Solaris systems or the ability of ISV's to provide applications, it's not something we need to make those sort of promises about. The new resources are declared Stable because we don't want admins to use them and then find when they upgrade to a future release that we've broken their settings or made it so their users can't login until they find and fix the problem.

Right now these promises are mostly expressed via the man pages in Solaris - especially the Attributes section in many man pages. The ARC team is also working to open up the processes through which these are set and the history of many of the past review cases that set important ones, via the OpenSolaris ARC community, so that developers will have greater insight into these compatibility promises and eventually even have a voice in shaping them.

[Technorati Tags: , , , ]

Friday Jun 09, 2006

X Changes in Nevada Build 41

I realized today when preparing the build 42 source drop that I had not yet posted the bug list from build 41 here - though it's been up on the ChangeLog page for two weeks, and the source was posted as Source Drop 20060516 at the same time.

6424349 prepare xscreensaver sources for OpenSolaris release
Sources for our highly-modified fork of xscreensaver 4.05 are now included in the main tarball. As an added bonus, the build 41 source drop included a separate tarball featuring a port of those changes to the recently released XScreenSaver 5.0 which is being tested for integration into a future build of Solaris. As you can see, we've made many changes, including the GTK+ unlock dialog box, a major overhaul of the PAM code to allow more complex PAM conversations than just “Type in your password,” support for accessibility helper programs, and a whole bunch of other things.
6425513 xproto 7.0.4 -> 7.0.5
This update simply pulled one of our previous Solaris patches directly into the upstream source since we contributed it back to X.Org after doing the 7.0.4 integration.
6425506 /usr/openwin/demo/maze segfaults when $DISPLAY not set
While this hasn't been released in our source drops yet, you can probably find lots of examples of this bug in other open source apps, especially those written on OS'es where passing NULL to printf routines doesn't cause core dumps as it does on Solaris (including the really old versions of SunOS this program was written for). The fix is trivial, since Xlib provides XDisplayName() for this very case:
   if ((dpy = XOpenDisplay(display)) == NULL)	{
-    fprintf(stderr, "Can\\'t open display: %s\\n",
-	    (display ? display : getenv("DISPLAY")));
+    fprintf(stderr, "Can\\'t open display: %s\\n", XDisplayName(display));

6424870 add symlinks in /usr/lib to, etc
Most of the X libraries have symlinks in /usr/lib so they can be easier found by ld and dlopen(), but we hadn't added these yet for the libraries added to support Xorg extensions like Xrandr. Java wanted to dlopen these, so we added the links to avoid having to make Java do LD_LIBRARY_PATH or other workarounds.
6397125 Radeon driver: fails to read hsync/vsync rates from EDID
6423278 auto-config improve: radeon – Sometimes does not sort modes correctly
Two more fixes from our project to improve the configurations the Xorg server chooses by default when you don't have a configuration file or generate one via Xorg -configure. Henry Zhao, the engineer working on these fixes, has also been submitting them to X.Org so they may appear upstream in the future as well.

[Technorati Tags: , , , ]

Tuesday May 16, 2006

X Changes in Nevada Build 40

For build 40, I got the source drop posted already, so if you want to follow along in the sources, I'll point out which files you can look in to see these changes, for those in the portions included in our open source releases so far.

6414453 ogl-select fails to start
A simple shell script quoting error - while not in the source drops, since this is a shell script, you can see the changes in /lib/svc/method/ogl-select in the installed files.
6388471 [Xorg bug 5897] xdm: race condition on $HOME/.xsession-errors being readable
The fix from the X.Org bug report has been applied to our Xsession script in /usr/openwin/lib/X11/xdm/Xsession.
6421217 xscreensaver cannot be launched on trusted jds
The Trusted Extensions changes made in the previous build forgot to initialize a variable in one code path - you'll be able to see the fixed code in the next build, when the Solaris xscreensaver sources are released for the first time.
6366490 Motif pull-down menus don't draw correctly with Xnest
Unfortunately, our Xnest is still based on the Xsun source, so this source is also not available [maybe trying to point to source wasn't a good idea - oh well, I'll plow on since I don't want to start over now]. The fix here was to correctly pass through the expose events from the underlying X server to the X client running on top of Xnest.
6386535 SUNWxorg-mesa package can fail to install symlink for /usr/include/GL
Aha! One I can point to the source for in our source drop! This was caused by the preremove script for SUNWxorg-mesa not being included in the package - the missing script can be seen in the source drop at XORG_NV/packages/SUNWxorg-mesa/preremove
6390453 SUNWxorg-mesa has broken links in snv nightly build for 2/24/2006
This was a simple typo (glxext.h instead of glext.h) in a symlink in the XORG_NV/packages/SUNWxorg-mesa/prototype.
6416841 [CVE-2006-1526] X.Org bug #6642: buffer overflow in Render extension in Xorg
Again - simply applying the fix from X.Org, which you can see in XORG_NV/sun-src/xc/programs/Xserver/render/mitri.c in the source drop.
6245381 Mozilla is feeding Xorg fonts – Xorg getting fat
This actually affects both Xsun and Xorg since it was in the TrueType library code they both share.
6419340 Upgrade Xorg ATI driver from 6.5.7 -> 6.5.8
Since the 6.5.8 module was intended as an upgrade for the 6.5.7 which was included in X11R6.9 & 7.0, and retained compatibility with the 6.9/7.0 ABI (unlike the Xorg ATI 6.6.0 driver series, which requires the upcoming Xorg 7.1 server), we were able to pretty easily drop this in to our source tree, with a little bit of glue in XORG_NV/sun-src/xc/programs/Xserver/hw/xfree86/drivers/ati/localmacros. The rules inserted there (which get automatically appended to the Imakefile by imake before it generates the Makefile) remove all the Xorg 6.9 ati driver source files and link in the ones from the 6.5.8 tarball, allowing us to use (for a short time) the 6.5.8 driver with the 6.9 build system. Besides all the bug fixes listed in the ChangeLog, this also fixes the previously reported Sun bug 6402721, of restarting Xorg hard hanging the system on Acer Ferrari 4000 laptops with Radeon X700 when it tried to requery VBE without having properly restored all the register states.
6372113 Xorg receiving or generating spurious left-right wheel events
I blogged about this earlier - this was pulling in the X.Org fix from X.Org CVS into XORG_NV/sun-src/xc/programs/Xserver/hw/xfree86/input/mouse/mouse.c.
6421514 Add PCI IDs for new nvidia cards for Xorg nv driver
PCI id to name mappings were added to our Xorg nv driver for some of the new Quadro cards nvidia announced last month so that they are not reported as "Unknown nvidia device" in the Xorg log files. You can see the change in the git source tree now being used for the Xorg nv driver or in our source drop in XORG_NV/sun-src/xc/programs/Xserver/hw/xfree86/drivers/nv/nv_driver.c.

[Technorati Tags: , , , ]

Thursday May 04, 2006

Workaround and upcoming fix for Xorg scroll wheel bug

I sent a note to our internal lists today and already got a half dozen "I thought it was just my system..." responses, so for other people suffering in silence, and not recognizing this, I'll post it here too:

There's a bug in Xorg 6.9 (included in Nevada builds 28 & later, S10U2, and S10 patches) in which if you scroll the wheel too fast on your wheel mouse, you get left/right wheel events instead of up/down, which is particularly annoying in Firefox, which interprets left/right wheel as back/forward page history.

From recent conversation, it appears a lot more people have been suffering in silence with this bug than have reported it to us.

If you're one of those, you may not have noticed that bug 6372113 has been updated with both a fix-in-progress, based on the upstream fix from X.Org, and a simple workaround of:

Change or add this line in the "InputDevice" section for your mouse in /etc/X11/xorg.conf:
        Option      "ZAxisMapping" "4 5 4 5"
(The default ZAxisMapping in Xorg 6.9 is "4 5 6 7".)

With the workaround, Xorg will still think it's getting left/right events, but will report them to applications as simply more up/down events. This will break people actually using left/right capable devices, like the internal-frkit enabled Ferrari 3400 scroll pad/button/thingy. The actual fix will allow both types of scroll events to work correctly, and has been put into our sources for Nevada build 40.

[Technorati Tags: , , , ]


Engineer working on Oracle Solaris and with the X.Org open source community.


The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter


« July 2016