Monday Nov 15, 2010

X11 changes in the 2010.11 release

Another OS release came out today, 2010.11, and as usual, it has a number of X11 changes. The biggest change in X is probably... Hmm, I can see by the look on your face, you're not buying the casual use of “as usual” there. Okay, you caught me, this OS release isn't quite following our previous pattern, so I guess we better get that out of the way first. Please remember I am not an Oracle spokesman, and can't speak on behalf of Oracle, so don't even think of quoting this as “Oracle says...”

In many ways, this release is simply the continuation of the OpenSolaris distro releases of the last few years. It's built the same way, using the IPS packaging system and repositories, and Caiman installers, as the OpenSolaris 2009.06 and prior releases were. Where OpenSolaris 2009.06 (the last full release) was the biweeekly build numbered 111b, and the release we'd planned to put out as OpenSolaris 2010.03 earlier this year (and which made it to the package repository, but was not put up as downloadable ISO's) would have been biweekly build 134b, this release is 151a. You should be able to upgrade to it from OpenSolaris 2009.06 or OpenSolaris /dev builds via the package repository following the instructions in the 2010.11 release notes.

So what's different about this OS release? Well, it's not named OpenSolaris anymore for starters - it's Oracle Solaris 11 Express. We'd always said that OpenSolaris releases were leading up to Solaris 11 eventually, and this name emphasizes we're getting closer to that (though still not there yet). It also recognizes that this release is built by Oracle, not Sun nor the OpenSolaris community. While it's built on the work done by the OpenSolaris community, and many portions of it are still developed as open projects on opensolaris.org, the kernel and core utilities are once again being developed behind closed doors, and the final assembly and testing are similarly done in house. The license terms for the free downloads have changed as well (though it's still offered under support contract for commercial production use as well), and the OS images include some of the encumbered packages we'd had to keep out of OpenSolaris in order to allow OpenSolaris to be freely redistributable. (Not all of them, since some were simply EOL'ed as they were for hardware well past the end of its supported lifetime, like many of the old SPARC frame buffers.)

So with that out of the way, back to the topic at hand - what's new in the X Window System in this release? Well that depends on how far back you're coming from. You can browse the complete changelogs for X going back to the point we branched the Nevada branches from the Solaris 10 release, so I'll try to stick to the highlights.

Changes since the last OpenSolaris X11 source release

None, since the X sources on opensolaris.org are still updated automatically from our internal master gate on each commit. (In fact, since the source gates currently reflect a point between biweekly builds 153 & 154, they have changes newer than this release, such as the integrations of libxcb and FreeGLUT.)

Changes since the last OpenSolaris developer build release (b134)

There were 17 biweekly builds between the last one published to pkg.opensolaris.org/dev in March and this release. The biggest change in the X packages in this period was their packaging. Previously we built our packages using the old SVR4 package format that was used since Solaris 2.0, and in many cases following the breakdown used in the old Solaris 2 releases (SUNWxwinc for most headers, SUNWxwplt for most libraries, SUNWxwman for most man pages), and then the release team converted those to the IPS format used in the OpenSolaris releases. Like several of the other consolidations, X has now converted to building IPS packages directly, and in the process refactored the X packages to better follow the way the upstream X.Org sources were split into modules at X11R7, which also happens to be more similar to the way most Linux distros break them up. This should allow easier creation of minimized environments with the subset of X packages you need.

As for headers and man pages, they are now included in the packages they are used with - for instance all the libX11 headers and API man pages are directly in the x11/library/libx11 package. System admins can still decide to include or exclude them in their installs though, since they are tagged with the devel and docfacets”, which are the IPS mechanism for controlling optional package components. To read more about how to use these with X or the other changes in the refactoring, see the heads up messages I posted when this work integrated.

Of course, there were also the usual updates to new upstream releases - Xorg 1.7.7, freetype 2.4.2, fontconfig 2.8.0, among many others. The X server packages now also include the mdb modules and scripts for getting client and grab information from the server that I blogged about back in April.

Changes since the last OpenSolaris stable release (2009.06 / b111b)

This period saw the completion of our multiyear project to completely replace the old Solaris X code base with the X11R7 open source code base from X.Org. Solaris 10 and earlier shipped with Sun's proprietary fork of X11R6, with bits of X11R5, X11R6.4, X11R6.6, & X11R6.8 mixed in. We're now set up to much more easily track upstream and are deviated from upstream in much fewer places than before (partially due to pushing a number of our previous fixes back upstream, in other cases, we determined the upstream code was better and went with it).

We also had a very large user-visible change in build 130: all the files moved from /usr/X11 directly into /usr/bin & /usr/lib, following the work done in other parts of Solaris to move files from locations like /usr/ccs/bin and /usr/sfw to the common /usr directories. We still have symlinks in /usr/openwin and /usr/X11 for backwards compatibility, so we shouldn't break your .xinitrc calls to /usr/openwin/bin/xrdb or /usr/X11/bin/xmodmap.

Since 2009.06, we moved from Xorg 1.5 to 1.7.4. Of course, with this upgrade, we got the HAL support for input device configuration working just as X.Org started moving off HAL upstream, something we still need to deal with for Solaris - for this release, input devices are still configured in HAL .fdi files. The xorgcfg and xorgconfig programs did go away as part of this move though - fortunately more and more systems are working without any xorg.conf at all, and when one is needed, only the sections being changed have to be included, lessening the utility of programs to generate full configuration files. The new Xorg also includes support for virtual consoles on systems with the necessary kernel driver support (all x86 systems and SPARCs with frame buffers supporting “coherent console”).

We also added the synaptics touchpad driver, synergy software for sharing input devices with multiple systems, the simple xcompmgr composite manager, the xinput client for input device configuration, and finally provided IPS packaged versions of the classic xdm display manager and xfs legacy font server. The Xprint server and several related commands did go away, but the libXp library was kept for binary compatibility.

Our VNC implementation was converted from RealVNC 4.1.3 to TigerVNC 1.0.1, which is being kept up-to-date with new Xorg releases, unlike RealVNC, which hasn't really been updating it's open source release in the last few years. xscreensaver was finally updated from 5.01 to 5.11, and was actually moved out of the X gate in OpenSolaris to building as a RPM-style pkgbuild spec file with the other higher-level desktop software - hopefully in the process we fixed some long-standing bugs in our forked code.

Graphics updates included Nvidia's driver support for various new devices and OpenGL 4.0, and Intel's DRI updates, including GEM support in their DRM module. Mesa was added on SPARC to provide a matching OpenGL implementation, but with only the software renderer, no hardware acceleration.

What else has changed?

Besides the official Solaris 11 Express release information, you can find more details on changes in this release on a bunch of other blogs, such as:

But here's some changes in other parts of the OS you may not see listed on those:

Of course, that's just a small sample, the full changelogs are a few thousand items long (and unfortunately, some of the consolidations haven't published theirs outside the firewall).

Sunday Jul 25, 2010

Revelations: 145

Phoronix published a sensationalist article last week claiming that my regular e-mail updates of our biweekly builds somehow signified some out of the ordinary newsworthy event, without bothering to do even the most basic of fact checking. While I pointed this out in their forums within hours of publication, I'm still seeing it cited by other web magazines that don't bother to fact check, as well as in various e-mails and blogs, so am publishing a more complete explanation here of why it really is a non-event.

The article claimed:

As the first email of its kind in months, Alan Coopersmith who is a known X.Org contributor and longtime Sun Microsystems employee now working for Oracle, has written a new email entitled "IPS distro-import changes needed for X packages for nv_145." Alan immediately began this public email by saying, "Just when you thought you'd never see another one of these biweekly mails..."

Sadly, all they needed to do to disprove the claim that it was the “first of its kind in months” was simply follow the links from the e-mail archive page they linked to, to see that I had sent a similar message two weeks earlier for the previous biweekly build nv_144. In fact, if they checked the archives for previous months, they would have found that, except for missing build 143 (a mistake on my part), I've sent these approximately every two weeks for every biweekly build for a very long time.

Perhaps I'd confused the article's author with the offhand comment he seems to have misinterpreted, but explaining that requires a bit of background explaining what these e-mails are and why I send them in the first place.

As many OpenSolaris users know, over the course of the last couple of years, we've been transitioning from the old SVR4 package system used in Solaris 2.0 through Solaris 10 to the new Image Packaging System (also sometimes known as “IPS” or “pkg(5)”) being developed by a team of Solaris engineers and community members. Initially, we maintained two parallel distros, Solaris Express: Community Edition (SXCE), which was built using the SVR4 packages and the old Solaris installer that worked with them, and OpenSolaris (originally codenamed “Project Indiana”), which was built using the IPS packages and the new Caiman install software designed to work with them. The teams providing the package contents continued to deliver SVR4 packages, and the OpenSolaris distro team converted those to IPS.

One of the goals of the OpenSolaris distro was to provide a set of ISO install images and package repository that was completely, freely redistributable, so that it could be easily mirrored, copied, and downloaded without having to deal with the various encumbrances required by some of the third-party licenses in the traditional Solaris and SXCE packages. Unfortunately, at that time, we had not yet finished separating the encumbered code from the open source code in our X packages so that they could be included, since when Sun made its proprietary fork of X11R6 in the early 90's, the engineers never figured we'd be open sourcing Solaris a decade later and need to easily separate out the encumbered bits they were merging into the main code base.

The initial Developer Preview releases of the OpenSolaris distro thus included a set of X packages that Moinak Ghosh built from the Fully Open X (FOX) project work we'd done to rebuild our source trees from the ground up using the open source code from the X11R7 modular releases in order to ensure everything was either open source or cleanly separated out. Over the first few months, we migrated from those to the packages our team delivered to the OS as we integrated the FOX project work into our main source tree. Because these changes weren't always obvious to the external observer, I started sending notes for each biweekly build to let the team maintaining the SVR4 to IPS conversion tables know which parts of our packages they could now include, as well as any other changes they needed to know about, such as version number changes. (Our SVR4 X packages all used the same version number, a holdover from the monolithic X source days, but we've migrated to using the upstream version numbers as much as possible in the new X IPS packages.) I did this not only to try to help the distro building team, but also to help myself keep on top of the changes they made in converting our packages, so that we could understand issues users hit and know how to help them, and to learn better what we'd need to do when it came time for us to start building the IPS packages ourselves.

Last year, Sun announced that the time had finally come to start converting the builds to generate IPS packages directly, taking the next step in transitioning off SVR4 and ending the production of the SXCE distro, since the SVR4 packages used to build it would no longer be made. The ON consolidation went first, converting the packages containing the kernel, drivers and core utilities in build snv_136. The Install consolidation went next, in build snv_143, and X followed in build snv_144.

So, when I wrote “Just when you thought you'd never see another one of these biweekly mails...”, I simply meant that after build 144, all the X packages are already delivered in IPS format, so there are no SVR4 to IPS conversion files to update for them any more, so I won't need to send those - except in cases like we had in 145, where I relocated one of the files that was listed as a dependency in one of the other packages still being converted by the distro builders, so they needed to update the dependency statement for it to list the new path to the file.

Despite what Phoronix seems to have assumed, I was not in any way referring to the limbo state the OpenSolaris distro is currently in (and unfortunately, as much as I'd like to explain that, I can't), nor stating anything about build 145 that is fundamentally different than the previous builds. It should come as no surprise to anyone that while build 134 was the last build to be publicly released, we have continued work on the Nevada builds after that - after all as we've said since 2005, Nevada is the code name for the development branch in which we're working towards the next full release of Solaris (i.e. not another Solaris 10 update release, but the one we may someday call Solaris 11) - while that's been released under various forms, as the OpenSolaris open source code, or via the binary releases of the original Solaris Express, Solaris Express: Community Edition, Solaris Express: Developer Edition, or the OpenSolaris 2008.05, 2008.11, and 2009.06 distros, we've always kept driving towards the same goal, with biweekly builds assembled to test the current progress.

My mail is hardly the only externally visible sign of this - you can see changelogs for the major consolidations (ON, SFW, X, and Desktop/JDS) for build 144 (the last fully finished build, as 145 is just finishing pre-integration testing now, with a delivery deadline of Monday for packages to be included, and the distro build process starting after that. Of course, the sources are also available, and there's plenty of activity on the various commit notification and discussion mailing lists showing that we're continuing to work away on Nevada.

So unfortunately, Phoronix succeeded in making a mountain out of a molehill, confusing their readers and fellow webzine authors, but likely meeting their goal of driving more traffic to their site to generate page views for their ad revenue as people passed the link around twitter, IRC & email or linked to it from their blogs & articles. As others have pointed out, checking the facts or contacting the developers to find out the story is less juicy than it seems doesn't play well with that business model (and that's not just true for Phoronix - look at any number of the columnists for other web-based "trade publications" that generate traffic via controversial posts, and the more outraged the community gets over them and angrily passes them around to denounce, the better their numbers are - you can just imagine how many of their articles are designed to bait Groklaw or Slashdot readers).

Wednesday Feb 18, 2009

Xorg 1.5.3 known issues in OpenSolaris 2009.06 & Solaris Express build 107

As Phoronix recently noticed, we upgraded the Xorg server from 1.3 to 1.5.3 in Nevada build 107 (currently available via Solaris Express Community Edition (SXCE) Install ISO’s and OpenSolaris IPS package updates in the /dev repo).

When we integrated, I sent out a heads up message with information about the driver compatibility changes and issues we knew about at that time.

So far it’s gone mostly smooth, thanks to the help of the people who tested the pre-integration builds I posted both internally and on the X community on opensolaris.org, but there’s a few issues that have hit some people so far, most with workarounds you can use to get past them:

6801598 Xorg 1.5 ignores kernel keyboard layout setting, always uses "us"

Fixed in build 109. Xorg on Solaris has always shipped with a patch to query the kernel for the console keyboard layout (which in turn either gets set automatically by the keyboard if, like Sun keyboards, it provides the layout in the optional USB HID layout identifier, or gets set by querying the user on initial installation), but the code we apply this patch to moved from the Xorg binary to the kbd_drv.so binary in Xorg 1.5 and our patch didn’t get migrated correctly. We didn’t notice in testing since our testing was done with input devices provided by HAL, which was handling this for us, but when that didn’t integrate as expected, this bug was exposed.

Workaround: Create an /etc/X11/xorg.conf file with a keyboard section specifying your desired layout, such as this one for a British layout:

Section "InputDevice"
	Identifier		"Keyboard0"
	Driver			"kbd"
	Option "XkbRules"	"xorg"
	Option "XkbModel"	"pc105"
	Option "XkbLayout"	"gb"
EndSection

6791361 SUNWxorg-xkb should deliver "base" rules

If you do need to set a localized keyboard layout, you may notice the default XKB rules file upstream changed from "xorg" (which is currently shipped in our packages) to "base" (which isn’t yet, though we’ve asked the localization teams who build our XKB packages to add it).

Workaround: Include an XkbRules option line in your keyboard settings as shown in the example above, or make a symlink from /usr/X11/lib/X11/xkb/rules/base to the xorg file in that directory. The fix for the above bug in build 109 should also set the default used by our Xorg builds back to "xorg" for now.

6801386 Xorg core dumps on startup if hald not running in snv_107

Fixed in build 109. Yet another instance of our old friend 6724478 libc printf should not SEGV when passed NULL for %s format. This was a design decision made in SVR4 and Solaris 2.0 many years ago, to force programmers to catch NULL pointer misuse - but since Linux & the BSD’s allow it, today it’s mostly become a source of pain in porting code that while technically incorrect, works on those platforms, so the decision was made last year to change Solaris libc to match. Unfortunately, that change is still undergoing standards conformance testing, so hasn’t integrated yet, and we still have to check for NULL first.

Workaround: Make sure hald starts before Xorg does. This shouldn’t be a problem for gdm users (default in OpenSolaris), since the gdm SMF service lists the hal service as a dependency that needs to start first, but may cause problems for dtlogin, xinit or startx users.

6806763 Xorg 1.5 doesn’t start if xorg.conf contains RgbPath entry

Fixed in 109, and in upstream head. This entry in the Files section of xorg.conf was obsoleted upstream, but became a parse error instead of just being ignored, which breaks people who upgraded a system with a working xorg.conf that happened to have this line.

Workaround: Remove RgbPath lines from /etc/X11/xorg.conf.

6799573 Metacity not starting on TX Xorg 1.5

Fixed in 108. The XACE framework used by extensions such as Xtsol & Xselinux added new permission types to check in the upstream release, but Xtsol doesn’t handle all of them yet. In this case, Metacity’s constant checking of round-trip time by appending to an existing property failed because Xtsol was only handling the WriteAccess check and not the BlendAccess check.

I don’t know of a workaround for this other than not running the desktop in multi-level mode, so users of the labeled/multi-level security desktop in Trusted Extensions may want to wait to upgrade until then.

6798452 X Server will not start on x86 systems with multiple graphics devices

Not yet fixed, either in our builds or upstream, but has a simple workaround of creating an xorg.conf file listing the devices you want to use. This may affect people with a single card if your motherboard has on-board graphics as well.

6797940 Can’t do gui installs on x86 systems with Nevada 107/Nevada 108

We haven’t yet tracked down this issue, which only affects the SXCE installer on x86/x64 platforms. Workarounds include using the OpenSolaris LiveCD installer or the text only mode of the SXCE installer instead. After installation, the installed Xorg works fine (modulo the above issues).

6686 SUNWefb[w] should be part of slim_install

Fixed in 108. The SPARC drivers added in OpenSolaris build 107 for Xorg on XVR-50, XVR-100, and XVR-300 graphics cards aren’t included in the default install. You can add them after the install by doing pfexec pkg install SUNWefb SUNWefbw and then rebooting. (These drivers are only available in OpenSolaris installs, not SXCE ones, since they conflict with the Xsun drivers for these cards included in SXCE.)

Saturday Jun 14, 2008

June 11, 2008 X Server security advisories

On June 11, iDefense & the X.Org Foundation released security advisories for a set of issues in extension protocol parsing code in the open source X server common code base that iDefense discovered and X.Org fixed.

Their advisories/reports are at:

Sun has released a Security Sun Alert for the X server versions in Solaris 8, 9, 10 and OpenSolaris 2008.05 at:

Preliminary T-patches are available for Solaris 8, 9, and 10 from the locations shown in the Sun Alert - these are not fully tested yet (hence the "T" in T-patch).

The fix for these issues has integrated into the X gate for Nevada in Nevada build 92, so users of SXCE or SXDE will get the fixes by upgrading to SXCE build 92 when it becomes available (probably in 3-4 weeks, though the first week of July is traditionally a holiday week in Sun's US offices, so may affect availability).

Fixes for OpenSolaris 2008.05 users following the development build trains will be available when the Nevada 92 packages are pushed to the pkg.opensolaris.org repo (also probably in about 3 weeks from now).

Fixes are planned for OpenSolaris 2008.05 users staying on the stable branch (i.e. nv_86 equivalent), but I do not have information yet on how or when those will be available.

Fixes for users building X from the OpenSolaris sources are currently available in the Mercurial repository of the FOX project in open-src/xserver/xorg/6683567.patch.

For users of all OS versions, the best defenses against this class of attacks is to never, ever, ever run “xhost +”, and if possible, to run X with incoming TCP connections disabled, since if the attacker can't connect to your X server in the first place, they can't cause the X server to parse the protocol stream incorrectly. This is not a complete defense, as anyone who can connect to the Xserver can still exploit it, so if you're in a situation where the X users don't have root access it won't protect you from them, but it is a strong first line of defense against attacks from other machines on the network.

Releases based on the Solaris Nevada train (including OpenSolaris & Solaris Express), default to “Secure by Default” mode, which disables incoming TCP connections to the X server. Current Solaris 10 releases offer to set the Secure by Default mode at install time. On both Solaris 10 & Nevada, the netservices command may be used to change the Secure by Default settings for all services, or the svccfg command may be used to disable listening for TCP connections for just X by running:

svccfg -s svc:/application/x11/x11-server setprop options/tcp_listen=false

and then restarting the X server (logout of your desktop session and log back in).

On older releases, the “-nolisten tcp” flag may be appended to the X server command line in /etc/dt/config/Xservers (copied from /usr/dt/config/Xservers if it doesn't exist) or in whatever other method is being used to start the X server.

See the Sun Alert for other prevention methods, such as disabling the vulnerable extensions if your applications can run effectively without them.

Thursday Oct 11, 2007

Stupid nv_74 cursor tricks

For those who have already grabbed Nevada build 74 from the Solaris Express Community Edition downloads, you can try out the new cursor code available with the integration of libXcursor (which most XFree86 and Xorg platforms have had for a while).

If your graphics card supports the Render extension & 32-bit alpha cursors, (i.e. most x86 graphics, but not Sun Rays nor SPARC graphics yet - I've mainly used it on nvidia cards and my laptop with a ATI Radeon chipset), you can see them in action by doing:

% su -
# mkdir -p /etc/dt/config/C/
# cat > /etc/dt/config/C/styleModern
#include "/usr/dt/config/C/styleModern"
Xcursor.theme: whiteglass
\^D
# exit

% echo 'Xcursor.theme: whiteglass' >> ~/.Xresources

and then logout & log back in.

Change 'whiteglass' to 'redglass' in the above if you want something that stands out more.

Thursday Aug 23, 2007

Silicon Valley OpenSolaris User's Group tonight: Solaris Desktop Update

Somehow our quick demo of compiz for the user group meeting tonight in San Jose turned into a full evening on X and the desktop in Solaris. Stuart and I whipped together some slides today, and have demos of both compiz and RandR 1.2 ready to go on a Toshiba Tecra M8 laptop with i945 graphics (which means it will indirectly be demoing DRI for i945 as well). For those who can't make it, the slides are here and call in numbers are on Alan Duboff's blog. For those who can make it, his blog also has the appropriate details on time and place.

Friday Jun 22, 2007

OpenSolaris and X: Year 2

open(2) logo

I'm a bit late this year - it's now a year and a week since I posted One Year of OpenSolaris and X, and it's been a busy time.

Unfortunately, I can't say that we've released lots more code than in our first year - there's still many client applications and libraries that need to be migrated to the X.Org versions - but we have moved the source base for what we've released from the Imake-built X11R6.9 monolithic source tree to source modules from the X11R7.2 release. With the 7.2 move, we also grew the set of servers delivered from the current X.Org sources from just a 32-bit x86 Xorg by adding 64-bit Xorg servers on both SPARC and x64, as well as the Xephyr nested server and Xvfb virtual frame buffer on both platforms.

Sun's China team integrated DRI support for Intel i845 through i945 graphics, with i965 coming soon. We've also added libraries from X.Org that we hadn't shipped before, including libXcomposite and libXinerama.

Work continues on converting the rest, and may pick up with some community members joining in. The OpenSolaris X community has been discussing a proposal to create a common source repository merging the X efforts of the Solaris Express, Belenix, and marTux distributions. Martin Bochnig has been working for a while on marTux, an OpenSolaris distro for SPARC - and the only one with open source SPARC Xorg drivers he's put a lot of effort into making work well on OpenSolaris. And in April, Moinak Ghosh announced he'd taken our code drops and packaging and filled in the rest of the X11R7.2 sources. By merging these, we hope to keep all distributions better aligned and make it easier for us to work together on the missing pieces.

What else are we working on? Well, like last year, the OpenSolaris suspend-and-resume team is still hard at work making their code as rock-solid as they can, including the graphics support in Xorg. We're looking at the best way to integrate Xvnc into our code base in support of the OpenSolaris Xen work. And of course, we've been playing with the projects now known as “Compiz Fusion”, using the builds from Erwann in the Desktop group, so that maybe by the time I write next year's entry, you will have a Solaris desktop option that looks a bit like this.

Thursday Jun 14, 2007

OpenSolaris Second Anniversary

Last year at the first anniversary of the OpenSolaris launch, they held a home page graphics contest to pick a graphic for the home page. Today is the second anniversary, and I didn't see a contest this year, but I have an entry anyway, suggested by conversation yesterday on the #opensolaris channel on freenode...it's hip, modern, Web 2.0, in sync with the stuff all over the Intertubes, fully overblown meme compliant:

(with apologies in advance to pretty much the entire Internet)

I can has cheeZFS?

Wednesday Jun 13, 2007

Xorg 7.2 Solaris status update

It's been a while since I've given an update on our Xorg 7.2 state in Solaris and OpenSolaris, and I was reminded by a new patch release today. For a summary of the new features included, and some bugs in the initial Nevada builds, see the Heads Up: Xorg 7.2 in Nevada build 58 message I sent to the OpenSolaris xwin-discuss list in February.

Besides the original release in Solaris Express: Community Edition and the OpenSolaris sources, Xorg 7.2 is now also available in:

For those on Solaris 10 with an older patch release, or the Solaris 10 7/07 Beta, patch 125720-08 was released today, which fixes several issues reported by early users, including:

  • A number of problems with auto-configuration of screen resolution
  • Xephyr's Caps Lock not being unlockable (bug 6539225)
  • A memory leak every time an X client connection was closed (6533650)
  • Crashes or failures using TrueType fonts in 64-bit Xorg servers (6535518)
  • xorgconfig listing the wrong keyboard driver in generated xorg.conf files (6559079)
  • Xorg -configure not including the bitstream TrueType font module in generated xorg.conf files (6546692)

Also included in the patch are:

  • A new version 2.0.2 of the “nv” open source driver for nVidia cards which adds support for the first set of GeForce 8000 series cards, though you can also get the updated nvidia closed-source/accelerated driver to support them as well. (A slightly older version of the nVidia closed driver is included in Solaris 10 7/07 already.)
  • Security fixes for two reported vulnerabilities: X Render Trapezoid divide-by-zero and XC-MISC memory corruption.
  • An update to the Radeon driver to check if you've plugged in an external monitor on your laptop (at least on Ferrari 3400's) when you run xrandr, and if so, activate the external display port without having to restart X. (A small step towards the more general and complete solution coming with X Resize-and-Rotate 1.2.)

There's a few more fixes in progress still (the -09 rev of the Solaris 10 patch is in QA now, and there will of course be more revisions in the future), including:

  • Fixing glxgears and glxinfo linking so they run (6560568)
  • Allowing Xorg to start with an old xorg.conf listing the "Keyboard" driver (capitalized) (6560332 - if you have this problem now, just change the driver name to "kbd" in your xorg.conf)
  • Adding the VMWare mouse driver, vmmouse_drv.so (6559114)
  • Fixing Xorg -configure in the 64-bit Xorg (6556115)
  • Making Xv playback work again in the ATI Radeon driver (6564910)
  • Making xorgcfg work again ( 6563653)

but the current state should be fairly usable by most people as it is today.

For those using Solaris Express/Nevada releases, or OpenSolaris sources, you can see which build the above changes went in at the X ChangeLogs page. (Solaris Express Developer Edition 5/07 corresponds to build 64 in that list.)

Monday Jun 11, 2007

The Irregular X11 DTrace Companion: Issue 2

In response to a Solaris 10 7/07 beta tester question last week, I updated the Xserver provider for DTrace web page to point out that besides the original method of patching the Xorg sources and rebuilding, pre-built binaries are now included directly in Solaris...

Getting an X server with DTrace probes built in

The Xserver dtrace providers are now included in the regular X server binaries (including Xorg, Xsun, Xvfb, Xephyr, Xnest, & Xprt) in recent Solaris releases, including:

For those who still want to build from source, the provider is integrated into the X.Org git master repository for the Xserver 1.4 release, planned to be released in August 2007 with X11R7.3, and into the OpenSolaris X code base for Nevada builds 53 and later.

Thursday May 17, 2007

The Irregular X11 DTrace Companion: Issue 1

All the cool kids are doing unscheduled update newsletters now, and I had a collection of X11 Dtrace things to post about piling up, so I figured I'd give it a whirl. (BTW, Nouveau guys - I'd read yours more if you had an RSS feed - having to go check the wiki on an irregular basis doesn't work so well.)

Doing newsletters on a regular basis is hard, as Glynn has learned, so don't expect updates too often...

Sample script for tracing events

Jeremy asked recently for an example DTrace script using the event probes in the Xserver DTrace provider. I had lost the ones I wrote when developing/testing that probe, but dug one out of my e-mail that was written by DTrace Master Guru Bryan Cantrill and posted it as ButtonPress-events.d.

He wrote this script when he'd upgraded to Nevada build 58 and found shift-click was no longer working. Using the script he was able to show that the Xserver was sending ButtonPress events without the ShiftMask set. I wrote a corresponding script to decode the VUID events the X server was getting from the kernel mouse and keyboard drivers to determine the keyboard wasn't giving us the shift key pressed events until another key was pressed, which wasn't happening for shift-clicks with the mouse. That script can be seen in the bug report I filed with the kernel keyboard team as Solaris bug 6526932.

Using DTrace to win a race

A bug was filed with X.Org recently stating “iceauth can dump a core in auth_initialize() if a signal is caught before iceauth_filename has been malloced.” As I was fixing bugs in iceauth anyway, I figured I'd look at it to see if I could roll in a fix to the new iceauth release I'd be making for those. Fortunately, iceauth is a small program, and I could see where I thought the race condition lied between the main code and the signal handler. Testing any race condition has always been a pain in the past, since if you don't get the timing just right you don't know if you really fixed it, or just changed the timing enough to not hit it.

So I pulled up the DTrace manual and DTrace wiki and created a one-liner to force the race condition by sending the signal when I knew it was in the right spot:

 # dtrace -w -c ./iceauth -n 'pid$target::IceLockAuthFile:return { raise(1); }'

For those who haven't memorized the DTrace manual, this can be broken down as:

  • -w - allows “destructive” actions - i.e. those that change the running of the program, and don't just monitor it - in this case, allows sending a signal to the process
  • -c ./iceauth - specifies the command to run with this script, and puts the process id of the command in $target in the script
  • pid$target::IceLockAuthFile:return - sets the probe we're looking for to be the return from any call to the function IceLockAuthFile in the process matching the pid $target
  • { raise(1); } - sets the action to be run when that probe is hit to send signal 1 (aka SIGHUP) to the target process

Ran that and a few seconds later had a nice core file to verify where the race was. Turned out I was right about where the signal had to come in for the race to cause a crash, but was slightly off about the point in the signal handler where it was crashing. The fix was easy, and thanks to DTrace, easy to verify, and is now in X.Org's git repository.

DTrace Wiki: Topics: X

After last month's Silicon Valley OpenSolaris User Group meeting, Brendan asked if I'd work on the “X” topic on the DTrace Wiki. I agreed to, and have put up a small placeholder while I figure out what to flesh it out with - if you have any suggestions or requests for what you'ld like to see in a short introductory article to using DTrace to probe X, especially using, but not limited to, the Xserver DTrace provider, send me e-mail or leave a comment here.

Monday Apr 23, 2007

Compiz for Solaris

Erwann has posted Compiz 0.5.0 packages for Solaris Nevada/Express on x86 today - this is the latest step of several months of work to make this happen.

We've been working on this for several reasons. Our accessibility architect became very enthused after seeing Beryl's screen magnification support and talking to the Beryl developers at last fall's Ubuntu Developer's Summit. Our (now-former) director of Solaris Open Source (including OpenSolaris and the Solaris Desktop & X teams) became a fan after seeing the demos at the X.Org Developer's Conference Sun hosted in February. Even the Solaris CTO has jumped on the bandwagon for making the Solaris Express Developer's Edition look more modern.

We were dreading the tough choice of having to pick Beryl or Compiz - at first glance, compiz looked more stable, but beryl had more plugins/effects and the accessibility support we were interested in. Fortunately, the compiz and beryl communities solved this problem by re-uniting their efforts.

There's still work to do before this is seamless though. Compiz requires newer versions of various X components than we've had in Solaris - even February's Solaris Express Developer Edition is too old for this. We already upgraded from the X11R6.9 Xorg server to Xorg server 1.2 (from X11R7.2) in Nevada build 58 (the February SXDE was based on Nevada build 55b). I posted last week to desktop-discuss a list of where we are on the X library updates and additions needed for compiz.

Will we integrate this directly into Solaris in the future? We don't know yet, but it's definitely an option being considered. Until then, if you have nvidia graphics on Solaris x86, try it out! (Intel graphics will be an option as well soon, once the Solaris DRI team brings the Intel DRI drivers up to the level required by the X11R7.2 Xorg. We may even be able to get it running on SPARC in the not-to-distant future, once the current project to port XVR-2500 support to Xorg gets to the OpenGL stage - right now, it's mainly got the 2D portions of the driver running.)

Monday Apr 16, 2007

X11 API's for querying multi-head screen layout

I should probably stick this in the X.Org wiki or OpenSolaris/GenUnix wiki, but I'm being lazy and just summarizing to here now from an e-mail thread last week on one of the Sun internal lists...

There are now 4 sets of API's to query the layout of display screens in a logical X screen when using a setup such as Xinerama, TwinView, or MergedFB:

  1. the original X11R6.4 XPanoramiX\* API
  2. Sun's Xinerama API
  3. XFree86's Xinerama API
  4. Xrandr 1.2

The first is pretty much universally available, but also universally deprecated. The functions for it are defined in libXinerama on Linux, libXext on Solaris, but I don't think we've ever shipped the header for it on Solaris, since it was deprecated long ago.

The second Sun created and proposed to the old X.Org industry consortium to become the standard, but they did not adopt it, crafted a replacement in committee that was universally incompatible with everything and never adopted that either. It's what the <X11/extensions/xinerama.h> header in Solaris represents, and was never documented, but you can find a description of the API in PSARC 2000/036. The functions from it are found in Solaris libXext, but nowhere else in the world that I know of.

The third, aka libXinerama, is available on most XFree86 & Xorg based systems, and finally came to Solaris in nv_62 and soon in Solaris 10 patches & the next update release. It uses the <X11/extensions/Xinerama.h> (note the capitalization compared to the other) header and is documented in the man page we created for the Solaris integration and contributed back to X.Org.

The fourth is a very recent development, having been released after X11R7.2's release in February. X.Org is currently in the process of releasing version 1.3 of the Xorg server package (release candidate 5 is out now) - this will be the first Xorg server feature release not tied to one of the whole X Window System releases. (X11R7.2 had Xorg server 1.2, X11R7.3 is expected to have Xorg server 1.4.) The xf86-video-intel driver 2.0 release candidate 4 is also out, it is the first driver to support the new Xrandr 1.2 additions, though work is in progress on other drivers, including the Radeon and nvidia drivers. We don't have a definite roadmap for including this in Solaris yet - we'll probably do it once the X.Org releases are finished and we're passed the upcoming milestone of the second Solaris Express Developer Edition release in May.

Friday Mar 23, 2007

OGB Elections: Contributors and Voters

After some discussions on #opensolaris IRC today, I downloaded the lists of contributors from poll.opensolaris.org and ran some stats. Each OpenSolaris community was asked to name both Contributors and Core Contributors over the months leading up to the election - the Core Contributors are those eligible to vote. By my count, there's 268 unique Core Contributors across the communities.

First, the breakdown of Contributors by community - “unique” refers to those in each category which are not listed as either a contributor or core contributor by any other community:

CommunityCore ContributorsContributors
TotalUniqueTotalUnique
Academic and Research 3 2 3 1
Appliances 0 0 0 0
Approachability 1 0 14 8
Architecture Process and Tools 9 5 4717
BrandZ 4 1 4 3
Chinese Users 0 0 0 0
Community Advisory Board (CAB)14 8 3 0
DTrace 5 1 0 0
Databases 0 0 0 0
Desktop 8 8 6 4
Device Drivers 0 0 0 0
Documentation 5 2 12 8
Fault Management 6 1 3 0
GNU Solaris 0 0 0 0
Games on OpenSolaris 0 0 0 0
HPC Developer 0 0 0 0
Immigrants 2 0 1 1
Installation and Packaging10 6 4 1
Internationalization and Localization 5 2 11 8
Laptop11 4 0 0
Marketing 6 0 3 1
CommunityCore ContributorsContributors
TotalUniqueTotalUnique
Modular Debugger (MDB) 5 0 0 0
NFS 3 2 0 0
Networking 8 2 15 2
OS/Net (ON)4816 8 0
Observability 0 0 0 0
OpenSolaris Printing 0 0 0 0
Performance 8 4 1 0
PowerPC 4 2 11 5
Security 8 4 4 0
Service Management Facility (smf(5)) 8 1 16 8
Solaris Volume Manager 0 0 0 0
Storage 0 0 0 0
Systems Administrators 0 0 0 0
Testing 9 5 2 1
Tools22 8 20 6
Unix File Systems (UFS) 0 0 0 0
User Groups8866 1 1
X Window System 3 3 0 0
Xen15 7 1 0
ZFS14 8 1 0
Zones 7 2 3 1

As you can see 13 communities have named no contributors, which will probably result in the new OGB taking a close look at them to determine if they are really dormant or just need leaders to step up and organize them.

I also did counts of how many contributors were recognized by multiple communities as either a contributor or core contributor - these aren't necessarily the biggest contributors to OpenSolaris, just those with the widest breadth of contributions that were recognized:

# of Communities# of peopleContributors
7 2Adam H. Leventhal, James D. Carlson
6 2Casper H.S. Dik, Mike Shapiro
5 3Daniel B. Price, Eric Schrock, Stephen Hahn
4 9Bart Smaalders, Ben Rockwood, Bill Sommerfeld, Dan McDonald, Darren Reed, Kais Belgaied, Keith M. Wesolowski, Peter Memishian, Richard Lowe
3 25
2 68
1245

If you're active in one of the communities and didn't get named as a contributor, it's too late for this election, but for future status you should ask the leaders of the community what criteria they used to choose the contributors and how you can be considered for recognition.

If you were named a core contributor and haven't voted yet - hurry up! You've got a little less than 72 hours left, and while we've just passed 100 votes cast, approval of the constitution requires 50% of the eligible voters, so we need at least 30 more (and that's assuming most people voted yes on ratification). Instructions on voting are at http://opensolaris.org/os/project/website/poll_instructions/. Two tips from debugging problems reported by a pair of voters today:

  • The machine name is poll.opensolaris.org, not polls. Four engineers on IRC could not figure out why none of us could ssh to polls.opensolaris.org and were just getting mysterious connection closed messages, until stevel piped up that we were all misspelling the hostname and hadn't noticed, because the ssh/socks5 proxies we were all using didn't actually report hostname unknown errors.
  • If you copied and pasted your ssh key from a terminal window that caused line breaks to be inserted in the middle of all the ascii hex dump portion of the key, remove them - the web form allows you to put them in and doesn't warn you that it's invalid. (Again, thanks to stevel for coming to the rescue to solve that user's problem.)

Sunday Mar 11, 2007

OGB Elections: Proper Positioning

I haven't had a chance to write full position papers like some of the other candidates, so for my positions, you'll mainly have to rely on these answers I wrote to some of the questions submitted via e-mail to the candidates:

I did vote in the poll being used to test the OpenSolaris elections system, so will share my vote there, and explain some of my choices to help further show my positions on the issues facing the community.

The poll asked (and the order of these is the one I was shown - the order is randomized for each voter to reduce bias if people just start voting "1 2 3 4..."):

QUESTION 1.1: ("Priorities") Which of the following items, presented
in a randomized order, should be prioritized by the Governing Board in
order to promote OpenSolaris and increase its developer mindshare?

1 - Deploy a public defect management system ('Defect_Mgmt')
2 - Remove inactive Communities or Projects ('Remove_Inactive')
3 - Provide an x86/AMD64-based kernel/project build facility
    ('X64_Build')
4 - Deploy a public Request To Integrate (RTI) system ('RTI')
5 - Provide a SPARC-based kernel/project build facility
    ('SPARC_Build')
6 - Create an infrastructure project to run opensolaris.org
    ('Infrastructure')
7 - Replace or remove Jive forum interface ('No_Jive')
8 - Reorganize the existing Community/Project organization
    ('Reorganize_Community')
9 - Replace opensolaris.org tools with an open source CMS or wiki
    ('New_CMS')
10 - Deploy a public wiki on opensolaris.org ('Wiki')
11 - Deploy a public code review facility on opensolaris.org
    ('Code_Review')
12 - Eliminate reliability issues with opensolaris.org web pages
    ('Reliability')

My vote was:

RESPONSE 1.1 --> 1 6 9 8 4 5 3 11 10 2 7
VERIFY:  Your ballot is:
Priorities: [Defect_Mgmt Infrastructure New_CMS Reorganize_Community
 RTI SPARC_Build X64_Build Code_Review Wiki Remove_Inactive No_Jive ]

So why these?

Deploy a public defect management system

Right now bugs.opensolaris.org is not truly a two-way communication system - for people outside Sun, it's Write Once Read Many (WORM) - they can file a bug, search for bugs, and read bugs, but not update them or have them assigned to them to work on. Additionally many of the bugs and bug fields are still restricted to internal users only, though that's a harder nut to crack due to various legal and customer privacy issues.

This means projects with external contributors are at a gross disadvantage, and has led to projects such as ksh93 using a separate bug tracking system (Steve Lau graciously set up a bugzilla for them on his system), which segregates their bugs from the rest of the community, so bugs.os.o no longer has all the relevant data. This needs to be solved before non-Sun employees can be first-class members of the development community.

Create an infrastructure project to run opensolaris.org

Right now, too many improvements are stalled because the team running opensolaris.org can't keep up with the everything they need to do to implement them. Open it up now, let the community help you, and let us all move forward.

Replace opensolaris.org tools with an open source CMS or wiki

The current infrastructure is customized and weird - Project & Community Leaders are the only ones who can edit pages in their areas, so to add someone to edit anything, even just one project subpage, they have to be made a leader, even if they're not doing any leadership. The syntax is also strange and confusing to those used to just about any other wiki-like system in the world.

Reorganize the existing Community/Project organization

This has been discussed to death elsewhere already, and a big part is just cleaning up the original set of communities created before projects were done. The election confused many people since those doing the most work (projects) are not given voting rights, while those just interested in the technology (communities) are.

Deploy a public Request To Integrate (RTI) system

This is specific to the ON family of consolidations that use the RTI system (Desktop consolidations, including X, don't - I don't know about all the others), but once the code repository is outside, this will be the biggest obstacle to letting non-Sun people commit code on their own without any internal sponsor.

Provide a SPARC-based kernel/project build facility
Provide an x86/AMD64-based kernel/project build facility

SPARC is higher priority than x86, since more people have x86 system access on their own, but commits to most OpenSolaris consolidations require you to build and test on both platforms first, and that's hard to do if you aren't working at Sun or another place with both kinds of machines around and installed with the build environment.

Deploy a public code review facility on opensolaris.org
Deploy a public wiki on opensolaris.org

For both of these, while it would be nice to have on opensolaris.org, we've got community ones already that work fine - for instance, I'd be happy just making wiki.opensolaris.org redirect to http://www.genunix.org/wiki/ and declaring that one done.

Remove inactive Communities or Projects

These clutter up crowded pages and cause confusion, but don't stop any real work from being done.

Replace or remove Jive forum interface

Again, as much as it annoys me, it's not preventing real work, and can be mostly ignored if you truly hate it. (It would be nice if it didn't break threads and cross-posting and such for people who use the mailing lists instead.)

Eliminate reliability issues with opensolaris.org web pages

As much as I was tempted to vote for this just to tweak stevel, I just haven't seen enough of a reliability issue to declare this a major problem that needs attention, so I just left it off my choices completely.

About

Engineer working on Oracle Solaris and with the X.Org open source community.

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today