Friday Nov 05, 2010

Is Wayland going to replace X?

The interwebs are full this morning of reports on Mark Shuttleworth’s announcement that someday he’d like to use Wayland instead of Xorg as Ubuntu’s primary display driver.

My favorite so far is Ryan Paul’s Ars Technica article for this dead-on estimate of the amount of effort it would take to completely replace X11:

Although the Linux ecosystem would benefit greatly from a lighter and more easily extensible alternative, a concerted effort to displace X11 on the Linux desktop hasn’t really emerged yet because the task of bringing drivers, third-party software, and all of the other layers of the stack into alignment with such a move would be prohibitively cumbersome. Like, in the sense that using only your toes to build a full-scale replica of the Statue of Liberty out of toothpicks is prohibitively cumbersome.

And that’s the important point - despite what some of the sensationalist headlines are saying, Ubuntu is not dropping X. They’re not even sure when it would be possible to start shipping Wayland - the announcement from Mark Shuttleworth was that this is the direction they want to go in hopes that a show of support will help get more people working on Wayland (which has mostly been a single person effort) in order to make it a usable solution in a year or more.

Even once they start shipping Wayland, since most applications will still be written for X, they’ll have to include an X server which now has to share the graphics devices with Wayland. (The Wayland site has architecture diagrams showing how this works.) Sure, their Unity desktop (the Ubuntu replacement for GNOME 3.0’s gnome-shell) will be written to Wayland, but more widespread adoption depends on how many other platforms also adopt Wayland, since then the creators of other applications will have to decide if its worth the effort of porting just for those distros when they can do nothing and continue to work on all X11 platforms, including those running Wayland.

One of the challenges in widespread Wayland adoption, especially on non-Linux kernels, is that Wayland requires a kernel Direct Rendering Module (DRM) with Kernel Modesetting Support (KMS) for every graphics device.

That’s especially challenging in Solaris, where even in the latest OpenSolaris and Solaris Express releases, we only have DRM drivers for Intel and ATI Radeon graphics (and the Radeon one currently doesn’t work, but that’s fixable).

That leaves these non-EOL graphics devices with no DRM/KMS support in Solaris:

  • AST (service processor/remote KVM in Sun servers)
  • ATI Rage and Mach64
  • Cirrus
  • Matrox
  • Nvidia
  • VIA Unichrome
  • Trident
  • VESA (fallback for hardware without specific driver support)
  • VirtualBox
  • VMWare
  • Sun XVR-50/100/300 (OEM ATI Radeon series, but with separate driver)
  • Sun XVR-2500
  • Sun Ray

Of course, that’s the list of devices Oracle is supporting in Solaris - other distros/forks from the OpenSolaris code base, such as OpenIndiana or Belenix, may choose to support more or less, but I’ve not yet seen any sign of them porting DRM drivers on their own.

For some of those, support could be provided by porting the existing open source dual-BSD/GPL-licensed DRM code - others require creating drivers where there are none.

Nvidia appears on their list since we ship their driver, which has their own kernel/driver architecture instead of DRI. Getting DRI/KMS support for nvidia either requires them to rearchitect their driver or moving from their driver to the open source / reverse engineered Nouveau project.

From Owain Ainsworth’s talk at this year’s X Developer Summit, we know that work is in progress to port KMS support to the BSD DRI, but again, that’s not yet done, though the BSDs do have a lot more of the device-specific DRM modules already ported than Solaris did.

So Xorg is in little danger of disappearing overnight - our desktop architecture continues to evolve, as it has over many years, and Wayland may play an increasingly larger role in it, or may end up an interesting side note, like Xgl before it, but either way it will be an evolutionary process, not a big bang flag day sudden change.

Friday Oct 08, 2010

Porting X apps to XCB

I pushed the release of xwininfo 1.1 to the X.Org release archives last week. xwininfo is a command-line utility to print information about windows on an X server. The major new feature of this release is the rewrite to use libxcb instead of libX11 for the connection to the X server.

For those who haven't heard, XCB is a new library (well, new compared to Xlib) to communicate to the X server over the X11 protocol. While Xlib is designed to look like a traditional library API, hiding the fact that calls result in protocol requests to a server, XCB makes the client-server nature of the protocol explicit in its design. For instance, to lookup a window property, the Xlib code is a single function call:

XGetWindowProperty(dpy, win, atom, 0, 0, False, AnyPropertyType,
                   &type_ret, &format_ret, &num_ret,
                   &bytes_after, &prop_ret);

Xlib generates the request to the X server to retrieve the property and appends it to its buffer of requests. Since this is a request that requires a response (many requests, such as those to draw something, do not generate responses from the server unless an error occurs), Xlib then flushes the buffer, sending the contents to the X server, and waits until the server processes all the requests in turn and then sends the response to the client with the property requested. Xlib also provides utility functions to wrap that to retrieve specific properties and decode them, knowing the details of each property and how to request and decode them, such as XGetWMName, XGetWMHints, and so on.

XCB on the other hand, provides functions generated directly from the protocol descriptions, so that they map directly onto the protocol, with separate functions to put requests into the outgoing buffer, and to read results back from them asynchronously later. The xcb version of the above code is:

prop_cookie = xcb_get_property (dpy, False, win, atom,
                                XCB_GET_PROPERTY_TYPE_ANY, 0, 0);
prop_reply = xcb_get_property_reply (dpy, prop_cookie, NULL);

The power of xcb is in allowing those two steps to have as much code as you want between them, letting the programmer decide when to wait for data instead of forcing you to wait everytime you make a request that returns data. For instance, xwininfo knows in advance from the command line options most of the data it needs to request from the server for each window, so it can request it all at once, and then wait for the results to start coming in. When using the -tree option to walk the window tree, it can request the data for all the children of the current window at once, batching even further. On a local connection on a single CPU server, this means less context switches between X client and server. On a multi-core/CPU server, it can allow the X server to be processing requests on one core while the client is handling the responses as they become available, better utilizing the multiprocessing nature of the system. And on remote connections, the requests can be grouped into packets closer to the MTU size of the connection, instead of whatever requests are in the buffer when one is made that needs a response.

For xwininfo, when I ported to xcb I tested with a GNOME desktop session on OpenSolaris with a few clients open and ran it as xwininfo -root -all starting at the root of the window hierarchy and climbing down the tree, requesting all the information available for each window along the way. In my sample session it found 114 windows (in X, a window is simply a container for drawing output and receiving events, and often windows in terms of the protocol are subsets of, or borders around, the objects users think of as windows). When running locally on my Ultra 27 with it's 4-core Nehalem CPU, both versions ran so fast (0.05 seconds or less) that the difference in time was too small to really accurately measure. So I used ssh -X to tunnel an X11 connection from my office in California to a server in the Sun office in Beijing, China, and from there back to my desktop, introducing a huge amount of latency. With this, the difference was dramatic between the two:

Xlib: 0.03u 0.02s 8:19.12 0.0% 
 xcb: 0.00u 0.00s 0:45.26 0.0% 

Of course, xwininfo is an unusual X application in a few ways:

  • It runs through the requests as fast as it can, then exits, not waiting for user input (unless you use the mode where you click on a window to choose it, after which it runs through as normal). Most X applications spend most of their time waiting for user input, so the overall runtime won't decrease as much by reducing the time spent communicating with the X server.
  • It uses only the core protocol and shape extension, none of the extensions like Render or Xinput that more complex, modern applications use. Xinput, XKB, and GLX are especially problematic, as those have not yet been fully supported in an XCB release, though support has been worked on through some Google Summer of Code projects.
  • It's small enough that a complete reworking to use xcb in one shot was feasible. I couldn't imagine trying that for something as complex as Firefox, Gimp, or Nautilus.
  • It used only raw Xlib - no toolkits. Any application programmer who wants to stay sane uses toolkits like Gtk or Qt and helper libraries such as Pango and Cairo to deal with all the things that all applications need to do, and shouldn't have to reinvent - widgets, input methods, accessibility support, interacting with window managers, complex text layout for non-Latin character sets, and so on.
  • It doesn't use much of the Xlib helper functions - not a whole lot more than the protocol calls, so is more directly mappable to xcb. Applications that rely on Xlib's input method framework, Compose key handling, character set conversion, or other functions, would be harder to port without duplicating all that work (though the modern toolkits handle most of that in the toolkit layer now anyway). It did rely on the helper functions for converting the window name property from other character sets, and the xcb version right now has a regression in that it only works for UTF-8 and Latin-1 window names, but since most modern toolkits use UTF-8, you may not notice unless you run older applications with localized window names.

Fortunately, XCB also provides a method for incremental conversion from Xlib to XCB, where you can use libX11 to open the display, and pass the Display pointer it returns to all your existing code, toolkits, and libraries, and then when you want to call an XCB function, convert it to an xcb_connection_t pointer for the same connection, allowing mixing calls to Xlib & XCB API's.

This is done by building libX11 as a layer on top of libxcb, so they share the same socket to the X server and pass control of it back and forth. That option was introduced in libX11 1.2, and is now always present (no longer optional) in the libX11 1.4 release coming out this fall (Release Candidate 2 is out now).

So as another example, xdpyinfo also prints a lot of information about the X server, but it calls many extensions, and a lot of its calls aren't blocking on a response from the server. If you add the -queryExt option though, then for every extension it calls XQueryExtension to print which request, event, and error ids are assigned to that extension in the currently running server. Since those are dynamically assigned, and vary depending on the set of extensions enabled in a given server build/configuration, this is critical information to have when debugging X error reports that reference these ids. Using xdpyinfo -queryExt is thus especially needed when reporting an X error message that come from custom error handlers like the one in the gtk toolkit that omit the extension information found in the default Xlib error handler, or the person reading the bug will have no idea which extension you hit the error in.

XQueryExtension takes one extension name at a time, sends a request to the X server for the id codes for that extension, and waits for a response so it can return those ids to the caller. On the Xorg 1.7 server on my Solaris 11 test system, there are currently 30 active X extensions, so that's 30 tiny packets sent to the X server, 30 times the xdpyinfo client blocks in poll() waiting for a response, and 30 times the X server goes through the client handling & request scheduling code before going back to block again on its own select() loop.

A simple patch to xdpyinfo replaced just that loop of calls to XQueryExtension with two loops - the first calling xcb_query_extension for each extension, and then when the entire batch was ready to go to the server, a second loop called xcb_query_extension_reply to start collecting the batched replies. Gathering system call counts with truss -c showed the expected reduction in a number of system calls made by the xdpyinfo client itself:

System callXlibxcb
writev4011
poll8022
recv11729
total\*464296

\* total includes all system calls, including many not shown since their count did not change significantly. There was one additional set of open/mmap/close etc. for loading the added libX11-xcb library.

Over a tcp connection, this reduced both the number of packets, and due to tcp packet header overhead, the overall amount of data:

 Xlibxcb
TCP packets9335
TCP bytes115547726

This sort of change is far more feasible for most applications - finding the hotspots where your user is waiting for data from the server, especially at the startup of the application when you're gathering the information about the server and session to initialize your application with, and converting just those to more efficient sets of xcb calls. This is much like earlier work to reduce latency by converting repeated calls to XInternAtom with a single call to fetch multiple atoms at once via XInternAtoms.

Of course, if you still have to maintain support for older platforms, such as Solaris 10 or RHEL 5, you'll have to keep this code #ifdef'ed, since those platforms won't have xcb support, but availablity is growing on newer systems. (We didn't get it into any of the OpenSolaris distro releases before the end unfortunately, nor the first Solaris 11 Express release, but are working on it for a future Solaris release.)

Sunday Jul 25, 2010

Revelations: 145

Phoronix published a sensationalist article last week claiming that my regular e-mail updates of our biweekly builds somehow signified some out of the ordinary newsworthy event, without bothering to do even the most basic of fact checking. While I pointed this out in their forums within hours of publication, I'm still seeing it cited by other web magazines that don't bother to fact check, as well as in various e-mails and blogs, so am publishing a more complete explanation here of why it really is a non-event.

The article claimed:

As the first email of its kind in months, Alan Coopersmith who is a known X.Org contributor and longtime Sun Microsystems employee now working for Oracle, has written a new email entitled "IPS distro-import changes needed for X packages for nv_145." Alan immediately began this public email by saying, "Just when you thought you'd never see another one of these biweekly mails..."

Sadly, all they needed to do to disprove the claim that it was the “first of its kind in months” was simply follow the links from the e-mail archive page they linked to, to see that I had sent a similar message two weeks earlier for the previous biweekly build nv_144. In fact, if they checked the archives for previous months, they would have found that, except for missing build 143 (a mistake on my part), I've sent these approximately every two weeks for every biweekly build for a very long time.

Perhaps I'd confused the article's author with the offhand comment he seems to have misinterpreted, but explaining that requires a bit of background explaining what these e-mails are and why I send them in the first place.

As many OpenSolaris users know, over the course of the last couple of years, we've been transitioning from the old SVR4 package system used in Solaris 2.0 through Solaris 10 to the new Image Packaging System (also sometimes known as “IPS” or “pkg(5)”) being developed by a team of Solaris engineers and community members. Initially, we maintained two parallel distros, Solaris Express: Community Edition (SXCE), which was built using the SVR4 packages and the old Solaris installer that worked with them, and OpenSolaris (originally codenamed “Project Indiana”), which was built using the IPS packages and the new Caiman install software designed to work with them. The teams providing the package contents continued to deliver SVR4 packages, and the OpenSolaris distro team converted those to IPS.

One of the goals of the OpenSolaris distro was to provide a set of ISO install images and package repository that was completely, freely redistributable, so that it could be easily mirrored, copied, and downloaded without having to deal with the various encumbrances required by some of the third-party licenses in the traditional Solaris and SXCE packages. Unfortunately, at that time, we had not yet finished separating the encumbered code from the open source code in our X packages so that they could be included, since when Sun made its proprietary fork of X11R6 in the early 90's, the engineers never figured we'd be open sourcing Solaris a decade later and need to easily separate out the encumbered bits they were merging into the main code base.

The initial Developer Preview releases of the OpenSolaris distro thus included a set of X packages that Moinak Ghosh built from the Fully Open X (FOX) project work we'd done to rebuild our source trees from the ground up using the open source code from the X11R7 modular releases in order to ensure everything was either open source or cleanly separated out. Over the first few months, we migrated from those to the packages our team delivered to the OS as we integrated the FOX project work into our main source tree. Because these changes weren't always obvious to the external observer, I started sending notes for each biweekly build to let the team maintaining the SVR4 to IPS conversion tables know which parts of our packages they could now include, as well as any other changes they needed to know about, such as version number changes. (Our SVR4 X packages all used the same version number, a holdover from the monolithic X source days, but we've migrated to using the upstream version numbers as much as possible in the new X IPS packages.) I did this not only to try to help the distro building team, but also to help myself keep on top of the changes they made in converting our packages, so that we could understand issues users hit and know how to help them, and to learn better what we'd need to do when it came time for us to start building the IPS packages ourselves.

Last year, Sun announced that the time had finally come to start converting the builds to generate IPS packages directly, taking the next step in transitioning off SVR4 and ending the production of the SXCE distro, since the SVR4 packages used to build it would no longer be made. The ON consolidation went first, converting the packages containing the kernel, drivers and core utilities in build snv_136. The Install consolidation went next, in build snv_143, and X followed in build snv_144.

So, when I wrote “Just when you thought you'd never see another one of these biweekly mails...”, I simply meant that after build 144, all the X packages are already delivered in IPS format, so there are no SVR4 to IPS conversion files to update for them any more, so I won't need to send those - except in cases like we had in 145, where I relocated one of the files that was listed as a dependency in one of the other packages still being converted by the distro builders, so they needed to update the dependency statement for it to list the new path to the file.

Despite what Phoronix seems to have assumed, I was not in any way referring to the limbo state the OpenSolaris distro is currently in (and unfortunately, as much as I'd like to explain that, I can't), nor stating anything about build 145 that is fundamentally different than the previous builds. It should come as no surprise to anyone that while build 134 was the last build to be publicly released, we have continued work on the Nevada builds after that - after all as we've said since 2005, Nevada is the code name for the development branch in which we're working towards the next full release of Solaris (i.e. not another Solaris 10 update release, but the one we may someday call Solaris 11) - while that's been released under various forms, as the OpenSolaris open source code, or via the binary releases of the original Solaris Express, Solaris Express: Community Edition, Solaris Express: Developer Edition, or the OpenSolaris 2008.05, 2008.11, and 2009.06 distros, we've always kept driving towards the same goal, with biweekly builds assembled to test the current progress.

My mail is hardly the only externally visible sign of this - you can see changelogs for the major consolidations (ON, SFW, X, and Desktop/JDS) for build 144 (the last fully finished build, as 145 is just finishing pre-integration testing now, with a delivery deadline of Monday for packages to be included, and the distro build process starting after that. Of course, the sources are also available, and there's plenty of activity on the various commit notification and discussion mailing lists showing that we're continuing to work away on Nevada.

So unfortunately, Phoronix succeeded in making a mountain out of a molehill, confusing their readers and fellow webzine authors, but likely meeting their goal of driving more traffic to their site to generate page views for their ad revenue as people passed the link around twitter, IRC & email or linked to it from their blogs & articles. As others have pointed out, checking the facts or contacting the developers to find out the story is less juicy than it seems doesn't play well with that business model (and that's not just true for Phoronix - look at any number of the columnists for other web-based "trade publications" that generate traffic via controversial posts, and the more outraged the community gets over them and angrily passes them around to denounce, the better their numbers are - you can just imagine how many of their articles are designed to bait Groklaw or Slashdot readers).

Monday Apr 26, 2010

Grabbing information from the X server

Jay came into my office last week complaining his mouse was grabbed by a client and wouldn't let him click in any windows, but he couldn't tell which one. He remembered I said I had a script to find this out, but hadn't told him where it was. So I logged into his workstation, su'ed to root and ran a couple commands:

root@overthere:~# ps -ef | grep Xorg
jacotton   881   880   0   Apr 08 vt/2      274:24 /usr/bin/Xorg :0 -nolisten tcp -br -auth /var/run/gdm/auth-for-gdm-O_aWTb/datab
root@overthere:~# /usr/demo/Xserver/mdb/list_Xserver_devicegrab_client -p 881
Device "Virtual core pointer" id 2: 
  -- active grab 3a00000 by client 29

Device "Virtual core keyboard" id 3: 
  -- active grab 3a00000 by client 29

Device "Virtual core XTEST pointer" id 4: 
  -- no active grab on device

Device "Virtual core XTEST keyboard" id 5: 
  -- no active grab on device

Device "mouse" id 6: 
  -- no active grab on device

Device "input" id 7: 
  -- no active grab on device

Device "hotkey" id 8: 
  -- no active grab on device

Device "keyboard" id 9: 
  -- no active grab on device

root@overthere:~# /usr/demo/Xserver/mdb/list_Xserver_clients -p 881
currentMaxClients = 41
CLIENT SEQUENCE #  FD  PIDS
    0           0 ??? - NULL ClientPtr->osPrivate
    1          22  33 880 
    2          20  35 912 
    3           9  36 915 
[...]
   29     6926083  59 15839 
[...]
15822 /bin/bash /usr/bin/firefox
  15835 /bin/bash /usr/lib/firefox/run-mozilla.sh /usr/lib/firefox/firefox-bin
    15839 /usr/lib/firefox/firefox-bin
[...]

So we knew it was firefox holding the grab, and killing that process released it.

The scripts are actually simple wrappers around a loadable module for the Solaris mdb modular debugger. While I prefer source level debuggers like dbx & gdb for interactive use, the mdb module system works well for this style of use. The mdb modules are compiled C code, so can use the headers from the X server to get the type information, instead of requiring debugging information such as stabs or DWARF be built into the X server binary.

These were originally written for Solaris 9, in the dark days before DTrace, but are still useful, because they get information you don't know you need until after it's happened - for instance, with the Xserver DTrace probes you can get the client information from the client-connect and client-auth probes when the client connects, or see which clients send grab requests from the request probes when they make the calls - but when your server is already grabbed it's too late to trace unless you know how to reproduce the issue.

These have been kept in our source tree for a while, but weren't built as part of the regular builds so were often out of date when the server internals changed how things like the devPrivate fields were stored, and thus needed porting to the new X server versions when you needed them. And even when they were up-to-date, you needed to have a full X server source tree to build them, since they used headers not included in the server SDK package.

To solve this, in Nevada build 135 (sorry, that was just after the stable branch for the 2010.03 release forked off, so it won't be in there) I integrated them into the normal build process and delivered them in the X server packages. The mdb module is delivered in /usr/lib/mdb/proc/ with links to it named Xorg.so, Xvfb.so, Xephyr.so and Xvnc.so so that it should be automatically loaded when mdb attaches to a process or core of any of those programs. The scripts were delivered in /usr/share/demo/Xserver/mdb, along with a README file explaining how to use either the direct mdb module or the wrapper scripts.

Porting these to other debuggers architectures for scripting should be possible, though you'd need to deal with either having the X server debugging information available or converting the C language headers & traversal algorithms to Studio dbx's ksh or gdb's Python or whatever scripting solution your debugger uses. The grab and client information should be identical across the X servers from the xorg-server sources on all platforms (though as noted above, varying by xorg-server version as the internals change), except for the client process id information. Most platforms get the client pid at connection time, so it can be logged in the client audit log if you run with -audit, or made available to the client-auth dtrace probe, but then discard that information. On Solaris & OpenSolaris however, we keep it in a client devPrivate for the SolarisIA extension to use to adjust the priority boost given to the process with focus, so the mdb module can look it up there. Of course, there's no guarantee that the client hasn't forked a child or somehow else passed control of its X connection in a way the server wasn't informed about, but it's accurate far more often than not. Since the file descriptor information is available on all platforms, you might be able to determine the client via that somehow, such as through lsof - for instance, I've implemented an enhancement to pfiles on Solaris to show peer processes for local IPC that hopefully I'll find time to get reviewed and integrated soon.

There's of course a lot more information in the X server that might be useful to find - these were each written to solve specific problems on machines we couldn't login to directly to debug ourselves. What other problems could be solved if we had similar modules to expose other information? We can certainly enhance these as we find more - let us know if you find some.

Wednesday Feb 18, 2009

Xorg 1.5.3 known issues in OpenSolaris 2009.06 & Solaris Express build 107

As Phoronix recently noticed, we upgraded the Xorg server from 1.3 to 1.5.3 in Nevada build 107 (currently available via Solaris Express Community Edition (SXCE) Install ISO’s and OpenSolaris IPS package updates in the /dev repo).

When we integrated, I sent out a heads up message with information about the driver compatibility changes and issues we knew about at that time.

So far it’s gone mostly smooth, thanks to the help of the people who tested the pre-integration builds I posted both internally and on the X community on opensolaris.org, but there’s a few issues that have hit some people so far, most with workarounds you can use to get past them:

6801598 Xorg 1.5 ignores kernel keyboard layout setting, always uses "us"

Fixed in build 109. Xorg on Solaris has always shipped with a patch to query the kernel for the console keyboard layout (which in turn either gets set automatically by the keyboard if, like Sun keyboards, it provides the layout in the optional USB HID layout identifier, or gets set by querying the user on initial installation), but the code we apply this patch to moved from the Xorg binary to the kbd_drv.so binary in Xorg 1.5 and our patch didn’t get migrated correctly. We didn’t notice in testing since our testing was done with input devices provided by HAL, which was handling this for us, but when that didn’t integrate as expected, this bug was exposed.

Workaround: Create an /etc/X11/xorg.conf file with a keyboard section specifying your desired layout, such as this one for a British layout:

Section "InputDevice"
	Identifier		"Keyboard0"
	Driver			"kbd"
	Option "XkbRules"	"xorg"
	Option "XkbModel"	"pc105"
	Option "XkbLayout"	"gb"
EndSection

6791361 SUNWxorg-xkb should deliver "base" rules

If you do need to set a localized keyboard layout, you may notice the default XKB rules file upstream changed from "xorg" (which is currently shipped in our packages) to "base" (which isn’t yet, though we’ve asked the localization teams who build our XKB packages to add it).

Workaround: Include an XkbRules option line in your keyboard settings as shown in the example above, or make a symlink from /usr/X11/lib/X11/xkb/rules/base to the xorg file in that directory. The fix for the above bug in build 109 should also set the default used by our Xorg builds back to "xorg" for now.

6801386 Xorg core dumps on startup if hald not running in snv_107

Fixed in build 109. Yet another instance of our old friend 6724478 libc printf should not SEGV when passed NULL for %s format. This was a design decision made in SVR4 and Solaris 2.0 many years ago, to force programmers to catch NULL pointer misuse - but since Linux & the BSD’s allow it, today it’s mostly become a source of pain in porting code that while technically incorrect, works on those platforms, so the decision was made last year to change Solaris libc to match. Unfortunately, that change is still undergoing standards conformance testing, so hasn’t integrated yet, and we still have to check for NULL first.

Workaround: Make sure hald starts before Xorg does. This shouldn’t be a problem for gdm users (default in OpenSolaris), since the gdm SMF service lists the hal service as a dependency that needs to start first, but may cause problems for dtlogin, xinit or startx users.

6806763 Xorg 1.5 doesn’t start if xorg.conf contains RgbPath entry

Fixed in 109, and in upstream head. This entry in the Files section of xorg.conf was obsoleted upstream, but became a parse error instead of just being ignored, which breaks people who upgraded a system with a working xorg.conf that happened to have this line.

Workaround: Remove RgbPath lines from /etc/X11/xorg.conf.

6799573 Metacity not starting on TX Xorg 1.5

Fixed in 108. The XACE framework used by extensions such as Xtsol & Xselinux added new permission types to check in the upstream release, but Xtsol doesn’t handle all of them yet. In this case, Metacity’s constant checking of round-trip time by appending to an existing property failed because Xtsol was only handling the WriteAccess check and not the BlendAccess check.

I don’t know of a workaround for this other than not running the desktop in multi-level mode, so users of the labeled/multi-level security desktop in Trusted Extensions may want to wait to upgrade until then.

6798452 X Server will not start on x86 systems with multiple graphics devices

Not yet fixed, either in our builds or upstream, but has a simple workaround of creating an xorg.conf file listing the devices you want to use. This may affect people with a single card if your motherboard has on-board graphics as well.

6797940 Can’t do gui installs on x86 systems with Nevada 107/Nevada 108

We haven’t yet tracked down this issue, which only affects the SXCE installer on x86/x64 platforms. Workarounds include using the OpenSolaris LiveCD installer or the text only mode of the SXCE installer instead. After installation, the installed Xorg works fine (modulo the above issues).

6686 SUNWefb[w] should be part of slim_install

Fixed in 108. The SPARC drivers added in OpenSolaris build 107 for Xorg on XVR-50, XVR-100, and XVR-300 graphics cards aren’t included in the default install. You can add them after the install by doing pfexec pkg install SUNWefb SUNWefbw and then rebooting. (These drivers are only available in OpenSolaris installs, not SXCE ones, since they conflict with the Xsun drivers for these cards included in SXCE.)

Saturday Jun 14, 2008

June 11, 2008 X Server security advisories

On June 11, iDefense & the X.Org Foundation released security advisories for a set of issues in extension protocol parsing code in the open source X server common code base that iDefense discovered and X.Org fixed.

Their advisories/reports are at:

Sun has released a Security Sun Alert for the X server versions in Solaris 8, 9, 10 and OpenSolaris 2008.05 at:

Preliminary T-patches are available for Solaris 8, 9, and 10 from the locations shown in the Sun Alert - these are not fully tested yet (hence the "T" in T-patch).

The fix for these issues has integrated into the X gate for Nevada in Nevada build 92, so users of SXCE or SXDE will get the fixes by upgrading to SXCE build 92 when it becomes available (probably in 3-4 weeks, though the first week of July is traditionally a holiday week in Sun's US offices, so may affect availability).

Fixes for OpenSolaris 2008.05 users following the development build trains will be available when the Nevada 92 packages are pushed to the pkg.opensolaris.org repo (also probably in about 3 weeks from now).

Fixes are planned for OpenSolaris 2008.05 users staying on the stable branch (i.e. nv_86 equivalent), but I do not have information yet on how or when those will be available.

Fixes for users building X from the OpenSolaris sources are currently available in the Mercurial repository of the FOX project in open-src/xserver/xorg/6683567.patch.

For users of all OS versions, the best defenses against this class of attacks is to never, ever, ever run “xhost +”, and if possible, to run X with incoming TCP connections disabled, since if the attacker can't connect to your X server in the first place, they can't cause the X server to parse the protocol stream incorrectly. This is not a complete defense, as anyone who can connect to the Xserver can still exploit it, so if you're in a situation where the X users don't have root access it won't protect you from them, but it is a strong first line of defense against attacks from other machines on the network.

Releases based on the Solaris Nevada train (including OpenSolaris & Solaris Express), default to “Secure by Default” mode, which disables incoming TCP connections to the X server. Current Solaris 10 releases offer to set the Secure by Default mode at install time. On both Solaris 10 & Nevada, the netservices command may be used to change the Secure by Default settings for all services, or the svccfg command may be used to disable listening for TCP connections for just X by running:

svccfg -s svc:/application/x11/x11-server setprop options/tcp_listen=false

and then restarting the X server (logout of your desktop session and log back in).

On older releases, the “-nolisten tcp” flag may be appended to the X server command line in /etc/dt/config/Xservers (copied from /usr/dt/config/Xservers if it doesn't exist) or in whatever other method is being used to start the X server.

See the Sun Alert for other prevention methods, such as disabling the vulnerable extensions if your applications can run effectively without them.

Wednesday Mar 26, 2008

Too many dry eyes in the house

I've been falling further and further behind on blogging for a while - maybe I'll catch up someday with the cache of posts I have stored in my brain half written (there's a new issue of the X11 DTrace companion, a post on the recent X server security fixes and MIT-SHM regression they introduced, several posts for the OpenSolaris elections which are too late now, and a post or two on X in Indiana bouncing around in there).

This post on the other hand, may actually be early, since Sjögren's Syndrome Awareness Month isn't until April, but I'm posting now since the 2008 Salt Lake City Sjögren's Walkabout is this coming Saturday, March 29, 2008. My mother is one of the organizers, and my sister was going to be one of the walkers until she broke her leg last month. I'm still pledging money to the Sjögren's Syndrome Foundation for both of them though.

For those who haven't heard of Sjögren's — and that included me a few years ago — the About.com article What is Sjögren's Syndrome? and the Wikipedia page on Sjögren's Syndrome can tell you far more than I can. If you're a female who has been noticing problems with dry eyes or mouth, or you know one, you probably want to read more about it, as those can be early symptoms of something far more serious. The foundation estimates 4 million Americans suffer from the disease, and that women are approximately 90% of those.

Even as widespread as that is, it still sounds like one of the random auto-immune disorders they throw out every week on an episode of House and then rule out and you never hear of again. For my wife and I though, it became a permanent part of our lives several years ago when coincidentally both my mother and my wife's mother were diagnosed with it, and they became active in trying to raise awareness of the syndrome in their communities (as you can see in those two articles).

While they've had to adjust to the dryness in many little ways, such as always carrying a water bottle and really appreciating a gift of saliva-inducing Xylitol gum, the constant tiredness is what has really affected them most. They were both busy professionals (a pediatrician and a dental hygenist), active in professional organizations and volunteering in their communities, but have had to cut back as they just didn't have the energy to keep doing it all any more.

Our bodies are mostly water, so you can imagine the problems that happen when various parts start drying up, and that's where the real dangers of the syndrome come in, as the body's immune system starts attacking other systems. While there are treatments available for the effects on some parts of the body, there is no cure, and the Sjögren's Syndrome Foundation is devoted both to research and to spreading the word so that sufferrers can be diagnosed earlier and begin treatments sooner.

So what can you do? If you want to join me in donating, you can do so at either my mom's firstgiving.com page or directly on the foundation's web page. If not, just remember what you've read here the next time you hear a woman near you complain her eyes or mouth are always so much dryer than they used to be and pass it on.

Thursday Oct 11, 2007

Stupid nv_74 cursor tricks

For those who have already grabbed Nevada build 74 from the Solaris Express Community Edition downloads, you can try out the new cursor code available with the integration of libXcursor (which most XFree86 and Xorg platforms have had for a while).

If your graphics card supports the Render extension & 32-bit alpha cursors, (i.e. most x86 graphics, but not Sun Rays nor SPARC graphics yet - I've mainly used it on nvidia cards and my laptop with a ATI Radeon chipset), you can see them in action by doing:

% su -
# mkdir -p /etc/dt/config/C/
# cat > /etc/dt/config/C/styleModern
#include "/usr/dt/config/C/styleModern"
Xcursor.theme: whiteglass
\^D
# exit

% echo 'Xcursor.theme: whiteglass' >> ~/.Xresources

and then logout & log back in.

Change 'whiteglass' to 'redglass' in the above if you want something that stands out more.

Monday Oct 08, 2007

X Font Server (xfs) Security Hole in Solaris

As noted in the ZDNet posting X Font Server flaw hits Sun Solaris hard, the recently announced X font server vulnerabilities not only affect Solaris, but are exposed to the network by default in some Solaris installs.

What the article fails to mention is that it's only older installs that are vulnerable by default - Solaris versions up through Solaris 10 6/06 run xfs by default from inetd listening to the network. Solaris 10 11/06 and later Solaris 10 releases ask you at install time if you want your network services to default to being open or closed. Solaris Nevada/Express just closes them all by default and requires you to turn back on the ones you want. (These changes came from the Solaris Secure by Default project, which has more information on its project pages.)

Our sustaining teams are producing patches and a Sun Alert covering this issue, but until then, if you don't need the X font server (on Solaris it's really only used for remote desktop sessions from computers without the standard Solaris fonts already installed - unlike some Linux'es, local sessions don't use it), you can easily turn it off in several ways:

  • On all Solaris releases: “/usr/openwin/bin/fsadmin -d”, which will either break the link that inetd uses (Solaris 2.6-Solaris 9) or use inetadm to disable the svc:/application/x11/xfs service (Solaris 10 & later).
  • On Solaris 10 and later, you can do the same thing explicitly with “/usr/sbin/inetadm -d svc:/application/x11/xfs:default”.
  • On Solaris 2.6 through 9, you can do the traditional editing of /etc/inetd.conf to disable it, then “pkill -HUP inetd”.
  • If you'll never need it, and want to be sure it's gone, remove the xfs package with “pkgrm SUNWxwfs”.

Update: Oops, had a typo in one of the instructions above - should have been “pkill -HUP inetd”, not kill. Also, as Paul noted in the comments the Sun Alert is now published, with interim fixes soon to follow, at http://sunsolve.sun.com/search/document.do?assetkey=1-26-103114-1.

Thursday Sep 27, 2007

Happy Belated Birthday X11!

Just noticed that according to Wikipedia's X Window System Release History, X11 was first released on September 15, 1987 - 20 years old and I didn't even remember to get a birthday card...

Thursday Aug 23, 2007

Silicon Valley OpenSolaris User's Group tonight: Solaris Desktop Update

Somehow our quick demo of compiz for the user group meeting tonight in San Jose turned into a full evening on X and the desktop in Solaris. Stuart and I whipped together some slides today, and have demos of both compiz and RandR 1.2 ready to go on a Toshiba Tecra M8 laptop with i945 graphics (which means it will indirectly be demoing DRI for i945 as well). For those who can't make it, the slides are here and call in numbers are on Alan Duboff's blog. For those who can make it, his blog also has the appropriate details on time and place.

Friday Jun 22, 2007

OpenSolaris and X: Year 2

open(2) logo

I'm a bit late this year - it's now a year and a week since I posted One Year of OpenSolaris and X, and it's been a busy time.

Unfortunately, I can't say that we've released lots more code than in our first year - there's still many client applications and libraries that need to be migrated to the X.Org versions - but we have moved the source base for what we've released from the Imake-built X11R6.9 monolithic source tree to source modules from the X11R7.2 release. With the 7.2 move, we also grew the set of servers delivered from the current X.Org sources from just a 32-bit x86 Xorg by adding 64-bit Xorg servers on both SPARC and x64, as well as the Xephyr nested server and Xvfb virtual frame buffer on both platforms.

Sun's China team integrated DRI support for Intel i845 through i945 graphics, with i965 coming soon. We've also added libraries from X.Org that we hadn't shipped before, including libXcomposite and libXinerama.

Work continues on converting the rest, and may pick up with some community members joining in. The OpenSolaris X community has been discussing a proposal to create a common source repository merging the X efforts of the Solaris Express, Belenix, and marTux distributions. Martin Bochnig has been working for a while on marTux, an OpenSolaris distro for SPARC - and the only one with open source SPARC Xorg drivers he's put a lot of effort into making work well on OpenSolaris. And in April, Moinak Ghosh announced he'd taken our code drops and packaging and filled in the rest of the X11R7.2 sources. By merging these, we hope to keep all distributions better aligned and make it easier for us to work together on the missing pieces.

What else are we working on? Well, like last year, the OpenSolaris suspend-and-resume team is still hard at work making their code as rock-solid as they can, including the graphics support in Xorg. We're looking at the best way to integrate Xvnc into our code base in support of the OpenSolaris Xen work. And of course, we've been playing with the projects now known as “Compiz Fusion”, using the builds from Erwann in the Desktop group, so that maybe by the time I write next year's entry, you will have a Solaris desktop option that looks a bit like this.

Thursday Jun 14, 2007

OpenSolaris Second Anniversary

Last year at the first anniversary of the OpenSolaris launch, they held a home page graphics contest to pick a graphic for the home page. Today is the second anniversary, and I didn't see a contest this year, but I have an entry anyway, suggested by conversation yesterday on the #opensolaris channel on freenode...it's hip, modern, Web 2.0, in sync with the stuff all over the Intertubes, fully overblown meme compliant:

(with apologies in advance to pretty much the entire Internet)

I can has cheeZFS?

Wednesday Jun 13, 2007

Xorg 7.2 Solaris status update

It's been a while since I've given an update on our Xorg 7.2 state in Solaris and OpenSolaris, and I was reminded by a new patch release today. For a summary of the new features included, and some bugs in the initial Nevada builds, see the Heads Up: Xorg 7.2 in Nevada build 58 message I sent to the OpenSolaris xwin-discuss list in February.

Besides the original release in Solaris Express: Community Edition and the OpenSolaris sources, Xorg 7.2 is now also available in:

For those on Solaris 10 with an older patch release, or the Solaris 10 7/07 Beta, patch 125720-08 was released today, which fixes several issues reported by early users, including:

  • A number of problems with auto-configuration of screen resolution
  • Xephyr's Caps Lock not being unlockable (bug 6539225)
  • A memory leak every time an X client connection was closed (6533650)
  • Crashes or failures using TrueType fonts in 64-bit Xorg servers (6535518)
  • xorgconfig listing the wrong keyboard driver in generated xorg.conf files (6559079)
  • Xorg -configure not including the bitstream TrueType font module in generated xorg.conf files (6546692)

Also included in the patch are:

  • A new version 2.0.2 of the “nv” open source driver for nVidia cards which adds support for the first set of GeForce 8000 series cards, though you can also get the updated nvidia closed-source/accelerated driver to support them as well. (A slightly older version of the nVidia closed driver is included in Solaris 10 7/07 already.)
  • Security fixes for two reported vulnerabilities: X Render Trapezoid divide-by-zero and XC-MISC memory corruption.
  • An update to the Radeon driver to check if you've plugged in an external monitor on your laptop (at least on Ferrari 3400's) when you run xrandr, and if so, activate the external display port without having to restart X. (A small step towards the more general and complete solution coming with X Resize-and-Rotate 1.2.)

There's a few more fixes in progress still (the -09 rev of the Solaris 10 patch is in QA now, and there will of course be more revisions in the future), including:

  • Fixing glxgears and glxinfo linking so they run (6560568)
  • Allowing Xorg to start with an old xorg.conf listing the "Keyboard" driver (capitalized) (6560332 - if you have this problem now, just change the driver name to "kbd" in your xorg.conf)
  • Adding the VMWare mouse driver, vmmouse_drv.so (6559114)
  • Fixing Xorg -configure in the 64-bit Xorg (6556115)
  • Making Xv playback work again in the ATI Radeon driver (6564910)
  • Making xorgcfg work again ( 6563653)

but the current state should be fairly usable by most people as it is today.

For those using Solaris Express/Nevada releases, or OpenSolaris sources, you can see which build the above changes went in at the X ChangeLogs page. (Solaris Express Developer Edition 5/07 corresponds to build 64 in that list.)

Removing XInitThreads from Totem

Bastien, you're not alone. If the JDS team complained to even Sun's X11 team about bugs after they applied that Totem patch, we'd tell them they're on their own and we'd never support an application that called X from multiple threads without calling XInitThreads first.

The Solaris Xlib is thread safe (though probably with a few bugs in corner cases still) based on the same X11R6 multi-thread code as everyone else, though we're currently working to reconcile the fixes from our fork with the current X.Org code as part of our ongoing project to un-fork Solaris X11 and bring it back to the X.Org code base. We're also looking at XCB in the future for a cleaner thread-safety model as well.

Update: As noted by trisk in the comments, this seems to have been caused by known Solaris libXi bug 6551484 - the engineer who had removed XInitThreads() from totem tried the test binary produced by the engineer working on the libXi bug and found it solved the hang problem she was seeing.

About

Engineer working on Oracle Solaris and with the X.Org open source community.

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today