Friday Oct 06, 2006

XACE merged into Xorg for X11R7.2

The XACE framework for handling security policy extensions has been merged to the Xorg server code base for the upcoming X11R7.2 release. This is the rough equivalent of the hooks that were put into the core Solaris Xsun & Xorg servers to call out to the Xtsol extension module as necessary to implement the security policy.

XACE was originally designed by Eamon Walsh at the NSA for SELinux, and working with us this summer, modified to add the additional hooks needed by Xtsol, so it could serve as a common framework acceptable to both SELinux & Solaris Trusted Extensions. (For instance, in the original design, there were no hooks for auditing, as the SELinux code did not audit X requests as Xtsol does.)

The actual policy extension modules (X-SELINUX & Xtsol) were not ready to merge in time for the 7.2 release, so they are planned for the 7.3 release. (X.Org is currently doing full releases every 6 months, May & November, but individual modules can release at any time they are ready, so just because we missed 7.2 doesn't mean we have to wait until next May to integrate Xtsol to X.Org.)

For those who aren't familiar with the technology, a brief overview may be found in the slides [PDF format] from my talk on Security Extensions in X from this summer's Desktop Developer's Conference. A much more detailed look at the Solaris Trusted Extensions OS as a whole, including the X server and desktop, can be found in the slides from the Trusted Extensions talk by Glenn Faden at last week's Silicon Valley OpenSolaris User Group meeting.

[Technorati Tags: , , , , ]

Friday Aug 25, 2006

X Changes in Nevada Build 47

I've fallen behind lately on posting the OpenSolaris X code drops - but I'm back on schedule today with the posting of Nevada Build 47. The big change this build is 4869280: Update xscreensaver to 5.0 (from our previous version 4.05). We still modify it heavily, though the changes to use the SCF smartcard API directly have been removed, and we now rely on PAM for our smartcard support. Other changes we made doing this upgrade time were noted in the ARC review.

Other changes in X in this build:

6457364 SUNW0xman & SUNW0xpmn prototypes should be autogenerated from main
SUNW0\* packages are generated to send to Sun's localization teams for translation - they are templates for localized packages containing the English text and just the files to be translated. Instead of having to update both the base packages and the templates every time we add a man page now, a perl script generates the l10n templates from the base packages.
6454339 Xorg modularization: libXau 1.0.2 (missing FILTER entries in libX11)
In the old X.Org monolithic build, a couple files from libXau were symlinked into the libX11 source and built into libX11 - a holdover from the days before shared library dependencies worked everywhere. Since the systems all supported by X11R7 handle shared library dependencies correctly, this was replaced in X11R7 modular builds by just having libX11 depend on libXau.
In build 46, libXau was replaced with the X11R7 modular version. Instead of symlinking from our modular build tree into our monolithic tree, our monolith libX11 had the files removed and uses the dependency just as the modular libX11 does. Unfortunately, since libX11 was shipped years before we started using linker scoping to hide symbols like this, the function names in libX11 were long exposed, though versioned as SUNWprivate so that appcert would warn that applications could be broken by relying on them. Preserving binary compatibility was easy though, simply by adding FILTER function entries to the libX11 mapfile, like this:
SUNWprivate {
        XauDisposeAuth          = FUNCTION FILTER;
        XauFileName             = FUNCTION FILTER;
        XauGetBestAuthByAddr    = FUNCTION FILTER;
        XauReadAuth             = FUNCTION FILTER;
Now any function that tries to call those functions in libX11 will get automatically redirected to the correct location in libXau. However, this mapfile change got accidentally missed in the original putback to build 46, so went in this build instead. Because of this, we found that the gnome-panel in Solaris was actually using these functions from libX11 instead of linking to libXau directly as it should have, so while build 47 will restore compatibility for it, the GNOME team is fixing it to link correctly against libXau. (This is the root cause of bug 6461529: gnome-panel crashes when selecting Launch ONLY if logged in using gdm in Nevada build 46.)

The rest are pretty well summarized in their bug reports:

6459557 remote logins to xdm fail since fix for 6398796
6460081 Xorg modularization: libXdmcp 1.0.2
6237253 Xserver man page should include SMF examples
6459143 Need to ship pkgconfig files for modular protocol header packages

[Technorati Tags: , , , ]

Sunday Aug 06, 2006

X Changes in Nevada Builds 43-45

I've fallen behind in both posting the code drops for the OpenSolaris X sources and the summaries of the code changes. I'll try to get back on track with the upcoming Nevada build 46, but for now here's some quick highlights of what went into builds 43, 44, and 45. (For the full list of changes, see the OpenSolaris X Community ChangeLogs page.)

Build 43 — Source Drop 20060612

6398796 Solaris-10: Unable to login thru xdm once password is aged
xdm suffered from the same problem as many legacy programs who had simple password checking code replaced with PAM - instead of implementing a full PAM conversation, where PAM could prompt the user for multiple items, or none at all if using non-keyboard-input authentication methods like smartcards or thumbprint scanners, it just continued to always ask for a username and password and pass them to PAM via a hack that simulated a conversation. While this bug could have been fixed by slightly extending that hack, our experience in trying that with xscreensaver and consultation with Sun's PAM gurus convinced us the best way to solve this was a complete rewrite of xdm's PAM code to offer a full conversation, so we didn't end up with multiple layers of hacks that still didn't offer the full PAM functionality and kept needing to be rewhacked for every place it was found a little more of PAM was needed. This rewrite has also been integrated to the X.Org xdm module, where it's planned for inclusion in an upcoming xdm 1.1 release.
6424854 Decomposition of SUNWxwplt [PSARC/2006/302]
In preparation for the upcoming switch of the Solaris x86 install mini-root from Xsun to Xorg, and to the delight of people everywhere who minimize their Solaris systems, the single SUNWxwplt package which previously contained both the Xsun server and the core X client libraries and applications has been split into three packages. Now SUNWxwplt is the core client-side of X, the parts other packages like Java depend on for libX11 and friends, while Xsun is in a new SUNWxsun-server package. The Xsun keytables were split out to a third package to allow us to hand off responsibility for that package to the localization centers in Sun who have been maintaining them in our package for several years now. And finally, Xprt was split out to a fourth package, to allow us to more easily change it from the old Xsun-based implementation to an Xorg-based version in the future.
6437461 Xorg modularization: common extension protocols
A whole bunch of headers for X11 protocol extensions (see the bug for the full list) were removed from our old X11R6 monolithic build tree and replaced with the corresponding X11R7 proto packages. Users shouldn't see anything, but people building will find us another step closer to being able to build the rest of the X11R7 stack.
And a whole batch of bugs from Henry Zhao's work to improve Xorg auto-configuration - he's working to get these upstream to X.Org as well.
  • 6420892 ATI ES1000: resolution too low on Sun 24-inch LCD
  • 6420309 auto-config improve: Need to move VBE DDC fallback probing from server to drivers
  • 6420320 auto-config improve: nv – Need to consider panel size in mode validation
  • 6420311 auto-config improve: Ferrari 4000 starts with blank screen without xorg.conf
  • 6437062 auto-config: radeon – reboot needed for CRT to function when connected later on Ferrari 4000

Build 44

6436994 radeon: negative refresh rates preventing resolution selection in JDS
Also reported as X.Org Bugzilla #6966, the refresh rates reported by Xrandr when using the ATI driver in MergedFB mode were the combined rates of both screens, which made them appear to be negative and thus caused the GNOME "Change Display Resolution" tool to declare them invalid and not let you change the resolution of a MergedFB display.
And a whole pile of fixes from the Solaris Trusted Extensions team for the XTSol X extension...

Build 45

6261914 Removal of STSF & Xst [PSARC 2006/087]
Sun stopped funding the STSF project a couple of years ago, and switched fully to using Xft in the JDS/GNOME desktop for the Solaris 10 release. This code has been causing problems lately for the integration of other projects, such as Project Looking Glass, so we've announced in the Solaris 10 6/06 release notes plans to remove in the future, and removed it now from Solaris Nevada.
6444546 ia_find_display has small memory leak / fails to cache
The bug report pretty much explains it all - fortunately, in most applications, this is called once during XOpenDisplay(), which most applications only call once, so you'ld lose only a handful of bytes per application. (This code is part of the SolarisIA extension for giving a kernel scheduler priority boost to the process with focus - we've made the code available to X.Org, but it's not integrated into any Xorg releases, so no one else has this particular leak either.)
6450019 root cannot unlock screen
A recent fix to the code in our xscreensaver to allow you to unlock a normal user's session with the root password unfortunately did not allow you to unlock a root session with any password. Oops! (But users who understand security don't login to a desktop session as root anyway.)

[Technorati Tags: , , , ]

Thursday Jun 15, 2006

X Changes in Nevada Build 42

Build 42 sources were posted last week in Source Drop 20060530, so it's time once again for the brief summary of what's changed.

6314490 X app dumps core with LC_ALL != C when XtOpenDisplay() is called twice
The bug report provides a pretty complete description - the fix from our i18n team updated the way we close dlopen()'ed locale modules in XCloseDisplay(). Our libX11 locale module handling is quite a bit different than the X.Org versions - hopefully we'll have the sources to that released to OpenSolaris in a few months.
6416842 [CVE-2006-1526] buffer overflow in Render extension in Xsun
We already released Xorg patches for the recent Render security hole for Solaris. The same code is present in the Xsun sources in Solaris 10 and later as well, but currently disabled at build time, so we checked in the fix to make sure that if we ever re-enable the code in Xsun we don't reintroduce the security hole, but aren't going to release patches that fix code that can't be run.
6425531 integer overflows in FreeType
The recent release of FreeType 2.2.1 included a number of fixes for integer overflows, to prevent crashes or memory corruption when processing fonts with invalid sizes for data tables. Unfortunately, it also includes changes which break the builds of many existing programs including GNOME's Pango library and the Xorg X server, so we can't just upgrade to it until all the software that uses it is fixed. Thus, we've pulled out the integer overflow checks and backported just those to our current FreeType 2.1.10 as a temporary fix.
And a huge pile of keytable and XKB layout data fixes from our localization teams:
  • 6310310 Belgian keytable file "Belgian5.kt" is not present in keytables directory
  • 6325002 Norwegian "no" keyboard layout contains some wrong symbols.
  • 6339418 Can't switch to Finnish keyboard layout using Xsun.
  • 6353678 Hungarian keyboard layout for TYPE6 keyboards is missing for Xsun.
  • 6370065 New keyboard layouts does not work with XKB extension on x86 + Xsun.
  • 6370108 Bulgaria6.kt and Russia6.kt files contain no cyrillic symbols.
  • 6370138 Some keyboard layouts in nevada don't work in Xorg with SunTYPE6 keyboards
  • 6370147 Keyboard software for Macedonia needed.
  • 6370441 Cannot login in login window(dtlogin) when keyboard layout is changed from US English to Hebrew
  • 6384899 Slovenian keyboard layout for x86+Xsun and Sparc contains some errors.
  • 6384921 In Icelandic keyboard layout for x86+Xsun and Sparc are missing some symbols.
  • 6386202 Russian Xorg symbols file needs updated as it includes some incorrect key mappings
  • 6386205 Bulgarian Xorg symbols file needs updated as it includes some incorrect key mappings
  • 6389541 Croatia keyboard layout does not work in Xorg.
  • 6421192 Cyrillic based country layouts should toggle latin/cyrillic using Altgr key
  • 6426647 fr_CH and de_CH keyboard layouts have several wrong mappings on x86 (Xorg).
  • 6426648 fr_CH and de_CH keyboard layouts should be available directly form xorgconfig

[Technorati Tags: , , , ]

Wednesday Jun 14, 2006

One Year of OpenSolaris and X

OpenSolaris 1 Year Anniversary

One year ago today, I wrote about how X fit into the OpenSolaris plans and why the portions released then were just the first of the Solaris consolidations to be released. Since then, OpenSolaris has expanded from just the core OS/Networking (ON) consolidation to include the X Window System, Java Desktop System, Installation & Packaging, Storage, and various other consolidations.

It took a little longer than originally planned to start getting our X sources out (the original schedule I posted in September assumed X.Org would be sticking to their scheduled release of 6.9 in October, but that slipped to nearly the end of December), but they were released at the end of March.

As of the latest posted biweekly code drop, we've released the build infrastructure, source changes, and packaging to the Xorg server; the Mesa, Xft2, freetype, and fontconfig libraries; our forked version of xscreensaver; and the header files in the xproto package at the root of the Xorg modular dependency tree. Work on converting our tree to use the Xorg modular sources continues, and every time we convert a package, that's one more bit of the tree for which our sources become published.

Additionally we've published the sources to the Xtsol extension which provides multi-level security in X, and came to us from the Trusted Solaris product, which is being reshaped into the Trusted Extensions add-on package for Solaris. One of my big current projects is reconciling this with the X-ACE & SELinux X work which was done in a branch of the old X.Org monolith tree and never merged in, so that we can get changes merged into the main X.Org source needed to support both projects.

Going forward, while we know we'll probably never match the X.Org release 100% and always have some customisations and fixes we apply to it, our goal is really to concentrate development in a way that most changes flow back into the main X.Org tree, and that our OpenSolaris releases become as much as possible just a thin layer of building and packaging around those. (That's not just my wishful thinking either, but the instructions we've been given by my boss's boss.)

So where are we going with X in Solaris? As you could probably guess from the presentations our team gave at this year's X Developer's Conference two areas currently getting a lot of our attention are improving the autoconfiguration of Xorg (we still ship Solaris with no xorg.conf by default, relying on autoconfiguration, and want to make that work well enough that xorg.conf files become a rarely used item for exception cases) and in getting power management, especially suspend-and-resume, working reliably on Solaris. Changes to Xorg to improve autoconfiguration have started appearing in recent Solaris Nevada builds and the OpenSolaris code drops, and there's a bunch more in testing and development now. Many of these have also been fed back to the X.Org bugzilla as well, where we hope they'll go in the main drivers and benefit all platforms.

Xorg is definitely our future, as we're concentrating new development there and Xsun is moving to more of a maintenance state. DRI for Solaris Xorg is coming from our compatriots in the Solaris driver team in Beijing, and we're hoping to see some nice performance boosts for the ATI and Intel graphics chips there, though they'll still probably lag a bit behind what NVIDIA's accelerated driver can do for their hardware on Solaris. I wish I could give timeframes for when Solaris will get DRI or Xorg support for SPARC graphics or Sun Ray, but unfortunately, my crystal ball doesn't work that well (as my far off the mark dates for our OpenSolaris X releases proved).

So there you have it - the past, present, and future of the X Window System in OpenSolaris one year into this wild ride. Who knows where we'll be after year two...

[Technorati Tags: , , , ]

Friday Jun 09, 2006

X Changes in Nevada Build 41

I realized today when preparing the build 42 source drop that I had not yet posted the bug list from build 41 here - though it's been up on the ChangeLog page for two weeks, and the source was posted as Source Drop 20060516 at the same time.

6424349 prepare xscreensaver sources for OpenSolaris release
Sources for our highly-modified fork of xscreensaver 4.05 are now included in the main tarball. As an added bonus, the build 41 source drop included a separate tarball featuring a port of those changes to the recently released XScreenSaver 5.0 which is being tested for integration into a future build of Solaris. As you can see, we've made many changes, including the GTK+ unlock dialog box, a major overhaul of the PAM code to allow more complex PAM conversations than just “Type in your password,” support for accessibility helper programs, and a whole bunch of other things.
6425513 xproto 7.0.4 -> 7.0.5
This update simply pulled one of our previous Solaris patches directly into the upstream source since we contributed it back to X.Org after doing the 7.0.4 integration.
6425506 /usr/openwin/demo/maze segfaults when $DISPLAY not set
While this hasn't been released in our source drops yet, you can probably find lots of examples of this bug in other open source apps, especially those written on OS'es where passing NULL to printf routines doesn't cause core dumps as it does on Solaris (including the really old versions of SunOS this program was written for). The fix is trivial, since Xlib provides XDisplayName() for this very case:
   if ((dpy = XOpenDisplay(display)) == NULL)	{
-    fprintf(stderr, "Can\\'t open display: %s\\n",
-	    (display ? display : getenv("DISPLAY")));
+    fprintf(stderr, "Can\\'t open display: %s\\n", XDisplayName(display));

6424870 add symlinks in /usr/lib to, etc
Most of the X libraries have symlinks in /usr/lib so they can be easier found by ld and dlopen(), but we hadn't added these yet for the libraries added to support Xorg extensions like Xrandr. Java wanted to dlopen these, so we added the links to avoid having to make Java do LD_LIBRARY_PATH or other workarounds.
6397125 Radeon driver: fails to read hsync/vsync rates from EDID
6423278 auto-config improve: radeon – Sometimes does not sort modes correctly
Two more fixes from our project to improve the configurations the Xorg server chooses by default when you don't have a configuration file or generate one via Xorg -configure. Henry Zhao, the engineer working on these fixes, has also been submitting them to X.Org so they may appear upstream in the future as well.

[Technorati Tags: , , , ]

Tuesday May 16, 2006

X Changes in Nevada Build 40

For build 40, I got the source drop posted already, so if you want to follow along in the sources, I'll point out which files you can look in to see these changes, for those in the portions included in our open source releases so far.

6414453 ogl-select fails to start
A simple shell script quoting error - while not in the source drops, since this is a shell script, you can see the changes in /lib/svc/method/ogl-select in the installed files.
6388471 [Xorg bug 5897] xdm: race condition on $HOME/.xsession-errors being readable
The fix from the X.Org bug report has been applied to our Xsession script in /usr/openwin/lib/X11/xdm/Xsession.
6421217 xscreensaver cannot be launched on trusted jds
The Trusted Extensions changes made in the previous build forgot to initialize a variable in one code path - you'll be able to see the fixed code in the next build, when the Solaris xscreensaver sources are released for the first time.
6366490 Motif pull-down menus don't draw correctly with Xnest
Unfortunately, our Xnest is still based on the Xsun source, so this source is also not available [maybe trying to point to source wasn't a good idea - oh well, I'll plow on since I don't want to start over now]. The fix here was to correctly pass through the expose events from the underlying X server to the X client running on top of Xnest.
6386535 SUNWxorg-mesa package can fail to install symlink for /usr/include/GL
Aha! One I can point to the source for in our source drop! This was caused by the preremove script for SUNWxorg-mesa not being included in the package - the missing script can be seen in the source drop at XORG_NV/packages/SUNWxorg-mesa/preremove
6390453 SUNWxorg-mesa has broken links in snv nightly build for 2/24/2006
This was a simple typo (glxext.h instead of glext.h) in a symlink in the XORG_NV/packages/SUNWxorg-mesa/prototype.
6416841 [CVE-2006-1526] X.Org bug #6642: buffer overflow in Render extension in Xorg
Again - simply applying the fix from X.Org, which you can see in XORG_NV/sun-src/xc/programs/Xserver/render/mitri.c in the source drop.
6245381 Mozilla is feeding Xorg fonts – Xorg getting fat
This actually affects both Xsun and Xorg since it was in the TrueType library code they both share.
6419340 Upgrade Xorg ATI driver from 6.5.7 -> 6.5.8
Since the 6.5.8 module was intended as an upgrade for the 6.5.7 which was included in X11R6.9 & 7.0, and retained compatibility with the 6.9/7.0 ABI (unlike the Xorg ATI 6.6.0 driver series, which requires the upcoming Xorg 7.1 server), we were able to pretty easily drop this in to our source tree, with a little bit of glue in XORG_NV/sun-src/xc/programs/Xserver/hw/xfree86/drivers/ati/localmacros. The rules inserted there (which get automatically appended to the Imakefile by imake before it generates the Makefile) remove all the Xorg 6.9 ati driver source files and link in the ones from the 6.5.8 tarball, allowing us to use (for a short time) the 6.5.8 driver with the 6.9 build system. Besides all the bug fixes listed in the ChangeLog, this also fixes the previously reported Sun bug 6402721, of restarting Xorg hard hanging the system on Acer Ferrari 4000 laptops with Radeon X700 when it tried to requery VBE without having properly restored all the register states.
6372113 Xorg receiving or generating spurious left-right wheel events
I blogged about this earlier - this was pulling in the X.Org fix from X.Org CVS into XORG_NV/sun-src/xc/programs/Xserver/hw/xfree86/input/mouse/mouse.c.
6421514 Add PCI IDs for new nvidia cards for Xorg nv driver
PCI id to name mappings were added to our Xorg nv driver for some of the new Quadro cards nvidia announced last month so that they are not reported as "Unknown nvidia device" in the Xorg log files. You can see the change in the git source tree now being used for the Xorg nv driver or in our source drop in XORG_NV/sun-src/xc/programs/Xserver/hw/xfree86/drivers/nv/nv_driver.c.

[Technorati Tags: , , , ]

Thursday May 04, 2006

Workaround and upcoming fix for Xorg scroll wheel bug

I sent a note to our internal lists today and already got a half dozen "I thought it was just my system..." responses, so for other people suffering in silence, and not recognizing this, I'll post it here too:

There's a bug in Xorg 6.9 (included in Nevada builds 28 & later, S10U2, and S10 patches) in which if you scroll the wheel too fast on your wheel mouse, you get left/right wheel events instead of up/down, which is particularly annoying in Firefox, which interprets left/right wheel as back/forward page history.

From recent conversation, it appears a lot more people have been suffering in silence with this bug than have reported it to us.

If you're one of those, you may not have noticed that bug 6372113 has been updated with both a fix-in-progress, based on the upstream fix from X.Org, and a simple workaround of:

Change or add this line in the "InputDevice" section for your mouse in /etc/X11/xorg.conf:
        Option      "ZAxisMapping" "4 5 4 5"
(The default ZAxisMapping in Xorg 6.9 is "4 5 6 7".)

With the workaround, Xorg will still think it's getting left/right events, but will report them to applications as simply more up/down events. This will break people actually using left/right capable devices, like the internal-frkit enabled Ferrari 3400 scroll pad/button/thingy. The actual fix will allow both types of scroll events to work correctly, and has been put into our sources for Nevada build 40.

[Technorati Tags: , , , ]

Friday Apr 28, 2006

X Changes in Nevada Build 39

The ChangeLog for the X Consolidation for Solaris Build 39 has now been posted. However, the source drop won't be available to next week due to some lab work that made our file server and build machines unavailable today, (and since I had to work on that, I couldn't prepare the source drop on other machines either).

For OpenSolaris source release, the most notable change in this build is that it contains the first replacement of existing Solaris X sources (from our not-yet-opened portion of the tree) with the equivalent sources from the X11R7 modular release, resulting in both a newer version of the sources being used in Solaris, and more sources available as part of our OpenSolaris release. It's nothing major - just the xproto-7.0.4 package which delivers the base X11 protocol headers and some headers used by the rest of the X stack, but it's the base of the modular dependency tree, and thus a necessary first step. (Functionality-wise, the most notable change in the headers is yet another batch of keysym name definitions.)

The full list of fixes is:

6406200 need trusted logo in xscreensaver lock program
When you lock the screen in a Solaris Trusted Extensions session, the logo will show you that, instead of showing the normal Solaris lock logo.
6385078 xlock is not passing PAM_CHANGE_EXPIRED_AUTHTOK to pam_chauthtok
A trivial fix noted by our PAM gurus after they found a similar problem in Solaris su - when a password has expired and you need to change it, you're supposed to call pam_chauthtok with the PAM_CHANGE_EXPIRED_AUTHTOK flag. The basic password Solaris PAM modules don't seem to have minded this omission, but others may need it.
6374699 FMRI application/x11/xfs should run as noaccess
For years on Solaris our inetd.conf entry to start the X Font Server listed it to run as the nobody user, even though that's really only supposed to be used on Solaris for NFS mounts when "squashing" root privileges. We copied that setting to the SMF manifest when we converted from inetd.conf to SMF, but have now updated that to the more appropriate noaccess account.
6411370 X sources should use FamilyInternet6 instead of FamilyInternetV6
When I first wrote the IPv6 changes for Solaris, I called the #define for the family name FamilyInternetV6, but when the X.Org standards committee reviewed it, they decided to drop the V to be consistent with other uses such as AF_INET6 in the BSD sockets API for IPv6. I finally updated the uses of this definition in the Solaris X sources to match.
6409332 infinite loop in XFlushInt() on x86/32-bit
See my previous blog entry on “The Compiler Bug that Wasn't”.
6411857 Xorg modularization: xproto-7.0.4
As noted above
6411989 makekeys needs to handle Unicode-mapped keysyms
Since the libX11 source hasn't been updated to the latest X.Org version yet, this change from the X.Org libX11 had to be pulled into the makekeys program used to generate the hash tables used in our existing libX11 to handle the new Unicode-mapped keysyms that are now in the keysymdef headers installed in the xproto-7.0.4 package.
6413255 xdm checks for username of "root" instead of uid 0 when doing non-console login check
The description pretty much says it all, and while we haven't released our xdm source yet, this change was given back to X.Org and is now included in the just-released xdm-1.0.4 module.
6303855 ATI driver performance is poor
As discussed in X.Org bug #5867, the ATI RageXL chips builtin to certain motherboards (including those of some Sun systems, like the Ultra 20) go faster if you tell Xorg not to use extra frame buffer memory to cache pixmaps. This is an updated fix for that which doesn't add a new configuration flag as was previously proposed.
6398094 default resolution too low on metropolis workstation
6406044 Screen off center with left margin on 24.1" monitor with analog input
Two more fixes from the team working to improve our Xorg autoconfiguration experience. These update the modeline selection code in Xorg and also incorporate the CVT code from that Luc Verhaegen wrote for the soon-to-be-released Xorg 7.1.

[Technorati Tags: , , , ]

Monday Apr 17, 2006

The compiler bug that wasn't...

People hanging out in the #opensolaris IRC channel last week may have seen a discussion between myself, slowhog, and dprice as we tried to debug why gaim was hanging on slowhog's machine - it seemed to be the compiler had generated an impossible-to-exit infinite loop: [Note: IRC transcripts below have had some portions dropped and/or reordered to make the discussion easier to read]

<slowhog> dbx shows interesting code
<slowhog> 0xd208b301: _XFlushInt+0x00e5:  testl    %eax,%eax
<slowhog> (dbx) stepi
<slowhog> t@1 (l@1) stopped in _XFlushInt at 0xd208b303
<slowhog> 0xd208b303: _XFlushInt+0x00e7:  je       _XFlushInt+0xe5        [ 0xd208b301, .-2 ]
<slowhog> (dbx) stepi
<slowhog> t@1 (l@1) stopped in _XFlushInt at 0xd208b301
<slowhog> 0xd208b301: _XFlushInt+0x00e5:  testl    %eax,%eax
<slowhog> as you suggested, it's a loop
<slowhog> curious from where is the eax register got updated.

Unfortunately, I've never learned x86 assembly opcodes, and while I know the basic concepts from doing 6502 assembly on the Apple II years ago, and MIPS assembly in CS classes in college, so can mostly follow, I don't know what exactly is going on without a bit of help which Dan and Henry were able to provide for me.

<dprice_at_sun> alanc, it is asking if eax & eax != 0, basically
<alanc> so it's just checking to see if any bits in eax are set?
<alanc> i.e. eax != 0 ?
<slowhog> testl src, dest (an AND that does not change dest, only flags)
<dprice_at_sun> sorry.  so slowhog, the sequence is:  testl eax,eax;  je 
<dprice_at_sun> is that right, just those two?
<slowhog> it's like while (eax!=0)
<slowhog> yup
<dprice_at_sun> right except that eax is a register
<slowhog> exactly
<dprice_at_sun> we're not reloading it right?
<slowhog> not from the two lines of assembly
<dprice_at_sun> you often see this kind of thing when someone forgets a "volatile"
<slowhog> dprice_at_sun, that's what I suspect
<alanc> is this from Solaris 10?
<slowhog> on nv b35
<alanc> X software on Solaris 10 x86 was built with a Studio 10 build that didn't
         always honor volatile when it should
<alanc> Nevada switched to a newer build long ago though

So the first step is to narrow down where in the code this loop was being hit:

<alanc> that's probably in the code that's checking to see if the other thread
         holding the lock has released it yet
<alanc> sadly cc -S doesn't give helpfully inlined source on x86 like it does on
         SPARC to more easily match to the right spot
<slowhog> I am building my lib/app with Sun Studio 11
<slowhog> dprice_at_sun, gaim on gtk on gdk on XPending
<alanc> I don't see any loops in _XFlushInt with no bodies
<alanc> the closest is: while (dpy->flags & XlibDisplayWriting) {
<alanc>             ConditionWait(dpy, dpy->lock->writers);
<alanc>         }
<dprice_at_sun> alanc, I looked too...
<dprice_at_sun> hmm

<dprice_at_sun> I see the same loop in my disassembly of nv_36
<alanc> the code in question is xc/lib/X11/XlibInt.c

For those of you following along at home, you can't see the exact version of the code we were looking at, but it's similar to the X11R6.6 version you can see here.

<dprice_at_sun> so, this looks pretty weird to me
<dprice_at_sun> the loop comes after _XSmeAllocateHeapBuffer() in the assembly
<dprice_at_sun> So yeah, it does look like the loop alan mentioned.  line 814 of XlibInt.c
<dprice_at_sun> alanc, that sure doesn't look correct to my tired eyes.
<alanc> ConditionWait should be a function call though, not just jump back to the test
<dprice_at_sun> alanc, yes.  which raises a spectre....
<dprice_at_sun> on SPARC, you can clearly see the test, backwards branch loop happening.
<dprice_at_sun> on the binary bits I have on x86, I don't (yet) see it
<dprice_at_sun> actually, it does look there is a loop here:`_XFlushInt+0x102:   jne    -0x23    
<alanc> now the cc -S matches
<dprice_at_sun> the comparison to 0x40 is the check for XlibDisplayWriting ; the 
                 comparison with 1 is the test for XlibDisplayIOError
<alanc> .CGB7.195:
<alanc>         movl       1220(%esi),%eax
<alanc> .CGA1.173:
<alanc>         testl      %eax,%eax
<alanc>         je         .CGA1.173
<dprice_at_sun> it'd be nice to know what ConditionWait is expanding to, once the
                 CPP is done with it
<alanc> #define ConditionWait(d,c) if ((d)->lock) \\
<alanc>         (\*(d)->lock->condition_wait)(c, (d)->lock->mutex)
<dprice_at_sun> well I'm not seeing it do the sort of  'call   \*%eax' that you 
                 would need for this code to be working
<dprice_at_sun> there is one later, at`_XFlushInt+0xf1:    call   \*%ecx
<dprice_at_sun> but I think that is the subsequent
                       for (ext = dpy->flushes; ext; ext = ext->next_flush)  loop
<dprice_at_sun> maybe anyway
<alanc> with no -xO flags it gives code that doesn't appear to loop back on itself
<dprice_at_sun> shocker :)
<alanc> instead it's a much longer loop with a call in the middle
<alanc> I see studio 11 also nicely added line number comments to x86 -S output

We did a lot of comparing of the different code outputs with the different optimizer levels that I'll spare you from here (mainly because they were done via pages that have now expired and I didn't archive them).

<dprice_at_sun> alanc, although one wonders why we're not seeing this 
                 problem with lots of X apps?
<alanc> not lots of multithreaded X apps
<dprice_at_sun> I guess if while (dpy->flags & XlibDisplayWriting)  fails 
                 the first time through the loop, we never hit this
<alanc> or rather, not lots of X apps that initialize the threading support 
         so they can call X from multiple threads
<alanc> instead of just having only one thread ever call X

So I filed a bug with our compiler team. I provided the output from cc -E so they had a single source file to debug and didn't need our full build environment. They determined that turning off the conditional eliminator made the loop go away and sent it to the engineers who know that part of the optimizer best. Unfortunately, they were able to prove in a few days that it was not a compiler bug, but the compiler was simply being very clever. It saw this code (expanded after cpp preprocessing):

  12710         while (dpy->flags &  ( 1L << 6 )) {
  12711              if ( ( dpy ) -> lock )
  12712                 ( \* ( dpy ) -> lock -> condition_wait ) 
                            ( dpy -> lock -> writers , ( dpy ) -> lock -> mutex );
  12713         }

and translated it to (line numbers thanks to the improved cc -S output for x86 in the Sun Studio 11 compilers):

        movl       %ecx,%eax                            ;/ line : 12710
        andl       $64,%eax                             ;/ line : 12710
        je         .CGA3.175                            ;/ line : 12710
        xorl       %eax,%eax                            ;/ line : 12711
        movl       1220(%esi),%ecx                      ;/ line : 12710
        cmpl       %eax,%ecx                            ;/ line : 12711
        je         .CGA1.173                            ;/ line : 12711
        movl       72(%ecx),%eax                        ;/ line : 12712
        movl       52(%ecx),%edx                        ;/ line : 12712
        push       (%ecx)                               ;/ line : 12712
        push       %edx                                 ;/ line : 12712
        call       \*%eax                                ;/ line : 12712
        addl       $8,%esp                              ;/ line : 12712
        movl       148(%esi),%ecx                       ;/ line : 12710
        movl       %ecx,%eax                            ;/ line : 12710
        andl       $64,%eax                             ;/ line : 12710
        jne        .CGB7.195                            ;/ line : 12710

It turns out the conditional eliminator had noticed that neither dpy->flags nor dpy->lock were declared volatile, so they could not change in the running of this loop in the case in which the condition_wait function was not called, so it rewrote it effectively to:

 if (dpy->flags & ( 1L << 6 ))
   if (dpy->lock) {
    do {
     ( \* ( dpy ) -> lock -> condition_wait ) 
           ( dpy -> lock -> writers , ( dpy ) -> lock -> mutex );
    } while (dpy->flags &  ( 1L << 6 )) && (dpy->lock));
   } else {
     while (1) {}

After further digging, we found out that Henry was using a gaim that was making X11 calls from multiple threads without calling XInitThreads() first, which violates one of the two most important rules of X11 programming:

  1. Never call Xlib from a signal handler.
  2. Never call Xlib from multiple threads without calling XInitThreads() first.

(Most OS'es actually require XInitThreads() be the very first Xlib call made in the application - Solaris allows you to call it after other functions in some cases - see the Solaris Xlib threading changes I made available to Xorg last year.)

Code that violates these is known and expected to run off the rails and do unpredictable things, but simply going into a infinite loop that's impossible to exit is something we still want to avoid. Two simple changes got rid of this. The first was simply to declare the flags member of the Display struct to be volatile, since it clearly expects to be accessed from multiple threads. (Though since it's not using any sort of atomic test-and-set, I'm not sure how it was ever expected to be free of race conditions - fortunately, the Xlib internals are in the process of being replaced in the open source community by a new model which through the magic of grad student labor [Correction: undergrad student & professor labor - see comment #1 below] has been formally proven to handle threading correctly - see the XCB project site, especially their conference papers, for more details on this effort.)

The second change we brought in from X11R6.7 which instead of just looping when called when no locks were initialized instead adds a call to _XWaitForWritable there instead:

 	while (dpy->flags & XlibDisplayWriting) {
+	    if (dpy->lock) {
 	        ConditionWait(dpy, dpy->lock->writers);
+	    } else {
+		_XWaitForWritable (dpy, cv);
+	    }

Between these two changes, the compiler can no longer reduce down to an infinite loop, and while we can't promise correct, or even very reasonable, behaviour when you fail to call XInitThreads() in your multi-threaded X code, at least we won't chew up the output of an entire CPU when you do.

[Technorati Tags: , , Sun Studio, , ]

Tuesday Apr 11, 2006

The First SPARC OpenSolaris distro with Xorg

Only last week I had suggested that an interesting project for a community member would be to figure out what it would take to make Xorg run on OpenSolaris SPARC with a graphics device for which Xorg has drivers for other platforms, such as the PGX-series boards based on ATI Mach64 or the XVR-100 based on ATI Radeon 7000. I figured this would take a little while and a bit of work for someone to figure out. I guess I was wrong.

Martin Bochnig announced yesterday that not only had he done just that, he'd gone ahead and built a LiveCD distro using it - marTux. He's promised ISO's later this week and there's several people here waiting to see it now with high hopes.

[Technorati Tags: , , , , ]

Friday Mar 31, 2006

X Changes in Nevada Build 37

Unfortunately, all the changes in this build happened to be in the bits we haven't released yet (though I'm hoping to get our xscreensaver sources out sometime soon), so I can't point you at the changes in our just released source drop, but when the Solaris Express build 37 images come out, you should see these changes in the binaries.

6377194 XST extension wrapping makes the Composite and Damage wrapping not work
In the X server, many extensions do their work by replacing entries in tables of function pointers with their own functions, that do some work, then call the previous functions. Our colleagues in the Project Looking Glass team found that the XST extension (from the STSF project) had installed several of these function wrappers in a way that broke the similar wrappers from other extensions. Since we're in the process of removing STSF from the system, these wrappers were disabled to allow these other extensions to work.
6255133 SunRay: Xinerama: memory leak in Xsun after calling XCreatePixmap(3X11)
The Xinerama extension allows combining multiple graphics devices into one large virtual screen. One of the things it does to allow this is to make a separate copy of every pixmap in the X server for each underlying device, so that different cards can operate on it in the most efficient way for them or store it in their on-board memory. For Sun Ray systems though, where all devices are always the same, and all pixmaps are just stored in the main system RAM, this duplication wastes RAM and CPU time (since all operations have to be repeated for each copy), so we allowed an Xsun ddx module to notify the system that it can share copies. A bug crept in though, where this wasn't registered correctly with the list of resources to be freed when the client exited, so clients that exited without releasing their pixmaps caused Xsun to leak memory. (This is also being patched for Solaris 9 and 10 Xsun.)
6232241 NSCM login takes username twice
The never ending struggle to get xscreensaver's PAM conversation to play nicely with Sun Ray's Non-SmartCard Session Mobility PAM modules goes on. Fortunately, Mahmood has been working on this, so I don't understand this enough to explain it.
6388473 xscreensaver needs to be modified for Trusted JDS
As part of the work to create a Trusted JDS desktop for the Solaris Trusted Extensions, xscreensaver had to be modified to allow admins to enforce system security policies, including deciding whether or not users can disable the screen lock or change the lock timeout, when running on a system with the Trusted Extensions installed and enabled.

[Technorati Tags: , , , ]

X code released to OpenSolaris

For now just a quick copy of the announcement - I'll write more once I catch my breath...

The first code drop from the X Window System Consolidation has been posted to It's a snapshot of a subset of the Solaris X Consolidation code from partway through Nevada build 38.

Details on what's included and links to downloads & licenses can be found on the X Community Sources page.

Source is not yet available in the OpenSolaris Source Browser, but work is in progress on preparing that for availability sometime next week.

For more information, or to discuss the X Consolidation, join the X Community on OpenSolaris.Org.

[Technorati Tags: , , , ]

Monday Mar 20, 2006

X Changes in Nevada Build 36

Another two weeks, another list of fixes checked in. The one with the biggest share of attention is also the one with the smallest code change - two missing pairs of parentheses - four simple characters that closed one big security hole.

6387822 Wrong library path in xft.pc file
Simple fix to the pkg-config data file we ship for libXft2 so it produces the right library path flags for linking so that GNOME 2.14 builds correctly.
6383556 Problem in allocating pixmap
The last security fix in X servers added checks to both Xsun & Xorg to prevent pixmap allocations from overflowing. Unfortunately one of the checks in Xsun clamped down too far - preventing pixmaps with dimensions larger than 8192 instead of the intended 32k limit.
6390864 nevada removal of ddxSUNWdials
We bow our heads for SunButtons and SunDials - faithful servants of almost two decades, now sent to permanent retirement. The hardware for these hasn't been sold for several years now and the kernel driver for them was removed, so we had to remove the Xsun support as well. (The official end of support notice should appear in the Solaris 10 Update 2 release notes, warning of removal in the a future release - but we normally don't remove support in update releases, so users still attached to theirs can stay on Solaris 10 without fear.)

If you've never seen these they were additional input devices - SunButtons offered a big pad of extra buttons, like a jumbo set of keyboard function keys, and SunDials offered a bunch of knobs you could twist. These were accessed via the X Input Extension by software such as CAD programs for more efficient interaction with their features.

6368334 common postscript-derived font names are no longer recognized
An updated set of font aliases to fix some problems reported with the ones added in build 34.
6390453 SUNWxorg-mesa has broken links in snv nightly build for 2/24/2006
The script integrated into build 34 to make symlinks to either the nVidia or Mesa OpenGL libraries was failing to create the right links to the Mesa libraries in certain cases.
6395871 integrate Solaris Trusted Extensions to X Windows (Xsun)
6395892 integrate Solaris Trusted Extensions to X Windows (
Sun's previous Trusted Solaris product is being replaced for Solaris 10 with the Trusted Extensions to Solaris. Instead of a separate fork of the OS, it will instead run standard Solaris 10, but with additional modules loaded to provide the multi-level security features. For X, this means shipping a new library (libXtsol) and putting hooks into the X server that the XTSOL extension loadable modules delivered in the Trusted Extensions for Xsun & Xorg can use to implement their own security checks as needed. We'll be offering this back to the open source X.Org community in the near future under the standard MIT/X11 license.
6396593 [Xorg Bug 6213] local user DoS and arbitrary code execution as root [CVE-2006-0745]
See previous blog post.

[Technorati Tags: , , , ]

Security hole in Xorg 6.9/7.0

[CVE-2006-0745] X.Org Security Advisory: privilege escalation and DoS in X11R6.9, X11R7.0

X.Org announced this morning a security hole in the Xorg 6.9 & 7.0 releases that allows a local user to create or overwrite files as root or to run code as root. More details can be found in the X.Org Security Advisory.

This bug affects Solaris users who have installed Xorg 6.9 or 7.0, either on their own or from Sun's releases. Xorg 6.9 is included in Solaris 10 patches 118966-14 and later and in Nevada builds 28 and later, which have been released via the Solaris Express programs.

The fix for Solaris 10 is available in a preliminary T-patch from the SunSolve web site - it's the same we plan to release as the permanent fix, it just hasn't finished the QA cycles required for official release yet. See SunAlert 102252 for details and the links to the patch. The fix for Solaris Express was integrated into Nevada build 36, which should be out via the SX: Community Edition in a couple of weeks.

There's also a simple workaround you can apply now to make it impossible to exploit the bug - remove the setuid bit from the /usr/X11/bin/Xorg binary. X servers on x86 need root access for accessing the video hardware directly - but it only has to be setuid root if you want a non-root user to be able to start the X server directly, such as via the xinit program. Most Solaris installs that use X don't do this, but have a display manager such as gdm or dtlogin start X with a login screen. Since those programs run as root, they can start the X server with the needed privileges without having the Xorg binary be setuid root.

Behind the Hole: The Untold Story of this Bug

A couple of weeks ago, the CTO of Coverity sent mail to the X.Org Developers offering access to the results of a code scan of the X.Org code base by their Coverity Prevent code scanner (which is based on the Stanford Checker project). Their scan of the entire X11R6.9 code base found 1681 potential issues, so about a dozen of us have been working our way through the list, triaging the real bugs from the false alarms, and determining which need to be fixed.

While I was working on this one day, I got tired of looking at yet another memory leak (there are tons in programs like xset, xauth, xhost, etc. - but since the programs only run for less than a second before exiting, how much do you care?), and went to the menu to search by report type to see what other bugs it had found. One of the bug types was "BAD_COMPARE" which I hadn't see yet so I went to look at what it found. Someone had already triaged 3 of these as false alarms and 2 as actual bugs, so I went to look at one of the bugs. It showed (and this is a very cut down version of what the actual report looks like in the browser, displayed in the context of the full source file):

1378 	  /\* First the options that are only allowed for root \*/
Event func_conv: Suspicious implicit conversion to function pointer: "&geteuid != 0"; did you intend to call the function?
1379 if (getuid() == 0 || geteuid != 0)

While I remember looking at the code around here a couple of times during the Xorg 6.9 release cycle, I had never before noticed that the parentheses were missing from the geteuid call. I think my brain simply subconciously autocorrected and inserted the parentheses for me when I read it. Fortunately, the Coverity checker has no subconcious to fool it, and automated attention to detail, so it found what we hadn't seen. Since without the parentheses, the code is simply checking to see if the geteuid function in libc was loaded somewhere other than address 0 (which is pretty much guaranteed to be true), it was reporting it was safe to allow risky options for all users, and thus a security hole was born.

So far that's the only security hole we've found in the Coverity reports - but we're only a little over half way through triaging the reports so far. (Of the 1681 potential issues found, our developers have determined 688 are actual bugs compared to 191 false alarms. Memory leaks are the biggest category, with NULL pointer comparison issues probably second. 63 bugs are already marked as fixed in the coverity reports, and anyone watching the xorg-commit traffic the last couple of weeks has seen a number of those fixes going into CVS for inclusion in the upcoming Xorg 7.1 release.)

P.S. Congratulations to the team at Red Hat and the members of the Fedora community on the release of Fedora Core 5 today, with the Xorg 7.0 modular codebase included. I know having a release-day security advisory isn't how you wanted to celebrate the FC5 launch, but I hope you're finding the new Xorg modular release model is making it much easier to get the fix out for it.

[Technorati Tags: , , , , , ]


Engineer working on Oracle Solaris and with the X.Org open source community.


The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter


« July 2016