Sunday Aug 04, 2013

Some times it’s a book sprint, other times it’s a marathon

One of the areas the X.Org Foundation has been working on in recent years is trying to bring new developers on board. Programs like Google Summer of Code and The X.Org Endless Vacation of Code (EVoC) help with this somewhat, but we’ve also been trying to find ways to lower the barrier to entry, both so the students in those programs can learn more and so that other developers have an easier time joining us.

Some of our efforts here have been technological - one of the driving efforts of the conversions from Imake to automake and from CVS to git was to make use of tools developers would already be familiar and productive with from other projects. The Modularization project, which broke up X.Org from one giant tree into over 200 small ones, had the goal of making it possible to fix a bug in a single library or driver without having to download and build many megabytes of software & fonts that were not being changed.

Another area of effort has been around the X.Org documentation, as I’ve written about before. Several years ago, Bart Massey suggested we try organizing a Book Sprint to write a guide for new X.Org developers, similar to the sprint he’d participated in to write a Google Summer of Code Mentoring Guide and a matching GSoC Student Guide. The Board agreed to let him try organizing one, but finding a time to bring everyone together was challenging.

We finally ended up holding a first session as a virtual gathering over the internet one weekend in March 2012. Bart, Matt Dew, Keith Packard, Stéphane Marchesin, and I used Gobby and Mumble to agree on an outline and write several chapters, along with creating some accompanying diagrams using graphviz dot. Unfortunately, the combination of the work from home setting, in which people kept dropping in and out as their home life intervened, the small number of people, and lack of an experienced organizer made this not as productive as other book sprints have been for other projects.

After the first weekend we had drafts of about 7 chapters and a preface, and outlines of several more. We also had a large chunk of prewritten material on graphics drivers that Stéphane had been working on already to link with. Over the next few months, Bart edited and formatted the chapters we had, while we got a little more written, including recruiting additional authors such as Peter Hutterer.

We then gathered again, this time in person, hosted by Matthias Hopf at the Georg-Simon-Ohm-University in Nürnberg, Germany, over the two days prior to the 2012 X.Org Developer’s Conference. We had some additional people join us this time, including Martin Peres, Lucas Stach, Ben Skeggs, and probably more I’ve forgotten. At this session, Bart, Matt & I worked on polsihing the rough edges on the guide we’d been creating, while the others worked more on the driver guide that Stéphane had started. Bart figured out the tools needed to generate ebook formats of the guide, such as epub, and made a cover and pulled together an overall structure for the guide, and by the end of that sprint, we had something we could read on the Android ebook reader he'd brought with him.

And then, we waited. Bart tried to figure out how to make the setup maintainable and reproducible, and how to make it available to our users, but his day job as a University professor kept taking time away from that. Bart also gave a lightning talk on our book sprint experience at the Write the Docs conference in Portland earlier this year covering what worked and what we learned didn’t work in our virtual sprint, and asking for ideas or help on how to go forward.

Meanwhile, the burden of fighting the spammers on the X.Org & FreeDesktop.Org wikis had gotten overwhelming on the current wiki software, so the freedesktop sitewranglers evaluated different solutions, and after looking at how well ikiwiki had been working for the past few years on the XCB wiki decided to move the rest of the FreeDesktop hosted wikis to ikiwiki as well. One major advantage is that it let us use the existing freedesktop authentication infrastructure for wiki accounts instead of having to find different ways to let our users in while keeping the spammers out. Another benefit is that it uses Markdown and git to update the wiki, so we can easily push files in Markdown format to the wiki to publish them now.

In a shocking coincidence, the geeks who wrote the X.Org Developer's Guide also used Markdown & git to author it, so with just a very little conversion to make links work between sections, it was possible to publish the guide to the wiki, where you can find it now:

It’s not perfect, it’s not even fully finished, but it’s better than nothing, and since it’s now on a wiki, it’s even easier for others to fill in the missing pieces or improve the existing bits.

Stephane is still working on the companion guide to graphics drivers, covering the stack from DRM and KMS in the kernel up to Xorg & Mesa. He posted the PDF of the draft as it was at the end of the March book sprint, but not yet a version with the contributions added from the September followup.

When working on applications higher up the stack, not hacking on the graphics stack itself, we refer developers to the documentation for their toolkits, or to general guides to OpenGL programming, as those are beyond what we can document ourselves in X.Org.

And for other open source projects, if you’d like a chance to have a professionally organized, in person doc sprint for your project, applications are being taken until August 7 for 2-4 projects to hold sprints as part of the Google Summer of Code Doc Camp in October, including travel expenses for up to 5 participants from the sponsors.

Wednesday Nov 09, 2011

S11 X11: ye olde window system in today's new operating system

Today's the big release for Oracle Solaris 11, after 7 years of development. For me, the Solaris 11 release comes a little more than 11 years after I joined the X11 engineering team at what was then Sun, and finishes off some projects that were started all the way back then.

For instance, when I joined the X team, Sun was finishing off the removal of the old OpenWindows desktop, and we kept getting questions asking about the rest of the stuff being shipped in /usr/openwin, the directory that held both the OpenLook applications and the X Window System software. I wrote up an ARC case at the time to move the X software to /usr/X11, but there were various issues and higher priority work, so we didn't end up starting that move until near the end of the Solaris 10 development cycle several years later. Solaris 10 thus had a mix of the recently added Xorg server and related code delivered in /usr/X11, while most of the existing bits from Sun's proprietary fork of X11R6 were still in /usr/openwin.

During Solaris 11 development, we finished that move, and then jumped again, moving the programs directly into /usr/bin, following the general Solaris 11 strategy of using /usr/bin for most of the programs shipped with the OS, and using other directories, such as /usr/gnu/bin, /usr/xpg4/bin, /usr/sunos/bin, and /usr/ucb for conflicting alternate implementations of the programs shipped in /usr/bin, no longer as a way to segregate out various subsystems to allow the OS to better fit onto the 105Mb hard disks that shipped with Sun workstations back when /usr/openwin was created. However, if for some reason you wanted to build your own set of X binaries, you could put them in /usr/X11R7 (as I do for testing builds of the upstream git master repos), and then put that first in your $PATH, so nothing is really lost here.

The other major project that was started during Solaris 10 development and finished for Solaris 11 was replacing that old proprietary fork of X11R6, including the Xsun server, with the modernized, modularized, open source X11R7.* code base from the new X.Org, including the Xorg server. The final result, included in this Solaris 11 release, is based mostly on the X11R7.6 release, including recent additions such as the XCB API I blogged about last year, though we did include newer versions of modules that had upstream releases since the X11R7.6 katamari, such as Xorg server version 1.10.3.

That said, we do still apply some local patches, configuration options, and other changes, for things from just fitting into the Solaris man page style or adding support for Trusted Extensions labeled desktops. You can see all of those changes in our source repository, which is searchable and browsable via OpenGrok on src.opensolaris.org (or via hgweb on community mirrors such as openindiana.org) and available for anonymous hg cloning as well. That xnv-clone tree is now frozen, a permanent snapshot of the Solaris 11 sources, while we've created a new x-s11-update-clone tree for the Solaris 11 update releases now being developed to follow on from here.

Naturally, when your OS has 7 years between major release cycles, the hardware environment you run on greatly changes in the meantime as well, and as the layer that handles the graphics hardware, there have been changes due to that. Most of the SPARC graphics devices that were supported in Solaris 10 aren't any more, because the platforms they ran in are no longer supported - we still ship a couple SPARC drivers that are supported, the efb driver for the Sun XVR-50, XVR-100, and XVR-300 cards based on the ATI Radeon chipsets, and the astfb driver for the AST2100 remote Keyboard/Video/Mouse/Storage (rKVMS) chipset in the server ILOM devices. On the x86 side, the EOL of 32-bit platforms let us clear out a lot of the older x86 video device drivers for chipsets and cards you wouldn't find in x64 systems - of course, there's still many supported there, due to the wider variety of graphics hardware found in the x64 world, and even some recent updates, such as the addition of Kernel Mode Setting (KMS) support for Intel graphics up through the Sandy Bridge generation.

For those who followed the development as it happened, either via watching our open source code releases or using one of the many development builds and interim releases such as the various Solaris Express trains, much of this is old news to you. For those who didn't, or who want a refresher on the details, you can see last year's summary in my X11 changes in the 2010.11 release blog post. Once again, the detailed change logs for the X11 packages are available, though unfortunately, all the links in them to the bug reports are now broken, so browsing the hg history log is probably more informative.

Since that update, which covered up to the build 151 released as 2010.11, we've continued development and polishing to get this Solaris 11 release finished up. We added a couple more components, including the previously mentioned xcb libraries, the FreeGLUT library, and the Xdmx Distributed Multihead X server. We cleaned up documentation, including the addition of some docs for the Xserver DTrace provider in /usr/share/doc/Xserver/. The packaging was improved, clearing up errors and optimizing the builds to reduce unnecessary updates. A few old and rarely used components were dropped, including the rstart program for starting up X clients remotely (ssh X forwarding replaces this in a more secure fashion) and the xrx plugin for embedding X applications in a web browser page (which hasn't been kept up to date with the rapidly evolving browser environment). Because Solaris 11 only supports 64-bit systems, and most of the upstream X code was already 64-bit clean, the X servers and most of the X applications are now shipped as 64-bit builds, though the libraries of course are delivered in both 32-bit and 64-bit versions for binary compatibility with applications of each flavor. The Solaris auditing system can now record each attempt by a client to connect to the Xorg server and whether or not it succeeded, for sites which need that level of detail.

In total, we recorded 1512 change request id's during Solaris 11 development, from the time we forked the “Nevada” gate from the Solaris 10 release until the final code freeze for todays release - some were one line bug fixes, some were man page updates, some were minor RFE's and some were major projects, but in the end, the result is both very different (and hopefully much better) than what we started with, and yet, still contains the core X11 code base with 24 years of backwards compatibility in the core protocols and APIs.

Sunday Jan 23, 2011

X11R7.6 Documentation Improvements

Late last month (about three months late in fact, sorry about that), X.Org announced the release of X11R7.6. While the announcement, release notes, and changelog give details on all the changes that went into this release since the X11R7.5 release a year earlier, the one I worked on the most (aside from the work of producing and posting the modules to be released) was the modernization of the X documentation.

When the X.Org Foundation inherited stewardship of the X Window System, the documentation was in a wide variety of formats. The programs and most libraries included man pages in the traditional nroff format. The source tree also contained a large number of specifications of the protocols and libraries, and these were in a variety of formats, including troff, TeX, FrameMaker, and LinuxDoc. These were mostly optimized for print output and generated Postscript documents which were included in the releases since few people had all the tools necessary to process all of those, especially the commercial FrameMaker software. This also was a problem for developers who needed to update the docs, and either didn't know all the different formats or who didn't have tools like FrameMaker.

One of the early decisions of X.Org was that we wanted all our docs to be in open formats with open source toolchains. The format we decided to standardize on for the long form documents such as the specifications was DocBook, a open format that was already adopted by projects such as GNOME. Later, during the split of the monolithic X Window System, we decided that once the documents were converted, they would be moved to the modules that they documented, so that the documents would be updated and released in sync with the code. Over the following years, we made a little progress, converting a few documents here and there, but not making a serious dent.

In 2010 though, two volunteers really turned this around - Matt Dew converted the bulk of our specifications to DocBook/XML, and Gaetan Nadon integrated those into the modules and set up the autoconf macros and Makefile build rules to convert them to the various output formats in a standardized fashion. The xmlto tool is used as a front-end processor to drive backend tools including xsltproc and Apache FOP to generate output documents in text, html, PostScript, and/or PDF formats.

You can see the effects of this if you compare documents from the X11R7.5 release with some from the X11R7.6 release. For instance, in the html versions of the Xlib spec, not only has the formatting greatly improved from the 7.5 version to the 7.6 version, the new one also now has a hyperlinked table of contents and index, and many cross-reference links between individual sections. In the pdf version, the 7.5 version is a simple encapsulation of the postscript output that is at least nicely formatted since the original troff was designed for print output. In the 7.6 version, fop generated the output directly for PDF, and was able to produce output that took advantage of the additional features of PDF, such as including the same extended set of hyperlinks as the HTML version, and a set of "bookmarks" for quick navigation to sections listed in the table of contents.

If you're building the sources, there's a standardized set of configure flags to control the documentation tools used during the build, such as --with-fop to enable the use of fop to generate Postscript and PDF output or --without-fop to disable it. If you're installing prebuilt packages, you may find the documents in various formats provided in the module specific subdirectory of the documentation directory for your system, such as /usr/share/doc/libX11 for the documents provided with libX11. Even if the packager didn't preformat the html, text, or pdf for you, the default Makefiles install the xml versions there so you can format them when needed, or read online with a Docbook-viewing tool such as GNOME's Yelp.

While this is a huge step forward, we're not done yet. Many of the API docs still need to be converted from old-style K&R C prototypes to the ANSI C89 style used in the code base, and other formatting cleanups are needed. Matt is working on ways to not just have hyperlinked cross-references inside a single document, but across the documentation set. There's plenty of work to go around for additional people to help, such as improving the style-sheets, proofreading the documents to find more areas to fix, adding cross-references where they could be useful, so more help is always welcome. And of course, the huge pile of docs we have are almost exclusively developer focused - end user documentation is mainly limited to man pages, which aren't always as helpful as they could be, but we need user feedback to let us know the areas that need more help, since the developers don't have to rely on the man pages to figure out how to use the software, so don't notice the gaps.

But now at least we've got good momentum going, and I'm hopeful each future release will continue to show improvement.

Friday Nov 05, 2010

Is Wayland going to replace X?

The interwebs are full this morning of reports on Mark Shuttleworth’s announcement that someday he’d like to use Wayland instead of Xorg as Ubuntu’s primary display driver.

My favorite so far is Ryan Paul’s Ars Technica article for this dead-on estimate of the amount of effort it would take to completely replace X11:

Although the Linux ecosystem would benefit greatly from a lighter and more easily extensible alternative, a concerted effort to displace X11 on the Linux desktop hasn’t really emerged yet because the task of bringing drivers, third-party software, and all of the other layers of the stack into alignment with such a move would be prohibitively cumbersome. Like, in the sense that using only your toes to build a full-scale replica of the Statue of Liberty out of toothpicks is prohibitively cumbersome.

And that’s the important point - despite what some of the sensationalist headlines are saying, Ubuntu is not dropping X. They’re not even sure when it would be possible to start shipping Wayland - the announcement from Mark Shuttleworth was that this is the direction they want to go in hopes that a show of support will help get more people working on Wayland (which has mostly been a single person effort) in order to make it a usable solution in a year or more.

Even once they start shipping Wayland, since most applications will still be written for X, they’ll have to include an X server which now has to share the graphics devices with Wayland. (The Wayland site has architecture diagrams showing how this works.) Sure, their Unity desktop (the Ubuntu replacement for GNOME 3.0’s gnome-shell) will be written to Wayland, but more widespread adoption depends on how many other platforms also adopt Wayland, since then the creators of other applications will have to decide if its worth the effort of porting just for those distros when they can do nothing and continue to work on all X11 platforms, including those running Wayland.

One of the challenges in widespread Wayland adoption, especially on non-Linux kernels, is that Wayland requires a kernel Direct Rendering Module (DRM) with Kernel Modesetting Support (KMS) for every graphics device.

That’s especially challenging in Solaris, where even in the latest OpenSolaris and Solaris Express releases, we only have DRM drivers for Intel and ATI Radeon graphics (and the Radeon one currently doesn’t work, but that’s fixable).

That leaves these non-EOL graphics devices with no DRM/KMS support in Solaris:

  • AST (service processor/remote KVM in Sun servers)
  • ATI Rage and Mach64
  • Cirrus
  • Matrox
  • Nvidia
  • VIA Unichrome
  • Trident
  • VESA (fallback for hardware without specific driver support)
  • VirtualBox
  • VMWare
  • Sun XVR-50/100/300 (OEM ATI Radeon series, but with separate driver)
  • Sun XVR-2500
  • Sun Ray

Of course, that’s the list of devices Oracle is supporting in Solaris - other distros/forks from the OpenSolaris code base, such as OpenIndiana or Belenix, may choose to support more or less, but I’ve not yet seen any sign of them porting DRM drivers on their own.

For some of those, support could be provided by porting the existing open source dual-BSD/GPL-licensed DRM code - others require creating drivers where there are none.

Nvidia appears on their list since we ship their driver, which has their own kernel/driver architecture instead of DRI. Getting DRI/KMS support for nvidia either requires them to rearchitect their driver or moving from their driver to the open source / reverse engineered Nouveau project.

From Owain Ainsworth’s talk at this year’s X Developer Summit, we know that work is in progress to port KMS support to the BSD DRI, but again, that’s not yet done, though the BSDs do have a lot more of the device-specific DRM modules already ported than Solaris did.

So Xorg is in little danger of disappearing overnight - our desktop architecture continues to evolve, as it has over many years, and Wayland may play an increasingly larger role in it, or may end up an interesting side note, like Xgl before it, but either way it will be an evolutionary process, not a big bang flag day sudden change.

Wednesday Feb 18, 2009

Xorg 1.5.3 known issues in OpenSolaris 2009.06 & Solaris Express build 107

As Phoronix recently noticed, we upgraded the Xorg server from 1.3 to 1.5.3 in Nevada build 107 (currently available via Solaris Express Community Edition (SXCE) Install ISO’s and OpenSolaris IPS package updates in the /dev repo).

When we integrated, I sent out a heads up message with information about the driver compatibility changes and issues we knew about at that time.

So far it’s gone mostly smooth, thanks to the help of the people who tested the pre-integration builds I posted both internally and on the X community on opensolaris.org, but there’s a few issues that have hit some people so far, most with workarounds you can use to get past them:

6801598 Xorg 1.5 ignores kernel keyboard layout setting, always uses "us"

Fixed in build 109. Xorg on Solaris has always shipped with a patch to query the kernel for the console keyboard layout (which in turn either gets set automatically by the keyboard if, like Sun keyboards, it provides the layout in the optional USB HID layout identifier, or gets set by querying the user on initial installation), but the code we apply this patch to moved from the Xorg binary to the kbd_drv.so binary in Xorg 1.5 and our patch didn’t get migrated correctly. We didn’t notice in testing since our testing was done with input devices provided by HAL, which was handling this for us, but when that didn’t integrate as expected, this bug was exposed.

Workaround: Create an /etc/X11/xorg.conf file with a keyboard section specifying your desired layout, such as this one for a British layout:

Section "InputDevice"
	Identifier		"Keyboard0"
	Driver			"kbd"
	Option "XkbRules"	"xorg"
	Option "XkbModel"	"pc105"
	Option "XkbLayout"	"gb"
EndSection

6791361 SUNWxorg-xkb should deliver "base" rules

If you do need to set a localized keyboard layout, you may notice the default XKB rules file upstream changed from "xorg" (which is currently shipped in our packages) to "base" (which isn’t yet, though we’ve asked the localization teams who build our XKB packages to add it).

Workaround: Include an XkbRules option line in your keyboard settings as shown in the example above, or make a symlink from /usr/X11/lib/X11/xkb/rules/base to the xorg file in that directory. The fix for the above bug in build 109 should also set the default used by our Xorg builds back to "xorg" for now.

6801386 Xorg core dumps on startup if hald not running in snv_107

Fixed in build 109. Yet another instance of our old friend 6724478 libc printf should not SEGV when passed NULL for %s format. This was a design decision made in SVR4 and Solaris 2.0 many years ago, to force programmers to catch NULL pointer misuse - but since Linux & the BSD’s allow it, today it’s mostly become a source of pain in porting code that while technically incorrect, works on those platforms, so the decision was made last year to change Solaris libc to match. Unfortunately, that change is still undergoing standards conformance testing, so hasn’t integrated yet, and we still have to check for NULL first.

Workaround: Make sure hald starts before Xorg does. This shouldn’t be a problem for gdm users (default in OpenSolaris), since the gdm SMF service lists the hal service as a dependency that needs to start first, but may cause problems for dtlogin, xinit or startx users.

6806763 Xorg 1.5 doesn’t start if xorg.conf contains RgbPath entry

Fixed in 109, and in upstream head. This entry in the Files section of xorg.conf was obsoleted upstream, but became a parse error instead of just being ignored, which breaks people who upgraded a system with a working xorg.conf that happened to have this line.

Workaround: Remove RgbPath lines from /etc/X11/xorg.conf.

6799573 Metacity not starting on TX Xorg 1.5

Fixed in 108. The XACE framework used by extensions such as Xtsol & Xselinux added new permission types to check in the upstream release, but Xtsol doesn’t handle all of them yet. In this case, Metacity’s constant checking of round-trip time by appending to an existing property failed because Xtsol was only handling the WriteAccess check and not the BlendAccess check.

I don’t know of a workaround for this other than not running the desktop in multi-level mode, so users of the labeled/multi-level security desktop in Trusted Extensions may want to wait to upgrade until then.

6798452 X Server will not start on x86 systems with multiple graphics devices

Not yet fixed, either in our builds or upstream, but has a simple workaround of creating an xorg.conf file listing the devices you want to use. This may affect people with a single card if your motherboard has on-board graphics as well.

6797940 Can’t do gui installs on x86 systems with Nevada 107/Nevada 108

We haven’t yet tracked down this issue, which only affects the SXCE installer on x86/x64 platforms. Workarounds include using the OpenSolaris LiveCD installer or the text only mode of the SXCE installer instead. After installation, the installed Xorg works fine (modulo the above issues).

6686 SUNWefb[w] should be part of slim_install

Fixed in 108. The SPARC drivers added in OpenSolaris build 107 for Xorg on XVR-50, XVR-100, and XVR-300 graphics cards aren’t included in the default install. You can add them after the install by doing pfexec pkg install SUNWefb SUNWefbw and then rebooting. (These drivers are only available in OpenSolaris installs, not SXCE ones, since they conflict with the Xsun drivers for these cards included in SXCE.)

Saturday Jun 14, 2008

June 11, 2008 X Server security advisories

On June 11, iDefense & the X.Org Foundation released security advisories for a set of issues in extension protocol parsing code in the open source X server common code base that iDefense discovered and X.Org fixed.

Their advisories/reports are at:

Sun has released a Security Sun Alert for the X server versions in Solaris 8, 9, 10 and OpenSolaris 2008.05 at:

Preliminary T-patches are available for Solaris 8, 9, and 10 from the locations shown in the Sun Alert - these are not fully tested yet (hence the "T" in T-patch).

The fix for these issues has integrated into the X gate for Nevada in Nevada build 92, so users of SXCE or SXDE will get the fixes by upgrading to SXCE build 92 when it becomes available (probably in 3-4 weeks, though the first week of July is traditionally a holiday week in Sun's US offices, so may affect availability).

Fixes for OpenSolaris 2008.05 users following the development build trains will be available when the Nevada 92 packages are pushed to the pkg.opensolaris.org repo (also probably in about 3 weeks from now).

Fixes are planned for OpenSolaris 2008.05 users staying on the stable branch (i.e. nv_86 equivalent), but I do not have information yet on how or when those will be available.

Fixes for users building X from the OpenSolaris sources are currently available in the Mercurial repository of the FOX project in open-src/xserver/xorg/6683567.patch.

For users of all OS versions, the best defenses against this class of attacks is to never, ever, ever run “xhost +”, and if possible, to run X with incoming TCP connections disabled, since if the attacker can't connect to your X server in the first place, they can't cause the X server to parse the protocol stream incorrectly. This is not a complete defense, as anyone who can connect to the Xserver can still exploit it, so if you're in a situation where the X users don't have root access it won't protect you from them, but it is a strong first line of defense against attacks from other machines on the network.

Releases based on the Solaris Nevada train (including OpenSolaris & Solaris Express), default to “Secure by Default” mode, which disables incoming TCP connections to the X server. Current Solaris 10 releases offer to set the Secure by Default mode at install time. On both Solaris 10 & Nevada, the netservices command may be used to change the Secure by Default settings for all services, or the svccfg command may be used to disable listening for TCP connections for just X by running:

svccfg -s svc:/application/x11/x11-server setprop options/tcp_listen=false

and then restarting the X server (logout of your desktop session and log back in).

On older releases, the “-nolisten tcp” flag may be appended to the X server command line in /etc/dt/config/Xservers (copied from /usr/dt/config/Xservers if it doesn't exist) or in whatever other method is being used to start the X server.

See the Sun Alert for other prevention methods, such as disabling the vulnerable extensions if your applications can run effectively without them.

Thursday Oct 11, 2007

Stupid nv_74 cursor tricks

For those who have already grabbed Nevada build 74 from the Solaris Express Community Edition downloads, you can try out the new cursor code available with the integration of libXcursor (which most XFree86 and Xorg platforms have had for a while).

If your graphics card supports the Render extension & 32-bit alpha cursors, (i.e. most x86 graphics, but not Sun Rays nor SPARC graphics yet - I've mainly used it on nvidia cards and my laptop with a ATI Radeon chipset), you can see them in action by doing:

% su -
# mkdir -p /etc/dt/config/C/
# cat > /etc/dt/config/C/styleModern
#include "/usr/dt/config/C/styleModern"
Xcursor.theme: whiteglass
\^D
# exit

% echo 'Xcursor.theme: whiteglass' >> ~/.Xresources

and then logout & log back in.

Change 'whiteglass' to 'redglass' in the above if you want something that stands out more.

Thursday Sep 27, 2007

Happy Belated Birthday X11!

Just noticed that according to Wikipedia's X Window System Release History, X11 was first released on September 15, 1987 - 20 years old and I didn't even remember to get a birthday card...

Friday Jun 22, 2007

OpenSolaris and X: Year 2

open(2) logo

I'm a bit late this year - it's now a year and a week since I posted One Year of OpenSolaris and X, and it's been a busy time.

Unfortunately, I can't say that we've released lots more code than in our first year - there's still many client applications and libraries that need to be migrated to the X.Org versions - but we have moved the source base for what we've released from the Imake-built X11R6.9 monolithic source tree to source modules from the X11R7.2 release. With the 7.2 move, we also grew the set of servers delivered from the current X.Org sources from just a 32-bit x86 Xorg by adding 64-bit Xorg servers on both SPARC and x64, as well as the Xephyr nested server and Xvfb virtual frame buffer on both platforms.

Sun's China team integrated DRI support for Intel i845 through i945 graphics, with i965 coming soon. We've also added libraries from X.Org that we hadn't shipped before, including libXcomposite and libXinerama.

Work continues on converting the rest, and may pick up with some community members joining in. The OpenSolaris X community has been discussing a proposal to create a common source repository merging the X efforts of the Solaris Express, Belenix, and marTux distributions. Martin Bochnig has been working for a while on marTux, an OpenSolaris distro for SPARC - and the only one with open source SPARC Xorg drivers he's put a lot of effort into making work well on OpenSolaris. And in April, Moinak Ghosh announced he'd taken our code drops and packaging and filled in the rest of the X11R7.2 sources. By merging these, we hope to keep all distributions better aligned and make it easier for us to work together on the missing pieces.

What else are we working on? Well, like last year, the OpenSolaris suspend-and-resume team is still hard at work making their code as rock-solid as they can, including the graphics support in Xorg. We're looking at the best way to integrate Xvnc into our code base in support of the OpenSolaris Xen work. And of course, we've been playing with the projects now known as “Compiz Fusion”, using the builds from Erwann in the Desktop group, so that maybe by the time I write next year's entry, you will have a Solaris desktop option that looks a bit like this.

Wednesday Jun 13, 2007

Removing XInitThreads from Totem

Bastien, you're not alone. If the JDS team complained to even Sun's X11 team about bugs after they applied that Totem patch, we'd tell them they're on their own and we'd never support an application that called X from multiple threads without calling XInitThreads first.

The Solaris Xlib is thread safe (though probably with a few bugs in corner cases still) based on the same X11R6 multi-thread code as everyone else, though we're currently working to reconcile the fixes from our fork with the current X.Org code as part of our ongoing project to un-fork Solaris X11 and bring it back to the X.Org code base. We're also looking at XCB in the future for a cleaner thread-safety model as well.

Update: As noted by trisk in the comments, this seems to have been caused by known Solaris libXi bug 6551484 - the engineer who had removed XInitThreads() from totem tried the test binary produced by the engineer working on the libXi bug and found it solved the hang problem she was seeing.

Monday Jun 11, 2007

The Irregular X11 DTrace Companion: Issue 2

In response to a Solaris 10 7/07 beta tester question last week, I updated the Xserver provider for DTrace web page to point out that besides the original method of patching the Xorg sources and rebuilding, pre-built binaries are now included directly in Solaris...

Getting an X server with DTrace probes built in

The Xserver dtrace providers are now included in the regular X server binaries (including Xorg, Xsun, Xvfb, Xephyr, Xnest, & Xprt) in recent Solaris releases, including:

For those who still want to build from source, the provider is integrated into the X.Org git master repository for the Xserver 1.4 release, planned to be released in August 2007 with X11R7.3, and into the OpenSolaris X code base for Nevada builds 53 and later.

Thursday May 17, 2007

The Irregular X11 DTrace Companion: Issue 1

All the cool kids are doing unscheduled update newsletters now, and I had a collection of X11 Dtrace things to post about piling up, so I figured I'd give it a whirl. (BTW, Nouveau guys - I'd read yours more if you had an RSS feed - having to go check the wiki on an irregular basis doesn't work so well.)

Doing newsletters on a regular basis is hard, as Glynn has learned, so don't expect updates too often...

Sample script for tracing events

Jeremy asked recently for an example DTrace script using the event probes in the Xserver DTrace provider. I had lost the ones I wrote when developing/testing that probe, but dug one out of my e-mail that was written by DTrace Master Guru Bryan Cantrill and posted it as ButtonPress-events.d.

He wrote this script when he'd upgraded to Nevada build 58 and found shift-click was no longer working. Using the script he was able to show that the Xserver was sending ButtonPress events without the ShiftMask set. I wrote a corresponding script to decode the VUID events the X server was getting from the kernel mouse and keyboard drivers to determine the keyboard wasn't giving us the shift key pressed events until another key was pressed, which wasn't happening for shift-clicks with the mouse. That script can be seen in the bug report I filed with the kernel keyboard team as Solaris bug 6526932.

Using DTrace to win a race

A bug was filed with X.Org recently stating “iceauth can dump a core in auth_initialize() if a signal is caught before iceauth_filename has been malloced.” As I was fixing bugs in iceauth anyway, I figured I'd look at it to see if I could roll in a fix to the new iceauth release I'd be making for those. Fortunately, iceauth is a small program, and I could see where I thought the race condition lied between the main code and the signal handler. Testing any race condition has always been a pain in the past, since if you don't get the timing just right you don't know if you really fixed it, or just changed the timing enough to not hit it.

So I pulled up the DTrace manual and DTrace wiki and created a one-liner to force the race condition by sending the signal when I knew it was in the right spot:

 # dtrace -w -c ./iceauth -n 'pid$target::IceLockAuthFile:return { raise(1); }'

For those who haven't memorized the DTrace manual, this can be broken down as:

  • -w - allows “destructive” actions - i.e. those that change the running of the program, and don't just monitor it - in this case, allows sending a signal to the process
  • -c ./iceauth - specifies the command to run with this script, and puts the process id of the command in $target in the script
  • pid$target::IceLockAuthFile:return - sets the probe we're looking for to be the return from any call to the function IceLockAuthFile in the process matching the pid $target
  • { raise(1); } - sets the action to be run when that probe is hit to send signal 1 (aka SIGHUP) to the target process

Ran that and a few seconds later had a nice core file to verify where the race was. Turned out I was right about where the signal had to come in for the race to cause a crash, but was slightly off about the point in the signal handler where it was crashing. The fix was easy, and thanks to DTrace, easy to verify, and is now in X.Org's git repository.

DTrace Wiki: Topics: X

After last month's Silicon Valley OpenSolaris User Group meeting, Brendan asked if I'd work on the “X” topic on the DTrace Wiki. I agreed to, and have put up a small placeholder while I figure out what to flesh it out with - if you have any suggestions or requests for what you'ld like to see in a short introductory article to using DTrace to probe X, especially using, but not limited to, the Xserver DTrace provider, send me e-mail or leave a comment here.

Monday Apr 16, 2007

X11 API's for querying multi-head screen layout

I should probably stick this in the X.Org wiki or OpenSolaris/GenUnix wiki, but I'm being lazy and just summarizing to here now from an e-mail thread last week on one of the Sun internal lists...

There are now 4 sets of API's to query the layout of display screens in a logical X screen when using a setup such as Xinerama, TwinView, or MergedFB:

  1. the original X11R6.4 XPanoramiX\* API
  2. Sun's Xinerama API
  3. XFree86's Xinerama API
  4. Xrandr 1.2

The first is pretty much universally available, but also universally deprecated. The functions for it are defined in libXinerama on Linux, libXext on Solaris, but I don't think we've ever shipped the header for it on Solaris, since it was deprecated long ago.

The second Sun created and proposed to the old X.Org industry consortium to become the standard, but they did not adopt it, crafted a replacement in committee that was universally incompatible with everything and never adopted that either. It's what the <X11/extensions/xinerama.h> header in Solaris represents, and was never documented, but you can find a description of the API in PSARC 2000/036. The functions from it are found in Solaris libXext, but nowhere else in the world that I know of.

The third, aka libXinerama, is available on most XFree86 & Xorg based systems, and finally came to Solaris in nv_62 and soon in Solaris 10 patches & the next update release. It uses the <X11/extensions/Xinerama.h> (note the capitalization compared to the other) header and is documented in the man page we created for the Solaris integration and contributed back to X.Org.

The fourth is a very recent development, having been released after X11R7.2's release in February. X.Org is currently in the process of releasing version 1.3 of the Xorg server package (release candidate 5 is out now) - this will be the first Xorg server feature release not tied to one of the whole X Window System releases. (X11R7.2 had Xorg server 1.2, X11R7.3 is expected to have Xorg server 1.4.) The xf86-video-intel driver 2.0 release candidate 4 is also out, it is the first driver to support the new Xrandr 1.2 additions, though work is in progress on other drivers, including the Radeon and nvidia drivers. We don't have a definite roadmap for including this in Solaris yet - we'll probably do it once the X.Org releases are finished and we're passed the upcoming milestone of the second Solaris Express Developer Edition release in May.

Sunday Feb 04, 2007

Moving x86 video drivers into the kernel

Since Keith's recent talk on X at linux.conf.au, I've been asked a few times and seen discussion elsewhere (such as in the comments on the linked article) what Sun thinks about moving various portions of the Xorg video drivers from the current userspace shared objects into true kernel drivers, as most people assume this means Linux kernel drivers and reducing the portability of Xorg to non-Linux platforms such as Solaris/OpenSolaris and the BSD's.

While I'm of course too far down the food chain to claim to speak officially for Sun, as the lead engineer for the Xorg integration to Solaris and someone who has talked to a number of the other Solaris engineers about it, I'd say the Solaris engineering teams agree having in-kernel drivers for video cards is something we want and need.

On Solaris, we've long had in kernel graphics drivers for all our SPARC frame buffers, since that was a small and controlled set, and writing the driver for it was simply part of the cost of producing the card. On the x86 side though, we had for years the same model as most other x86 Unix-like OS'es - a generic set of kernel modules, and all device-specific logic in the X server. The Solaris x86 kernel modules were vgatext and xsvc. vgatext provides the text console, using standard/ancient VGA interfaces which virtually all PC graphics boards support. xsvc (which was closed source due to legal encumberances, but has recently been replaced with a newly written open source version by the Solaris Xen project team) provides the mappings to the PCI bus that allow Xorg to probe the bus hardware to find devices, and then map in the video card once it's found.

Recently, new kernel modules have started appearing in Solaris - the first two for 3-D rendering acceleration, nvidia's driver and the DRI driver for Intel graphics. Two more are coming soon from the Suspend-and-Resume project, to provide the hooks to save and restore necessary state across system suspends - they'll initially support the ATI Rage XL from the Sun Ultra 20 and ATI ES-1000 in the Ultra 20 M2. (Dave and Jay gave a talk on their implementation at last years' X Developer's Conference.)

Additionally, our security guys have been pushing for more kernel frame buffer drivers to reduce our need for running Xorg as root. They'd also like to find a way to close the now infamous Ring 0 security weakness. (We've invited them to drop in to this week's X.Org Developer's Conference to talk about this - they're planning to show up Thursday morning for the Secure X talk, so we may try to have a breakout session on this topic at that time to discuss this.)

Now, we still can't write drivers ourselves for every x86 video card on the market, which is one of the big reasons we chose to move from Xsun to Xorg on Solaris, so we'd still prefer if a model could be found that allowed sharing drivers between Solaris, Linux, and BSD (which means agreeing on both licensing and interfaces with the kernel), similar to what we already have in DRI. If we can work out a model like that, then I think Sun would support it - either way, we're going to have to have more graphics driver functionality in the kernel, and sharing would be better than going our own way.

[Technorati Tags: , , , , , , ]

Thursday Nov 09, 2006

X Changes in Nevada Builds 50 - 52

I posted our OpenSolaris code drop Tuesday for Nevada Build 52. The last build published was build 49, so this release includes all the changes from builds 50, 51, and 52. Yet out of the 27 changes in those three builds, almost all the questions I get are about just one of them, which can be summarized in 3 little letters:

DRI

This build contains the first release of DRI support in Xorg on Solaris. It only supports 3-D hardware acceleration on the Intel i845 through i915 graphics chipsets for this first release, and requires the kernel DRI support included in the Solaris Nevada Build 51 and later kernels, but is a huge first step. Both the kernel and Xorg/Mesa sides are now published in source form on OpenSolaris.org as well, so those who want to see how it was done can take a look, and the truly brave could start thinking about porting other drivers.

If you want pre-built binaries, you'll have to wait another week or two for the Build 52 ISO's to be published as Solaris Express: Community Release. More information on DRI in Solaris can be found in the DRI heads up message from the DRI team. You can browse the kernel side of the sources in the OpenSolaris source browser in these directories (sorry X source isn't on the source browser yet):

One word of warning though - the kernel driver may cause panics on some i855 machines - see bug 6487609 for details.

But wait! As if that wasn't enough, there's still 26 other changes included in this valuable package! And they're not available in stores! For the full list, see the changelog for builds 50-52. Highlights:

6465198 xdpyinfo shows incorrect extensions list on snv11_46
Dynamically loaded extensions like OpenGL have been failing to load in Xsun in the last few Nevada builds - this fixes that.
6423858 [Xorg bug 5898] file creation race condition in Xsession
A small security issue in the default Xsession if you use xdm instead of gdm2 or dtlogin. Patches for Solaris 8, 9, and 10 are coming for this fix.
6475968 Desegregation of X11 External Libraries [PSARC/2006/557]
When we added libXrender, libXdamage, libXfixes, and libXRes to Solaris, the Sun policy called for keeping libraries which we didn't guarantee ABI stability of in a separate path so users didn't “accidentally” depend on them and create applications not covered by the Solaris Binary Compatibility Guarantee. Sun policy has since changed to prefer usability and to not require users to jump through hoops to build software using them, so we've adapted to the new policy by moving these libraries from /usr/openwin/sfw/lib to /usr/X11/lib and putting links to them in /usr/lib.
6477006 keycode 22 reports keysym 0/NoSymbol on Sun Type 6 Unix keyboard
The Unix layouts of Sun's keyboards have a blank key between Help and F1, in the slot Esc uses on the PC layout. (See these diagrams for comparison.) This allows using the same hardware with just different labels on the keys. Unfortunately, this key has always just been assigned a to keycode 0/keysym "NoSymbol", which is pretty useless. Some intrepid users have manually assigned it to a keycode so that it can be assigned to a hotkey in JDS or CDE. To make it easier for others, the blank key now defaults to a keysym name of (wait for it...) “Blank.” Yes, that's right, X11 defines a keysym XK_Blank — and since it doesn't define XK_Any for those who long ago decided it was the mythical key referenced in Press Any Key to Continue, that seemed the best choice. Sadly, only the old serial keyboards send a keycode for this key – the USB ones send nothing up to the X server, so this doesn't help there.
6476476 Xorg modularization: libXfixes
6477401 Xorg modularization: SUNWxorg-client*
We've converted to the Xorg 7.x modular versions of the libraries we and applications we used to get from the X11R6.9 source tree, including libXrandr, libXv, libXvMC, libXxf86misc, libXxf86vm, libxkbfile, xgamma,, xrandr, xvidtune, xvinfo. We've also updated libXfixes, which was previously brought in from the X11R6.8 source to the modular release of libXfixes.

[Technorati Tags: , , , ]

About

Engineer working on Oracle Solaris and with the X.Org open source community.

Disclaimer

The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today