Recently I had the pleasure of attending the recent O'Reilly Euro OSCON in Brussels, Belgium. I though that I might share some of this with the masses...
As you may, or may not, know I work in the JDS group, so have a particular focus on the desktop, so I apologise in advance for my comments being directed as such.
Overall the conference was good. While the conference itself did appear to have a Web 2.0 focus, the OpenSolaris.org booth got a lot of visitors mainly asking about things like ZFS, SMF, Zones, DTrace. Others were curious about the difference between Solaris and Open Solaris, and when informed were then interested in looking at the Live CDs we had - we managed to give away over 60 of the DVD Starter Kits with the Live DVD (several distros) and the Install DVD for Solaris, Nexenta as well as the Open Solaris sources and the compilers. Actually would have been good to have more information about the PPC port for all those MacOS/X users...
From a desktop point of view, there wasn't too much interest without some sizzle to demonstrate - as far as people are concerned it's still just GNOME which is on every other Linux distro. The majority of laptops I saw were Apple's, the others varied, but many were running Ubuntu with h/w acceleration enabled. So it's a tough audience from this respect. I'm looking forward to when DRI is available so we can show we are at least as good as others, and then have the opportunity to better it!
The speaker quality varied, but on the whole were quite good. There were many good talks, the details are below, and although many were Web 2.0 focused, I think a lot of what they said still applies to the desktop.
Interestingly Open Data was a hot topic - i.e. the person who owns the data can essentially force some lock-in - hence there is a call for more data to be released under more open/free licenses. This is an area people seem to expect a large amount of growth - as well as uses of this data in "Mash-ups" (i.e. the bringing together of various data sources to give a new view on things).
Louis Suarez-Potts (Collab.Net - hosts of OpenOffice.org) gave an interesting talk about his experiences with the OpenOffice.org open source community which I think we could learn a few things from w.r.t. Open Solaris.
Other interesting points were Usability, Xgl (Open GL desktop), Jabber (IM, XML streaming), Ajax (improved user experience on the web) and Gnash (an open source Flash implementation). Day 1 (Monday)
Managed to attend two tutorial sessions today, one on Jabber and one on Ajax. I found both to be very interesting:Jabber
For those that don't know, Jabber is another name for RFC 3920, 3921, and others. It's main use today is for Instant Messaging, but there is the possibility of so much more. Ajax
It's essentially Real-Time messaging, that routes XML from Point A to B. It uses federated addressing (similar to e-mail), has built-in presence, strong security and, to date, no spam. It also uses UTF-8 encoding for all it's text, SSL/TLS for channel encryption and SASL for authentication (allowing use of certificates, Kerberos, etc.).
As mentioned, it currently is mainly for IM applications, but has other possible uses like - Content Syndication, White-boarding (using SVG), Work-flow processing, etc. - basically it can be used for sending snippets of XML. It is part of the architecture of the FreeDesktop.org Telepathy project where it is used to maintain presence information and in the connection management.
It can even be found in some Wireless APs where it is used for the handing over of connections, etc. There are other similar uses in the management of h/w.
Jabber is at the heart of Google Talk, which has been the source of several enhancements to the protocol. Enhancements for the Jabber (XMPP) protocol - are all managed via the Jabber Foundations "Jabber Enhancement Proposal" (JEP) process.
While Jabber is not meant for streaming large data volumes (e.g. Video / VoIP) it is used in the negotiation phase of setting up such connections. Such a project is called Jingle, which is JEP-0166.
Some suggested uses, which might be useful for us to look at, are in the collaboration space - such as Inkscape (for the SVG white boarding) and Thunderbird integration (presence information is monitored and Thunderbird can "see" that the sender of an e-mail, or people listed on a thread, are currently on-line and could initiate a chat. We certainly should take a look at Telepathy too.
There are lots of clients for Jabber, GAIM is one we have in JDS. There is also a client which supports only Jabber, and was used for the demos, called Gajim. Gajim appears to have more complete support for the Jabber spec than GAIM - this is an area which it might be worth looking into improving.
Day 2 (Tuesday)Open Source 2.0.
Contrary to popular use of the term, Ajax is not meant to be about the technologies involved in it's actually meant to be about the presentation of the data to the user and the interaction with the user, essentially the user's perception of the UI. Ajax is an approach to interaction that eschews the need for full-page refreshes.
I found that although this talk wasn't specifically targeted at the GUI/Desktop platform, much of what was being said was relevant to the complete user experience.
Traditional web applications tend to present the user with a poor user experience. As things like Google Mail have shown to the masses, it is possible for the user experience to be greatly improved.
Unfortunately many old (bad) ideas are being sold under the moniker of Ajax - CSS doing "roll-over" (changing the image for instance) is not Ajax.
The presenter kept speaking about the fact that coders should not be the people to design the interface ("Friends don't let Friends Design [UI] and Develop"). The main reason being that the UI designer needs to be thinking from the perspective of the user and not that of the system. At a minimum, if a coder needs to design the UI they should do nothing else that day except focus on the user's perspective.
Interaction design can be seen as a "Strategy" - it communicates what the system _is_ and _does_. Interaction designers are paid to care about the users and they should negotiate with developers. Ajax can be then seen as a "Tactic" which would allow you to reach Interaction Design goals better and faster than before, without goals tactics are trivial. There is a constant battle between the System Model and the User Model - the users, and their expectations _are_ the system until they hit a wall.
Some Interaction Design Principles "borrowed" from Jeff Veen are [and I think we should remember this in our work on JDS]:
- Discoverability : The user should be able to discover the boundaries by feeling around.
- Recoverability : We should strive to make errors by the user "cheap", in that they are not seriously penalised by a mistake [e.g. closing a tab in gnome-terminal by mistake - it should be possible to say "oops, shouldn't have done that" and let the user learn/recover].
- Context : Provide information that is relative to the user's current "location" in a process. Contextual information can increase the utility of the interfaces.
- Feedback : Provide visual stimuli so that the user knows what's happening. Data transitions like Fading, Grow/Shrink, Wipe-in/out can enforce the fact that something has changed to the user - an example provided by the presenter was replacing one piece of text with another, the use of a transition rather than a simple immediate replacement made it more obvious to the eye that something had changed. Similarly, the use of "in-progress" indications can greatly improve perception.
A good reference provided was the book called "Don't make me think", by Steve Krug.
The presenter was insistent that Ajax didn't solve all problems - and that it should only be used if nothing else would do the job.
There are a lot of technical issues with Ajax - mainly to do with the wildly varying APIs supported by browsers. For this reason it was suggested that when implementing Ajax for a web application people should use one of the main "libraries" - there are lots out there, but not all of them are good. 4 good open source libraries are: Dojo, Yahoo UI, MochiKit and script.aculi.us/prototype - it's also important to note that the choice may be affected by the affinity of the library to the server.
There was a lot of information about things that worked and didn't work, the pros and cons of many of the technical aspects - for this information the slides should be referred to.
Some Ajax UI Design Patters can be found at:
- Yahoo Design Patters library
Unlike the GNOME HIG, there doesn't appear to be any equivalent Ajax HIG published yet.
Tim O'Reilly Keynote
Attention, Please! Who Are We?
This was mainly focused on Web 2.0 ideas.
The new face of Open Source is in "Mash-ups" (web services that being data together from different sources), Frameworks (e.g. Ruby on Rails, Django, dabbledb, Ning), the movement of proprietary apps/APIs to open source.
Closed Data can be seen as a form of lock-in.
There is the open question of how do we re-invent the values of open source when the value is in the data rather than just the s/w used.
An example of how even small companies can be successful on the internal, he showed a list of the top 10 sites on the internet (2005) and showed the one called "craigslist" as being ranked as no. 7 with only 18 employees, which others were all 5000+ employees.
An example of a mash-up is in www.housingmaps.com - where Google Maps and craigslist are brought together. The legality of such mash-ups have yet to be determined.
The real message is that we don't know yet ...
- ... what the platform of the future will look like
- ... what new business opportunities are created by self-improving systems
- ... what are the sources of the lock-in and where the competitive advantage will be.
- ... what it means for something to be "Open".
Industrial-Strength Email and Calendaring with the Freedom and Flexibility of Open Source
Interesting speaker - mainly concerned with "Why do people develop Open Source?".
Most people believe that we are self-interested in economics. This doesn't appear to be the case...
It seems to be mainly down to 3 things: Glow / Show / Flow
- Glow : We need other people. When you do nice things it makes you feel very happy - Oxytocin rises.
- Show : If you take a picture of a peacock, it is still possible to recognise, even with all the colour taken away. Darwin speculated that "live is about survival" - so why does a peacock advertise? Surely it makes it more likely to be eaten by predators? Attention might give you sex, but it is also dangerous.
"To prove that one has vast resources they waste them, take risks."
Altruism - take risks, be generous, give comfort.
Dare, Care & Share - do things that other people see as risky, show you have so much that you can share.
- Flow : 1.5 Tonne of matter goes through a human body each year (air, food, etc.). 98% of our atoms are replaced in a year. We appear to be in a permanent state of re-incarnation, except we never lose the information.
Modern food is alien to us - meat / veg / fruit are staple. Grain, etc. are all new to us (relative to whole existence)
"Stone Agers in the Fast-Lane".
Web 2.0 : [Although I think it applies more widely].
We are moving towards a "Pre-agricultural food, gift economy, sociology".
We are about to become hunter-gatherers again.
Florian von Kurnatowski, Director of Open Source Programs, Scalix Corporation
Project Hosting on Google Code
Scalix was not open source until recently. They provide an Exchange alternative and have a strong user community (no developer yet since open-sourcing is recent).
E-mail is still the "killer app" for most people - the most used office tool - more important that phone to many.
Users identify E-mail using the client (e.g. Outlook) and tend to be very unaware of the server. Mobile e-mail is still very low penetration into the market.
Administrators find e-mail configuration painful, complex and full of viruses, etc.
CIOs consider e-mail an high-price operation, but business can lose money if it's not there.
Outlook is very integrated with the rest of the desktop, and without a solution to replace Outlook ...
- ... you can't replace Office with OOo
- ... you can't replace Windows with Linux
- ... you can't replace Exchange with XXXXX
- ... you can't replace AD with LDAP
The biggest problem with OpenOffice.org is that there is no e-mail.
This can be the source of a lot of lock-in - 90% of email admins have never done an e-mail migration - the remaining 20% will never do it again. [Intentional mistake]
Open Source is moving up the stack - it started with the infrastructure (Kernel, etc.), middle-ware and now is at applications.
Partnerships using Opens Source allow for "Best of Breed" partnerships. Open source removes the need for NDA and a lot of legal process. The limitations of monolithic s/w development are becoming obvious.
The creation of a full stack of enterprise-class open source apps is one of the most important new agenda items.
CalDav appears to be the direction that people are going for the Open Calendaring Format - Vista and the next MacOS/X release are supposed to support this.
Greg Stein, Uber Tech Lead/Manager, Google
[This presentation was marked as TBD on the schedule, so no one was really sure what was going to be discussed]Open Data AWOL
Google have set up a project hosting server at code.google.com. It isn't meant to content with the likes of sourceforge.net [but I think it does]. The aim is to have a simple interface. It makes use of the GMail mechanisms such as tagging, etc. to provide the tracking interface. A new backend was implemented for SVN using the BigTable database that Google use throughout the organisation - this was done to enable it to be fast and scalable, but also makes it fit nicely into the existing infrastructure used for the web searching.
New features, coming soon, will include the ability to down load files and issue change notifications.
Any one is able to set up a project, but it's likely to be removed quickly if found to be bogus. Space is limited at first to approx 200 Mb, but there is no problem about getting extra space, if you are a real project.
The interface looks quite simple to use, and is extensible/configurable due to the use of tags/labels to provide the priorities, etc. (e.g. tag Priority-High,Priority-Med, etc. are represented in the UI as simply a column with High/Med/Low) but it's all configurable to suit the projects needs. The main reason it's so flexible is that it's all done using searches.
Each project, when started picks a license (which can be changed later). They picked a limited set of Licenses since this is the direction that even the OSI are going - though the set isn't as complete as the 9 that the OSI have listed recently due to Google doing theirs first. [I understand from others that the CDDL is one that the OSI have picked, which AFAIK was announced at OSCON 2006 (in the US)].
The bug entry screen is very simple, avoiding a lot of the detail that really only developers can decided (such as priority, etc.).
[A question was asked about interfacing with other bug systems, but the answer was simply that feeds will be provided to allow other bug systems to read data, but from what I understood the question was meant to be the other way around - i.e. google aggregating other bug systems.]
Journalism versus Computer Programming
I found this to be an interesting perspective on the direction of the Open Source business model.
The title of the talk was changed to be "Free Data AWOL" (as opposed to Open, which is being over used). Some examples of free data are:
Open Data comes from several sources:
- Government (US, mostly static)
- Academia (tends to be closing off so they can charge for the research)
- You (this is the fastest growing source).
Coase's Penguin (a.k.a. Linux and the Nature of the Firm) refers to Commons Based Peer Production. For this to work the process of generating the data needs to be Fun, Quick and Easy.
The speaker asserts that "Open Systems are a done deal" (e.g. Ubuntu) but that Open Data is NOT - it's a pure necessity mainly because there is little of it.
We are reaching the limits of what we can do with software.
Open Data isn't as cumulative as Open Systems - many people think that they need 100% data before it us useful, this isn't the case.
Adrian Holovaty (Washington Post)
The speaker asserts that "Journalism, in it's current form, is broken". What is means is the way that a reporter takes "clean" data (such as what, where, when, etc.) and converts this in to human readable form (verbose) which is not processable by machines.Open Usability
There are great frameworks out there that are desperate for data (GoogleBase, craigslist) while Newspapers are the opposite, with lots of data and no framework.
We should be able to tell a story with an application, not words. Examples are then given such as the chicagocrime.org and others. [All of which I believe the speaker has produced, or been involved in]. The main point is that the raw data is being taken and presented in a different way.
We need to be smart about data. Open source, makes the code available, Journalism via code makes the data available.
Encourages understanding via transparency, also encourages derivative works.
Making It Work: How to Build a Successful Open Source Project
"Usability is the result of a process"
- Know your user
- You are not your user!
The Open Usability project began life as the "Linux Usability Study". It created a high awareness of usability in the community. There were some very quick reactions to the "accusations" made about s/w usability and fixes were quick to appear from the communities. Next came an invitation to the KDE Work Summit. There was a mixed interest in Usability there.
There are two steps needed : Understanding (Know more and understand factors) and a Framework (to collaborate).
So to build the infrastructure, it was found that existing channels were not fitting enough where they were designed primarily for developers.
The basic idea is that a project must register to show interests and commitment - "If you want someone, then we will find you someone".
At the moment there is in excess of 160 projects registered.
- Most developers think that usability is "trivial"
- How can we convince developers to be interested at all.
- There is a need for a painless workflow for both the Usability Engineer and the developers
Open Source Software Factors:
- Development is done for the community not the market.
- Contributions are mainly via code.
- No definition of the target users (everyone).
- Feedback channels are for hackers
- "Features" do not create user experience by themselves
- Usability Engineers influence is strongly dependent on trust.
So how do we get the Usability Engineer "inside"?
There is not "one" User Engineering Process, it can vary largely depending on : Size (of the project), Impact (depends on way of introduction), Community Resources and Direct Communication.
Louis Suarez-Potts (Collab.Net - hosts of OpenOffice.org)
Day 3 (Wednesday)OpenSolaris Booth (.org day)
There are two types of Open Source Software:
- Organic (Garage Project) : Linux, Drupal, Plone, Ruby On Rails, etc.
- Sponsored : OpenOffice.org, Mozilla, Eclipse, OpenSolaris.org, etc.
Organic projects start small, with a common initial goal. Decisions and processes are obvious at the start.
Sponsored projects stem from corporate goals. The corporation is the primary contributor resulting in a "power imbalance". There tends to be a lot of legacy code with architecture decisions/knowledge that's lost over time. The biggest question about creating a community is "How?".
What needs to be done?
- Pick the right license : you can change a license if needed (OpenOffice.org changed it's license after going public).
- Pick a neutral environment (people tend not to trust a corporation).
- Have Transparent Governance : Submission process, clear Acceptance or Rejection criteria (who is reviewing the code - where is the power?).
- Transparent Processes : it should be possible to know "where" code is in the process.
- Public decisions process : need to have clear decision paths.
- Good communication tools: (IM/IRC provide instant gratification).
- Useful, easy, controllable infrastructure.
- Instant gratification (e.g. Wikis) [shouldn't be underestimated].
- Documentation (Getting Started Guides)
- Mentoring (Google SOC)
- Focused but unstructured camps (e.g. Foo Camp)
- Bug Hunting parties, coding days, etc.
- "Guerilla Marketing" tactics - make things fun.
- Information for users, examples.
It should be possible to answer questions like:
- Why join? (when compared to another project)
- How to join?
- Why use it?
- why open source? (it's not a universal solution).
What OpenOffice.org is doing now
- Making it easy to contribute
- Extensions like Firefox
- Regional Developers
- Outreach : companies, schools, people, government (Make developers)
- Goal: To enable \*anyone\* to contribute to OpenOffice.org (art, translations, etc.)
Much of the day was spent between spending time on the booth and attending some sessions.OpenSolaris BoF
We set up the book in the early morning - have to say that the organisers were very helpful in getting us things we needed.
I certainly didn't expect a huge amount of interest given that much of the conference appeared to be focused towards Web 2.0, but I was pleasantly surprised to see so many people come up.
Some people wanted to see some sizzle - but there wasn't a lot that we could show - at least nothing in the base Solaris - this is definitely an area that needs some work for any future conferences - it's hard to draw people eyes using a terminal!
People familiar with OpenSolaris (or even Solaris) wanted to know more about how Solaris differed, and were very interested in the Starter Packs with the install DVDs and the live DVDs. In fact we managed to get over 60 of them handed out to people that seemed to have genuine interest. Given the number of people we saw with MacOS/X laptops I think a PPC live disc would have been of great benefit. Interestingly we also had requests for the DVDs for SPARC systems, again something we didn't have to hand.
I had an interesting, and long, chat with Rob Savoye, a long-time GNU Hacker, and author of Gnash - a freeware Flash alternative. Gnash sounds really interesting, and many people are interested from the point of view of not having to pay Adobe/Macromedia crazy licensing fees. It's a clean-room implementation based on the Flash file specifications that Adobe/Macromedia published. He has also recently started a company with an ex-RedHat guy (can't remember name) to provide a Macromedia Streaming Server alternative, but using ogg/Theora and other free formats and conversions based on ffmpeg, etc.
Currently Gnash is Flash v7 compatible, with ongoing work to make it v9 compatible. It appears to be gathering a lot of momentum and many business requests (especially in the embedded space) for it to be made available as an alternative to Embedded Java.
Certainly one to watch.
Gnash hasn't yet been compiled on Solaris - but I think we managed to get Rob interested in setting up a build server using Xen, so time will tell... It's also open source, so any one could give it a go!
Klone: An SDK for Embedded Web Applications
Despite many promises of people attending the BoF turn out was pretty poor with only 2 people attending (3 if you include the person that came in for some water!).
It's hard to know if this was OpenSolaris or the fact that it conflicted with a Make magazine makefest, and an Ubuntu BoF, and to top it off free drinks available, courtesy of Fotango... Could we have it timed any worse.
For the people that did turn up, the main interests were in ZFS, SMF, zones and DTrace, for which some demos were done, such as moving zones between machines using a USB hard-disk and a demo of Visual Panels.
Klone is a lightweight Embedded Web Server which is able to exist in 140Kb of ROM with approx 70Kb RAM. It supports TLS/SSL, HTTP 1.0, Standard Session Features (by storing session info on client).Day 4 (Thursday)Architecting Babel
It's very portable with Dev Kits for Linux, Windows and MacOS/X.
Generally the complete webserver, files and everything are compiled into a single object file, but it is possible to use different backends to enable access to filesystem storage.
Web pages can have dynamic elements that are converted to native code at compile time, and executed when accessed.
The webserver is configurable to match the host environment and desired uses.
More information can be found at http://www.koanlogic.com/
Robert "r0ml" Lefkowitz
Xgl and Compiz - New X11 Features and the OpenGL Accelerated Desktop
Robert's talk focused on the way we currently develop s/w from the perspective of spoken languages (hence the Tower of Babel reference).
Currently the majority of source code is written in English - providing an immediate barrier to entry for perspective open source developers that don't know how to read/write English. This is a specific requirement by the Debian project before you can commit any code.
If we were to use the most spoken language in the world to develop s/w then we would all be writing in Mandarin Chinese.
Also, if a person takes the sources of a current project and translate all the variable names to another locale - is this considered a fork?
So how can this be resolved - the answer is to localise the source files - not just the final application. Of course this provides it's own problems and not easy to resolve using the current tools.
So do we Sub(title) or Dub? There's no obvious answer.
Matthias Hopf, SuSE Linux R&D
X is over 20 years old, and generally enhanced by extensions.
The base X has changed with the additions of the Extensions : Render, Compose, Damage and Client-side Fonts. New Acceleration mechanisms have been added : EXA and Open GL.
Development of the X.org server has changed in that the sources have been modularised and have moved to automake/conf projects. They have also made a move to using "git" as a replacement to CVS.
The Render Extension provides new graphics primitives (transparency, polygons, filters, graduations, etc.). It makes use of "Porter & Duff" compositing (semi transparent primitives). Rendering is no longer "pixel exact". The primitives match a lot of what Cairo offers for vector graphics.
H/W Acceleration is in transition. First there was the XAA mechanisms, which have been replaced with EXA which provides some just 2 primitives - SolidFillRect and CopyRect, but this is enough for most uses.
Xgl is an X server implementation that uses OpenGL for all it's drawing. It is NOT a new display mechanism, it's still X11, and it doesn't accelerate OpenGL or provide "Cool Transition Effects". It is a frontend to X.org (which it starts in the background) and uses the Glitz library as a mid-level abstraction of OpenGL. It has limits, it provides only indirect rendering for proper OpenGL applications and the off-screen memory has to share the same address space.
AIGLX (Accelerated Indirect GLX) provides OpenGL tunnelling for normal OpenGL applications. This only works with DRI and the nVidia binary driver doesn't work.
The Compositing Model provides off-screen rendering of windows. The composite manager then brings these rendered windows into view on-screen as required. This then enables you to provide window transformations, cross-window transparencies, etc. The Composite Manager application is a user-space process. It has some problems such as it not 100% matching the X model - window content is generally RGB but the window manager decorations tend to be RGBA - this means that it has to have very tight integration with the Window Manager.
Compiz is the main implementation of a compositing manager - there also exists a fork of this Compiz-Quinn, which is a test bed for many of the leading-edge features, but is always kept in sync with the original Compiz implementation.
This is certainly an area we need to take specific interest in from a desktop perspective.