Friday Jul 18, 2008

The changing face of the FSF

When I first started out using the Internet, there was a small collection of machines at MIT where anyone could login as a guest to compile and run software. These were the part of the GNU project. It was free access to machines and disk space that you might not otherwise have, at the time. This was circa 1990 - before the WWW - and the most common method for downloading software, at that time, was via ftp. Some people would buy CDs of free software because it was cheaper/quicker/easier than downloading 600-700MB on a tail of the Internet, by modem.

Almost 20 years later and the default interaction with the FSF and GNU project is via the web. And what a difference it is. Today when I go to a web page to try and download free software from www.gnu.org it is all Donate to the FSF or Buy our distribution. Oh dear. I'm sure if I point my browser at an ftp URL, I won't be plagued with such nonsense, but the fact remains that the focus of the FSF and its GNU project has visibly changed from being a conduit for free software to give us money. To see what I mean, visit http://www.gnu.org/software/software.html and read down to the How to get GNU software. Buy or download and please donate. Sounds more like shareware than freeware.

What used to be about free software now appears to be about commercial software. What used to be about people donating their time to something they love doing now seems to be about employment.

It is somewhat ironic that in the past, the mantra of FSF/GNU was that software should be free and you shouldn't have to pay for it - programmers who work on it would have other real jobs. It would seem that the FSF/GNU have had a rather substantial change of heart, given the blatant self advertising (for money) their web pages now do.

One might be lead to believe that perhaps the success of GNU software has now lead to the project becoming corrupted by its success: everywhere out there people are using GNU software to make money, so why shouldn't the project itself get some of that reward?

How long then, until the FSF becomes a part of or itself a for-profit organisation?

Thursday Jul 17, 2008

GPLv4 anyone?

First, I should state that I am not a lawyer and nor have I any training or schooling in law.

Over on one of the NetBSD mailing lists for its users, netbsd-users, someone from outside posted an email declaring that they were creating GPLv4 such that it was more compatible with using software freely. The initial email CC'd lots of interesting parties. As one might expect, RMS chimed in with the predictable response.

This made me stop and think. Why can't someone call something they create the GNU GPL? Or more to the point, is there anything stopping me from creating a document and calling that GPLv4? I summarised these questions in a followup email to Richard's reply, to which there is no response yet.

For the sake of curiousity, I went and had a look at the section of the license that allows you to upgrade to another version. It reads like this:

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

This paragraph is interesting on a couple of grounds:

  • 1. The use of "the License" could possibly be argued to be ambiguous as nowhere in the document itself is the term "the License" defined. It is common in legal documents to see something like this: ...the GNU General Public License (hereafter referered to as "the License")... or even to define the term "License" in a similar fashion to the manner in which Program and others are defined much higher up in the document.
  • 2. The reference to "any later version" is again potentially ambiguous. Do all GNU General Public Licenses have to have from the FSF? Why can't I write my own GNU General Public License? Furthermore, this phrase refers directly to the License which is not clearly defined so if we're not sure what that means then this also has no meaning.

Question is, if you have a couple of loose ends like that, can you thereafter unravel the entire GPL or use them as grounds for defense against litigation for an alleged infringement of the GPL?

The catch here is that this is a legal document and as such, precise wording and phrasing is required to ensure that the correct meaning is conveyed. It may be that the use of English here isn't vague in some courts of law (or to some people) but put a couple of laywers in court to fight over the GPL and I'm sure that the meaning of ambiguous phrases like this would be entertainment for them for days.

I suppose the real question to ask here is what is to stop anyone from authoring anything titled GNU General Public License Version 5 and thereafter claim it is a later version of the GNU General Public License, originally written by the FSF? Can you imagine if Microsoft found a way to legally author a later version of the GNU GPL that could be used in place of the existing GPL?


In case you have forgotten, I'm not a lawyer and all of the above is just my personal opinion having spent 30 seconds looking and thinking about it. Well maybe less than 30 seconds - I try hard not to think about the GPL, it makes me want to vomit.


Another twist on this is what exactly does Free Software Foundation refer to? The document doesn't define that, either. Does this mean that in a country where there is no entity called the Free Software Foundation that there are limitations on enforcing the GPL? Again, what if Microsoft were to register an organisation known as the Free Software Foundation in Nigeria or some other unobvious country? I'm actually curious if it is possible to register such an entity separately in California to Massachusetts? Or is the register for foundations country wide?

Wednesday Nov 14, 2007

Popularism Software Engineering

In most other open source communities, whether or not something happens is a reflection of the desire of people to make something happen by spending time on a project. So if it is a good idea, the idea itself leads to change without anything specific needing to happen. If an idea is particularly bad, this will generally be pointed out at some point in discussion, whether this is when the idea is being floated or prior to integration. OpenSolaris has invented a new mechanism for determing whether or not a project gets the thumbs up: voting.

Integration of voting into the formal structure of an open source community is at first the means to formally record the explicit desire of an open source community. This works so long as those eligable to vote do indeed vote. However it introduces a new risk for projects: it requires that the person introducing the project actually be popular enough within the community to attract votes - visibly casting a positive vote for an unpopular person (regardless of the technical merit of a project) may carry social ramifications that voters are unwilling to take on. Thus the software engineering is subject to how popular the principals are and hence this leads to popularism software engineering.

There's another risk associated with this too: if the person proposing a project is perceived as being too popular or that is necessary to be seen to be supportive of said person through positive votes, it becomes increasingly likely that poor positive decisions will be made about what projects are approved. Thankfully OpenSolaris has an ARC (Architectural Review Committee) that is required to review proposals for architectural change (this is the case with nearly all projects) which diminishes the possibility of popularism voting for projects leading to their eventual delivery into the code base.

Sunday Sep 30, 2007

Which free un\*x for a database today?

The quest to be the fastest webserver or database server or whatever is no different to any other part of the IT market, with the leader constantly changing, depending on who's where with their current development cycle.

Over at one of the other open source projects that I'm involved with, NetBSD, some of the project members have been doing some testing with MySQL on some "older" SMP machines they have at their disposal.

The first graph shows NetBSD-current (the equivalent of "nevada" or OpenSolaris) against an earlier release of NetBSD, OpenBSD, FreeBSD and Linux.


After seeing the above graph, a few people (including myself) were curious about how NetBSD's performance stacked up against OpenSolaris - afterall, Solaris is \*the\* platform for people running production databases in many environments today. The results are quite surprising but very pleasing to the NetBSD project. (For details of the test and hardware used in the test below, see http://www.netbsd.org/~ober/

Sunday Aug 26, 2007

Manufacturing a community

OpenSolaris, the open source community that is bravely forging ahead in unchartered waters for the company formerly known as Sun Microsystems, now known as JAVA. OpenSolaris is the creating of Sun, designed, architected and implemented by Sun. But can a manufactured community work? And how long does it take to become a real community?

In thinking about how the OpenSolaris community came to be, I find the temptation to liken the construction of OpenSolaris with that of modern city planning: someone sits down, designs where roads will go, where houses will be, where shops will be, etc. The end result is carbon copied suburbs/communities with neatly mowed lawns, 2.5 kids per house, etc. Compare this to parts of cities that have grown organically, before we tried to plan everything: location, geography, access to water and transport formed the foundation for where communities grew. In some cases there have been extraordinary influences (gold rushes) that have led to a surge in local development of a community, only for them to be ruins today because the foundations were rotten. Compare how the core of a city such as London functions with that of its outer suburbs: readily available public transport, need for a car is low as many things that are necessary for day to day life are not far away, etc. The community has grown and built itself up in a way that enables it to function. In contrast, modern communities are designed around an expectation that people will go there and function in a specific way (ie drive a car) rather than evolve. To visit a city with history is akin to visiting a living organism, whereas those communities that have been created feel "stale" by comparison.

Step back to OpenSolaris. Someone came up with a grand plan for how OpenSolaris would be built, with communities, community leaders and a bunch of process to match. Why? Because there was a perceived need to be able to present OpenSolaris as an 2-minute instant noodle style of dish: add some water, push the button and shortly thereafter, voila, baked open source community. Compared to all of the other significant open source projects, there has been little or no evolution. Leaders haven't emerged: they've been selected because of what they do or who they are at Sun. The structure within OpenSolaris has been loosely modelled on Sun: create communities that reflect the organisational structure. But here's the catch: for an instant bake OpenSolaris structure, was there any other way that we Sun folk could recognise and function with?

What is wrong with this approach? Well, where (for example), is the user community for OpenSolaris? Or is the user community simply the OpenSolaris community itself? Is there a developer community? Is there a kernel hackers community? What about a systems administrator community? Or one of people interested in appliances? Some of these do exist today, which is good, but similarly we have communities that are dedicated to single pieces of technology where they otherwise might not exist as explicit communities if OpenSolaris had grown.

The best example of this is SMF. No other open source platform has a community around what it uses to boot the system up into a state that it can be used. Does SMF need to be a community in its own right? Yes, it deserves and needs its own discussion forum, but SMF is a central topic for any OpenSolaris systems administrator and in part for some application developers too, although their involvements are quite different. As part of this consideration, it is important to acknowledge that every platform is different so maybe this construct is correct for OpenSolaris.

Part of this issue is that in the construction of OpenSolaris, thinking has been limited to communities. Community this, community that. Everyone wants to have or be a community. In contrast with other open source projects, there is no concept of "leaders" and communities form around individual products that are in general independant of operating systems. People gravitate to be around something that interests them. The last point is important to consider as there is only one product being delivered by OpenSolaris: OpenSolaris itself. The reflex action in OpenSolaris to support a group of people that want to do X is to say "you need a community to do that" - this puts the cart before the horse. For example, if someone came to OpenSolaris and wanted to port it to the ARM CPU, they'd be offered an ARM-OpenSolaris community. Sure, this gives them a great platform to do lots of wonderful thigns and share what they're doing, but for the peron(s) involved, it creates more infrastructure that they need to manage (web pages, leaders, etc) than what they're usually interested in at first: getting it to work. It could be argued that a community of people porting OpenSolaris to various platforms would have more to offer as people traded tips on how to get it working on their hardware (some issues in tasks like this are platform/machine dependent, others relate to the architecture of the operating system.) The catch here is that it is unlikely with the current OpenSolaris makeup for such a community to happen.

It is almost like OpenSolaris has been built to be a competitor of the existing open source communities, rather than being a participant of the open source movement/community as a whole.

If OpenSolaris was made up of living pieces of Lego and we dismantled the current building, would it put itself back together the same way as it is now? Would it even look the same? And dare we dismantle it now or in the future so that it can build itself up in the fashion that OpenSolaris (and not a group of architects at Sun) desires?

Tuesday Jul 17, 2007

Scripting with cscope

The cscope (and cscope-fast) tools provide an excellent way for us to search for particular instances of phrases, etc, inside the Solaris source code. The only downside is that the tool is interactive: it wants to use curses to ask for input and display its output. How then to make use of its knowledge in scripts?

Whilst the proper solution is to have cscope and cscope-fast modified to have a non-curses output format, a workaround that can be used in the mean time is to define a special "tty" that ensures no escape sequences (or other nasties) end up in the output.

  • 1. Create two directories $HOME/.terminfo and $HOME/.terminfo/c
  • 2. Create a termcap definition file, $HOME/.terminfo/c/cscope.termcap. For my first attempt on this, the file I created was a termcap definition by copying vt100's and making all of the escape sequences into tab's. The contents of this file look like this:
  • 
    cscope:\\
            :do=\^J:co#9999999:li#9999999:cl=\^I:sf=\^I:\\
            :le=\^H:bs:am:cm=\^I:nd=\^I:up=\^I:\\
            :ce=\^I:cd=\^I:so=\^I:se=\^I:us=\^I:ue=\^I:\\
            :md=\^I:mr=\^I:mb=\^I:me=\^I:is=\^I:\\
            :rs=\^I:ks=\^I=:ke=\^I:\\
            :ku=\^I:kd=\^I:kr=\^I:kl=\^I:kb=\^H:\\
            :ho=\^I:k1=\^I:k2=\^I:k3=\^I:k4=\^I:pt:sr=\^I:vt#3:xn:\\
            :sc=\^I:rc=\^I:cs=\^I:
    
    
  • 3. Convert the termcap into a terminfo file that can be used by curses, using captoinfo and then tic. To do this, I used the following script:
    
    #!/bin/sh
    cd $HOME/.terminfo/c
    setenv TERMINFO $HOME/.terminfo
    captoinfo cscope.termcap > cscope.terminfo
    tic cscope.terminfo
    
    
  • 4. Write a front end to cscope-fast.
    
    #!/bin/sh
    TERMINFO=$HOME/.terminfo
    export TERMINFO
    TERM=cscope
    export TERM
    cat /dev/null | cscope-fast -d -0 $1 | egrep '\^[1-9]'
    
    

And the end result is I can do this:


$ ~/bin/cscope-grep ip_input
1       ip.c    <global>        15017   ip_input(ill_t  \*ill,   ill_rx_ring_t  \*ip_ring,        mblk_t  \*mp_chain,
2       ip.h    <global>        3225    extern  void    ip_input(ill_t  \*,     ill_rx_ring_t    \*,      mblk_t  \*,
3       ip_if.c ill_capability_dls_capabl       2925    dls.dls_rx      =      (uintptr_t)ip_input;
4       ip.c    ip_rput 14993   ip_input(ill,   NULL,   mp,     NULL);
5       ip_netinfo.c    ip_ni_queue_func_impl   1293    ip_input(ill,   NULL,  packet->ni_packet,       0);
6       ip_squeue.c     ip_soft_ring_assignment 754     ip_input(ill,   NULL,   mp_chain,       mhip);                          

Update

As Alan Burlinson mentioned below, cscope-fast has a command line option, "-l", that allows for it to generate output that is not screen orientated. This can be used like this:


$ cscope-fast -l -d -0 ip_input
uts/common/inet/ip/ip.c <global> 15017 ip_input(ill_t \*ill, ill_rx_ring_t \*ip_ring, mblk_t \*mp_chain,
uts/common/inet/ip.h <global> 3225 extern void ip_input(ill_t \*, ill_rx_ring_t \*, mblk_t \*,
uts/common/inet/ip/ip_if.c ill_capability_dls_capable 2925 dls.dls_rx = (uintptr_t)ip_input;
uts/common/inet/ip/ip.c ip_rput 14993 ip_input(ill, NULL, mp, NULL);
uts/common/inet/ip/ip_netinfo.c ip_ni_queue_func_impl 1293 ip_input(ill, NULL, packet->ni_packet, 0);
uts/common/inet/ip/ip_squeue.c ip_soft_ring_assignment 754 ip_input(ill, NULL, mp_chain, mhip);
>>

While this gives us a leg up, there are two problems:

  • 1. It is still expecting input from the user so we need to do "cat /dev/null | cscope-fast -l..." to make it terminate.
  • 2. There is no clear field delimiters such as the TAB's in the other output. As long as none of the first 4 fields contains a space, this isn't too much of a problem as it can be worked around.

To wrap this up, I've used a perl script below because I'm more familiar with perl being able to carve up an array and print it out than I am with shell.


#!/bin/perl
$args = join(' ',@ARGV);
open(I, "cat /dev/null | cscope-fast -l -d $args|") || die $!;
while (<I>) {
        last if (/\^>>/);
        @F = split(/ /);
        @B = splice(@F,3,$#F);
        @A = splice(@F,0,3);
        print join("\\t",@A)."\\t".join(' ',@B)."\\n";
}
close(I);
exit(0);

Thanks Alan for the pointer and reminder.

Friday Jun 29, 2007

Split personalities

Working inside a company that is backing an open source project poses a interesting challege for those who really believe in open source: sometimes you need to take two different perspectives on a situation.

As an employee of Sun, there's no question about what I want to see as the outcome from the OpenSolaris project, but sometimes this is at at odds with what I want from OpenSolaris as an open source community member, where my goals and expectations are much more idealistic. That isn't to say there is anything necessarily wrong with one or the other, just that the goals aren't (and can't) always be aligned.

At what point does having a leg in both camps represent a conflict of interest between the two legs and how do I not end up being torn in two? Do I remove a leg from one camp? Or try and bring both into alignment? Or just give in and wind up crazy?

One element that is becoming increasingly clear is that interacting with the OpenSolaris community using an @sun.com email address, carries a burden that in some instances is too heavy to wear and can quite possibly impede the interaction I'd like to have with the community.

How do other people manage this type of situation?

  • Just give up on trying to be in both places, adopt one and withdraw from the other?
  • Construct boundaries around the amount/style of interaction with one?
  • Or do you never find yourself in such a position because you never see conflicts of interest?
  • Does anyone know how people that work on Linux for IBM (or others) fare in this situation?

Thoughts welcome.

Sunday Jun 24, 2007

Sliding value of the USD...

For many people coming to the USA means coming to a land of endless riches and wealth, and maybe 5 years ago I would have said the same, but right now, as an Australian, moving to the USA increasingly looks like a bad decision as the value of the US dollar continues to slide vs the Australian Dollar. How bad is this slide? Look at this graph from yahoo Year to date, USD/AUD.

At the start of the year, the US dollar bought 1.26 Australian dollars, but since then there's been a drop of around 6.4% to around 1.18. To an Australian working in the USA that has outstanding debts (such as a mortgage) in Australia, this effectively amounts to a real degradation in the value of my salary by the same amount - not exactly what you would call uplifting but at the same time, outside of the control of everyone in my immediate sphere. To put it more succintly, at the start of the year USD$1000 = AUD$1260 but today, USD$1000 = AUD$1180. At this end of the scale, it is only $80, but multiple that number by 10 or 20 or more and it becomes significant.

What is responsible for this change? Lots of things. For one it is intrinsicly tied to the strength of the Australian economy - exporting of coal to places like Japan and iron ore and other basic materials to China is resulting in a windfall to various Australian operations as the hunger of that country to meet its demands from growth. Consequently inflation is running close to the high watermark in Australia, prompting the Reserve Bank of Australia to keep nudging the interest rate up - Australia's interest rates are around 0.75% above the global average meaning that short term cash deposits in Australia are generating a higher return than elsewhere - the current cash rate there is 6.25%. Looking on the US Federal Reserve web site, it would appear that the official interest rate in the USA is about 1% lower - a difference of 16%. Yes, if I was an American looking to invest and make money, I'm sure I'd have a greedy eye or two on what is happening down under.

Update 2/7/2007: at 1USD=1.17AUD, the drop from 1.29 is now 7.14%. Depressing. They say what goes up must come down, but what was up and what was down? 18 hours later, USD$1 = AUD$1.16. Sigh

Friday Jun 15, 2007

The next step for packet interception in [Open]Solaris

The packet filtering hooks project (PSARC/2005/334) delivered a first take on a framework to deliver the ability to intercept packets inline with the processing flow without requiring any additional STREAMS modules to be inserted between IP and the network device driver. Whilst this has been successful in improving performance for IPFilter, the other primary goal of exposing an interface for external use has not yet been achieved. The rest of this blog entry will summarise what the state of play is here and what we need to be doing for the future. The design document for PSARC/2005/334 can be found at: Packet Filtering Hooks Design, 3 Sep 2006

Interface Changes

The goal of software engineering fot the Solaris kernel is have all of the Committed public APIs become stable, so that code can osentsibly be recompiled, or even reused as binary blobs, on any later release. This means that it is necessary to understand in great depth how the interface is going to be used.

For the packet filtering hooks, the design presented for PSARC/2005/334 was for a world without IP Instances (PSARC/2006/366.) IP Instances introduced something very important that was previously absent: a pointer that was the context in which all of the networking code needed to run. For packet filtering, this impacts the interface at which packets are delivered to IPFilter. It also impacts the initialisation/teardown as IPFilter was changed to provide a seperate set of rules for each IP Instance. This one change, introducing context for IP, would have meant an incompatible change was required to the kernel interfaces published - not a position we would ever want to be in by design.

Life being what it is, the existing documentation for PSARC/2005/334 was written without taking into account the changes from PSARC/2006/336, so at this point in time, the documentation of the interfaces is not 100% correct in describing the function calls and parameter lists. Looking forward, we need to revisit this area and update internal drafts of documentation to better reflect these changes so that they can be included in the distribution and published at the appropriate point in time.

Functionality Changes

The biggest problem with PSARC/2005/334 as it existed going into PSARC review was how to manage multiple hooks for the same event, especially when hooks are given license to change the contents of a packet. The first proposal was to use a numbering system that implemented a priority mechanism, similar to what Linux uses today, but this was rejected. The current thoughts on how to proceed with this are to allow two different mechanisms:

  • When registering a callback for a particular hook event, allow one of following attributes to be specified:
    • first - put the callback at position #1 in the list
    • last - put the callback at end of position the list
    • before - require that this callback be before another callback
    • after - require that this callback be after another callback
  • Allow a command line utility to specify a more absolute ordering, such as "a,ipf,\*,y" that overrides the above

Without a final design that is accepted to answer this capability, it is not possible to promote the current API to being suitable for public use. This is one of the major issues that is outstanding today - only one callback can be registered for hooks such as inbound IP packets.

Kernel Socket Interface

At the time the first design was put togehter, there was no work being done on an interface to present an API for kernel socket programming, so some functionality that exists in user space via ioctls was implemented internally with direct function calls instead. An easy example of this is obtaining an IP address from a network interface. Fast forward to today where we do have a project underway to deliver Kernel Sockets, we need to reconsider whether or not it is necessary to retain that direct function call interface or discard it because it duplicates functionality.

Summary of the state of play today

So where are we at today?

Today we have a programming interface that exists in [Open]Solaris where the primary user is IPFilter. The API used is considered to be private (or internal), so strictly speaking it is not available to programmers outside of Solaris development. However, because it will result in various header files being shipped and that the source code can be easily observed, it seems like it is there, ready, waiting to be used.

At this point, the best we can say is that people should look at the PDF above, look at the header files and start to experiment with it, get a feel for how the interface works, how to integate it into your greater source code tree, etc, with the knowledge that whilst it may compile and work cleanly for you now, the programming interface will change, it will fail if another hooks beats you to registering and most importantly, it is unsupported, meaning if someone else outside of Sun has a problem (i.e panic) using it, Sun is not in any way obliged to either fix the problem or provide a patch.

Invitation to ISVs

If you're reading this blog entry and you work for an ISV that is intending to target Solaris for your product or you're working to build a product on top of Solaris and wish to make use of this interface at some point in the future, the best advice is to contact your local Sun sales office and get in contact with someone from our MDE team.

Discussion on changes

If you would like to either comment on the above design doucment for PSARC/2005/334, have some requirements that are currently not being met by this or planned future work, or anything else you'd like to discuss relating to this topic, please join us in the OpenSolaris networking community in the networking-discuss forum.

Monday Apr 30, 2007

Installing operating systems

On the weekend, I completed an installation of 10 operating systems in under an hour, all by hand. All of them were Unix, all of them using media I had downloaded and burnt to CD.

The operating system of choice? FreeBSD. In less time than it takes me to do 1 install of OpenSolaris, I did 10 of another open source platform. Granted the other install didn't include java and GNOME and StarOffice \*but\* what it did include was the X-Windows utilities (xterm, etc), the base FreeBSD operating system AND the kernel source code.

Each installation used less than 2GB of disk, delivered me a working envionment for software development (compiler - gcc, editors - vi, debugger - gdb) that needed less than 64MB of RAM to run in a remote fashion where I could launch Xterms back at my main console.

That is what you call a low barrier to entry for someone who wants to install and play around with an operating system.

Tuesday Mar 06, 2007

What is OpenSolaris success?

Now that opensolaris has been progressing for almost 2 years, some people are starting to ask the question "is it a success?"

Which begs the question of how do you measure its success?

Is it by the number of developers who subscribe to be a part of OpenSolaris?

Is it by the number of projects?

Is it by the activity in the OpenSolaris discussion groups?

Or something else?

I'd argue that none of the above are good measures for judging its success.

When the question of how to measure its success came to my mind, I started to think about how long it would take for OpenSolaris communities to be able to contribute in a worthwhile manner and the rationale goes something like this:

  • for most college students, by the time they're past the first half of their first year, if they're the type of student who will become involved in the open source movement then they've already chosen an allegience.
  • college students seem to be (roughly speaking) the set of people with most time on their hands to dedicate to work on open source projects.
  • a college student is ordinarily going to require 2-3 years of learning before being ready to contribute to OpenSolaris in a meaningful manner.
  • given the 2-3 year lag and wipe out the first year of OpenSolaris (2005), this establishes a period of 3 years before we can expect to see meaningful input from college students.
  • to attract the dedicated geeks who start hacking in their teens, we've a longer wait, possible 4 or 5 years, for someone to emerge from either starting out playing with OpenSolaris and run with it and/or for OpenSolaris to gain a big enough name amongst those who work on open source things to be worthy of attention.

As an example of the problem here, although I've seen multiple people (and in different areas) invite outsiders to be a part of an OpenSolaris project to provide new functionality, I've yet to see anyone step up to the plate to do more than provide just bug fixes.

Is this a problem? Not directly. What's more important is that stake holders in the OpenSolaris project, inside Sun, need to understand that it may take a length period of time (my estimate is 3-8 years, with anything meaningful inside 5 years a bonus) before there is a start to any real return on the investment in it as an open source project.

The problem for Sun, entering the open source space with OpenSolaris today is that many of those who it would like to have contributing are already at work elsewhere on other projects. Sun is somewhere between 10 and 15 years "late" and that in being "late" it faces a much harder job of being successful. Kind of like how it would be that much more difficult to start up a new ISP or search engine or internet auction web site, today, and compete with the giants that exist already.

What about other contributors? The number of people who will "defect" from one open source project to another is small but not 0. In general the reasons for this are:

  • becoming disenchanted with where the project is going
  • personal conflict issues

Neither of these are common events.

Against all of this, in the background there is the "problem" that OpenSolaris is a product of Sun and that anyone who works on an OpenSolaris project is quite clearly going to be contributing work, "for free", to a company that will directly benefit. That this is the case is driven home by the agreement required to participate in OpenSolaris forums. The agreement spells out that you grant Sun any rights required to use any ideas you discuss in its forums. For Sun, this is a natural requirement and as an employee, I agree 100% that it needs to be present. That it isn't enforced through the mailing lists is something of a glaring loophole and could be a problem in the future. Is OpenSolaris and Sun the only people who do this? No. The NetBSD project also requires developers to assign copyright to The NetBSD Foundation for all code committed there and submitting code changes to some FSF projects (such as gcc, gdb) requires additional paperwork to assign copyright. Why is this an issue? Well, the lack of there being a single person having copyright over the Linux kernel means that no one person can decide to change it from being GPLv2 to GPLv3 or something else.

Although there are constant complaints about the licence choice (CDDL vs GPL), this is not likely to be a deciding factor for anyone. OpenSolaris needs to be "cool" in the minds of those who have time to burn, first. Won't changing the licence make it "cool"? I have my doubts about this. Being "cool" will mean making it easy to do different things with OpenSolaris (i.e. use it on something with 32MB of RAM and a 32MB CF card), being able to get your name inside a .c file for submitting a driver patch for a new card you have, etc.

So what does this discussion have to do with OpenSolaris succeeding? Because for OpenSolaris to succeed, it needs to become bigger than just what it is at Sun. And it can only do that by attracting more people to it than just from at Sun.

Recently on one of the opensolaris.org forums I added the comment that OpenSolaris will be successful when it is in a position to exist without Sun. The reason for is are somewhat obvious: if Sun folds then OpenSolaris goes with it as all of the infrastructure (servers, people updating the source code tree, etc) comes from Sun. Without Sun, there is no way for OpenSolaris to continue evolving. On top of infrastructure, it is also a reference to the developer base. So long as the majority of projects committed (or putback) into the OpenSolaris code base come from within Sun, if Sun goes away then so does the majority of the activity behind OpenSolaris - there would be a high risk of OpenSolaris just withering and dieing at that point.

How does Sun address this?

Firstly it needs to assist in making the project function independently of the work that happens inside Sun. In other open source projects, there is no concept of the role "gatekeeper". At Sun this role needs to be focused on bringing code in from OpenSolaris, to build the product called "Solaris" rather than controlling what gets putback into OpenSolaris.

Secondly, management at Sun needs to step back, to the point where managers, directors, VPs, etc, should have no involvment with the project as an employee or agent of Sun. Successful open source projects are managed/run/driven by the folks who do the technical work, not their "managers/minders/directors".

Thirdly, Sun needs to create a non-profit organisation that "owns" OpenSolaris and donate money/equipment to that and invite other individuals/companies to do the same.

But most of all, patience is required. The success of OpenSolaris cannot be measured like normal projects.

So, at the end of all of that, if I weren't a Sun employee, would I register with the OpenSolaris community? No and primarily because of the intellectual rights issue. I would limit myself to being a subscriber to the mailing lists. There are other issues but they're less tangible and more to do with the "feeling" of the OpenSolaris community forums when compared to others, elsewhere.

Wednesday May 31, 2006

Solaris Packet Capture

If an application today wishes to capture packets from a network device, it is required to use the time honoured tradition of opening the device, talking to it with DLPI messages and pushing onto it the bufmod/pfmod modules.

However, for those who are serious about packet capture, it quickly becomes apparent that bufmod is a bit on the slow side.

In an answer to this, back in 1999, Casper Dik spent some time working on a module called membuf. This actually integrates BPF filtering, enabling libpcap based tools to work easily and efficiently with it. The catch here is that we're still waiting for membuf to find a way into (Open)Solaris.

If you hunt around on the internet, you can possibly find another approach in the guise of bpfmod. Somewhere I found a reference to the source code for it (can't finda a good one now), downloaded it and got it working with Solaris. Yes, I've got the source code for that, but it is the result of me hacking on the code with someone else's license (in addition to the BSD one for BPF) so I'm reluctant to make that available until it has been properly reviewed (not a quick process). For something that is just a quick hack, this seems like overkill so I'm leaving it tucked away in a corner. It wasn't a complete port, however, as one area of BPF I couldn't work out how to do with STREAMS on Solaris - cause a read to return after (say) 100ms if the capture buffer wasn't full and/or earlier if it does fill.

A current project that could have a positive impact in some areas of packet capture is Crossbow. Crossbow will be making it possible to specify low level filters (supported by hardware, where available) for classification of packets. So if you want to capture all port 25 TCP traffic, hardware support pending, there would be no filtering in software, it would be carried out by the NIC's classification. Of course there are details in making that work.

There are some other ideas being kicked about, such as benefits of being able to DMA directly into buffers that are mapped both in kernel and user space, hopefully saving the need to copy the packet n times as it goes between internal code paths, execution modes, etc.

Of course The Holy Grail, as far as packet capture is concerned, is to use a NIC like that sold by Endace.

Monday May 22, 2006

Hardware cryptographic support for OpenSolaris

Over in the OpenSolaris Security Community, Darren Moffat has added a new page that details supported cryptographic hardware. The great news is we don't support only the Sun cryptographic accellerators and if you're into writing device drivers, there are some openings there for people to further improve our hardware support in that arena.

This is a worthwhile reference if you're looking to build your own VPN appliance and want to offload some of that hard grunt crypto to dedicated hardware.

Tuesday May 16, 2006

ifgrep - new tool for Solaris?

In a discussion last month, the idea to create a new command line program called "ifgrep" came to mind. On the weekend just past, I spent half a day working on some code to make this happen. On Monday, I sent out an email to networking-discuss-AT-opensolaris.org to announce this work.

If you would like to try it out, you can download it from http://www.opensolaris.org/os/community/networking/files/ifgrep.c. Please bear in mind the comments in the announcing email if you find problems or would like to see/make changes.

Enjoy!

Solaris for Network Appliances

Over in the OpenSolaris Appliances community, I today uploaded some new content talking about using OpenSolaris for a network appliance and projects that relate to this field

So if you're interested in seeing Solaris work on a SBC (single board computer), as a firewall or router or VPN device, please drop by and join in the community.

About

avalon

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today