Building a Scorecard for Open Source

Perching Gull

In my previous posts, I've drawn an analogy between open source software and organic food, hinting that in both cases the rush to create a working brand lost some of the essence of the vision. I've suggested that having businesses identify "open source" purely on the basis of one "input" - using an OSI-approved license - is no longer adequate, because the success of the open source approach has led so many different companies to want to exploit the name. The need is clear; so many companies want to describe themselves as "open source businesses" that debates about "open-core" and "open source business models" were dominant at Open World Forum.

To address this, I'm proposing the Open Source Initiative go beyond the Open Source Definition and the Free Software Definition to devise some sort of a Software Freedom Definition which articulates a holistic vision of software freedom against which businesses can be benchmarked. I propose also creating a self-certified score-card which companies can complete to indicate the approach they are taking to promote software freedom as part of their business model - maybe "the Open Source Audit". I'd then expect abuses to be policed by the community at large with final arbitration from OSI.

What would be included in the two? My initial thoughts are that it should include 7-10 elements, each of which have a "yes/no" answer and each of which should be backed by a more detailed definition to make clear whether the answer is yes or no. Sample questions might include:

  • Is the license OSI-approved?
  • Is the copyright under diverse control?
  • Is the community governance open?
  • Are external interfaces and formats standards compliant?
  • Does your community operate under a patent peace arrangement?
  • Are trademarks community controlled?
  • ...
and so on. Suppliers could then state "This product achieves 4 stars on the 10-point Open Source Audit" as they self-certify. In addition, procurement policies could then state they required a minimum number of stars for products and services they procure. And the only companies that could claim to be "an open source business" would have all products scoring 10/10 - probably very, very few. A focus on software freedom - the code, rather than the company - is the answer to the issue.

[Also posted on my OSI blog. You can also watch the talk.]


I think this is a great idea.

In particular, it will definitely lead to situations where not everybody has 10 stars... but the same way that a hotel doesn't need to be 5 star to give a great experience, companies that have 7, 8, 9 points definitely still have very interesting stories to tell.

Now, do you imagine having weights for each of the 10 points, and then computing a weighted sum of the points to give a total value on the "open source scale"? Some points may be more critical than others?


Posted by Gilles Gravier on October 08, 2009 at 08:28 AM PDT #

Gilles: No, I believe the points should be weighted equally. That goal should itself lead to each of the definitions being substantial enough to be worth considering alone (as the current OSD is).

Posted by Simon Phipps on October 08, 2009 at 08:37 AM PDT #

Simon, you should check out the Foundations of Openness that Pia Waugh and Randy Metcalfe did a while back which covered similar ground, giving a project a 'scorecard' on 5 key areas; Open Source, Open Standards, Open Knowledge, Open Governance and Open Market.

Posted by Paul Cooper on October 08, 2009 at 09:01 AM PDT #

Fantastic idea, especially wrt to governance being considered as a measurable item.

Posted by che kristo on October 08, 2009 at 09:46 AM PDT #

If you are going to start talking about software freedom, then you need to start talking about freedom for users. You bullet list is the same old crap that OSI has spouted for years. Stand up, man. Stand up for freedom.


Posted by Thomas Lord on October 08, 2009 at 11:14 AM PDT #

Paul: Thanks for the pointer, very useful.

Thomas: One step at a time. Getting OSI to actually implement what it has talked about for years is a necessary predicate to protecting users freedoms by promoting transparency. I figure you'd probably rather wind it up though.

Posted by Simon Phipps on October 08, 2009 at 11:25 AM PDT #

Simon, you wrote: "I figure you'd probably rather wind it up though."

No, not really. I'd rather see a strong software freedom stance from OSI. I'm skeptical of "one step at a time" approaches for historical reasons.


Posted by Thomas Lord on October 08, 2009 at 12:05 PM PDT #

"Is the copyright under diverse control?"

You mean ownership - or can ownership and control be divorced? Surely consoldiated ownership of copyright is healthier, but the perhaps the owner needs to be under diverse control, or at least, be recognised by contributors as appropriate.

Posted by Patrick on October 08, 2009 at 07:56 PM PDT #

Patrick: I envisage that a project would get a star in that category if either it had the copyright spread among many authors, like Firefox or the Linux kernel, or if it had the copyright concentrated in a body that itself was truly under diverse control, such as a non-profit organisation. While it has other benefits, I suspect that concentrating the control of copyright of a project in the hands of a single person or entity is not beneficial for software freedom.

Posted by Simon Phipps on October 08, 2009 at 08:12 PM PDT #

Makes perfect sense to me.

Posted by Patrick on October 08, 2009 at 08:27 PM PDT #

But it seems, Simon, that actually far fewer companies care about describing themselves as "open source companies" anymore. It's becoming a passe term as everyone \*uses\* open source, rather than \*is\* open source. I think this would have been helpful a few years ago when we were in the midst of the thicket, but I'm not sure it resonates today.

Posted by Matt Asay on October 08, 2009 at 11:28 PM PDT #

Matt, you wrote:"everyone \*uses\* open source, rather than \*is\* open source".

That is why it is desirable that OSI join the free software movement and start advocating for the freedom of users.

Historically, OSI has avoided talking about freedom and instead talks only about certain (supposed) economic or software quality advantages of using open source software and of using community development. Because that has been their emphasis, they lack any principled basis on which to state that a "mixed model" should get "fewer stars" than a firm that protects the software freedom of its customers.

In part, this is because of the way in which OSI has centralized the OSD in its advocacy. Remember that the OSD started out in life as the Debian Free Software Guidelines. In the context of Debian, it was a list of rules for licenses that are \*acceptable for use\* in a free software distribution of a complete operating system. It was never intended to directly express the concept of software freedom or speak to the importance of software freedom - it was legal advice for a free software distribution project.

Because of its role in Debian, the Guidelines (now OSD) say nothing at all against such things as mixed-model licensing or businesses. Those things weren't important to Debian's needs. If there was suitably licensed code, Debian could use it whether or not some third party was holding back proprietary enhancements or doing other freedom-robbing tricks. Somehow, OSI wound up embracing that narrow, technical matter that pertained to Debian, and making it their "mission". Having adopted a document that's not about freedom as their statement of principles, it's going to be difficult for them to credibly start talking about software freedom now.


Posted by Thomas Lord on October 09, 2009 at 02:24 AM PDT #


It seems to me that a good first step towards a scorecard would be for OSI to embrace and endorse the "four freedoms" of which the free software movement speaks. Then, the rationale for any item on the scorecard can be related to those in a clear, easy to understand way.


Posted by Thomas Lord on October 09, 2009 at 02:27 AM PDT #

Simon, you won't avoid people using weighted criteria on their own... since everybody has their priority, so why not acknowledge it and implement it from the definition part. Even if only to say that in a generic world each criteria has the same weight as the others... but that each individual is free to adapt weights to their own context...

Posted by Gilles Gravier on October 09, 2009 at 02:44 AM PDT #

I'd like to get more of a sense of what problem you're trying to solve. I.e., say open source scorecard data about various companies is available. What sort of decisions can that information usefully influence, and how might it influence them?

For example, say I am choosing an operating system. I could choose from a variety of Linux distribution vendors (diffuse copyright ownership; no vendor is complete copyright owner), or a variety of Solaris vendors (concentrated copyright ownership; one vendor owns copyright, others don't). What would the scores of various vendors be, and what would be considered good or bad?

Some of these metrics could be seen as positives or negatives depending on who is looking and their needs. For example, diffuse copyright ownership can be considered a negative, in that there is no entity which can legally defend the code; or as a positive, in that no vendor is privileged above others by the rights conferred by copyright ownership. It seems hard to turn this into a linear scale and end up with something that will reliably be a good guide to decision-making for anyone who wants to use open-source software - which brings me back to my original question, what is this good for?


Posted by Tim Boudreau on October 09, 2009 at 05:51 AM PDT #

Hi Simon,

I've had a nagging question in my head about this - how are you differentiating the Software Freedom Definition from the Free Software Definition? It seems to me that the answer to that question will frame the discussion you are proposing.


Posted by Matt Aslett on October 12, 2009 at 07:10 PM PDT #

Post a Comment:
Comments are closed for this entry.

Thoughts and pointers on digital freedoms and technology markets. With a few photos too.


« June 2016