Freeware Jet Engines Dangerous: Does free software imply low quality results?

In a recent NY Times article on the R data analysis language Anne H. Milley implied that her personal happiness stems from quality of jet engines that can only be achieved through the use of commercial software.

If I were in Ms. Milley's position I too would be happy if people were paying to use my product. What does not particularly matter to me is the cost of software which is used to design those engines. What I truly care about is the quality of the engineering, construction and validation procedures. In any case as a passenger I am unsure how I can determine if the equipment I am flying on was designed using free software or not. I will have to settle for being happy if my flight is on time, security lines are short and I get home to see my wife.

“I think it addresses a niche market for high-end data analysts that want free, readily available code," said Anne H. Milley, director of technology product marketing at SAS. She adds, “We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.”

[From R, the Software, Finds Fans in Data Analysts - NYTimes.com]

Given the implication that happiness, value and quality are only available for products designed using commercial software; I find myself retroactively saddened by all of the things that I have seen which were designed before commercial software packages were available. I would like to thank Ms. Milley for sucking the joy out of my life.

Comments:

“I think it addresses a niche market for high-end data analysts that want free, readily available code," said Anne H. Milley, director of technology product marketing at SAS. She adds, “We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.”

This is in reference to using an open source language, "R" , as opposed to a commercial language.

Having worked for the commercial and military avionics industry I agree with Milley's statement completely and with no caveats. Any system, software or hardware, on which any failure results in loss of life is a clear 7x24 mission critical system . This is a system that \*cannot\* be allowed to fail. This concept, while repeated and lip service paid, is clearly not understood by many.

All our avionics software was written in "C" or assembly language. Any "C" code written, once compiled was further reviewed by assembly gurus to ensure the assembly generated was acceptable. All code was subjected to a long integration and test effort, by a separate team which specifically DID NOT include the developers. After that phase it was further subjected to exhaustive tests by another separate team.

Because of the nature of the work, any bugs or problems in the tools were escalated immediately to the vendor and fixed in a matter of days, if not hours, at the highest priority. That was possible because it was purchased, with further purchased support contracts. The level of support we required is simply not available for Linux.

The completed flight systems I worked on had triple redundancy in both hardware and software, with constant diagnostics running in the background. Because of the level of criticality, this kind of system CANNOT and SHOULD NOT be allowed to run on "freeware" or even "open source" of any kind.

Once I left avionics and worked in "commercial" venues I was appalled at the so-called "QA" that I saw. I was further shocked to find that QA departments are often the first to suffer from layoffs.

Freeware and open source does not have the kind of rigorous testing and version control that flight systems have. It is literally unaffordable. The number of logic paths to be tested are astronomical! Alpha and Beta customers are relied upon to wring the major bugs out, and it still takes numerous bug fixes after that. Witness the number of bugs in linux, java or open solaris that migrate to a later revision. In a flight system that is absolutely intolerable. Lives are at stake.

Remember the Intel Arithmetic chip misfunction? If used in a flight system, that could have caused a crash. Or how about the infamous US Navy Missel frigate that had converted to Windows NT to run the entire ship's system - they experienced the "blue screeen of death" and was "dead in the water" for over four hours off the coast of Virginia ? Because of that debacle, the Navy quietly removed it, declared their "off-the shelf product" project a success, and shelved it indefinitely. And those were merely "commercial products" (rather than open sourced) that had not been rigorously tested.

>Given the implication that happiness, value and quality are only available for products designed using commercial software; I find myself retroactively saddened by all of the things that I have seen which were designed before commercial software packages were available. I would like to thank Ms. Milley for sucking the joy out of my life.

I fear you are confusing the issue for the sake of a "light comment" to try to put "open systems" in a better light. It is not uncommon for those who don't truly understand these kinds of "mission-critical, life-critical" issues.

Until the incredibly costly processes, testing, and process controls of my prior experiences are put in place on "open systems" I will gladly "suck the joy out your life" in return for a more rigorous tested and safer jet engine.

and please don't try to use as examples the "success stories" at hospitals running "open source" . Networks that can be rebooted on failure are not in the same league, and any good network/server design in such an application will use clusters with failover and a "trailing edge" O/S version that has been sufficiently demonstrated to be reasonably bug-free.

Even the MS based London Stock Market was down for 7 hours .... I would love to see Torvald getting a 3am call from some banking CEO because the "freeware-based" stock market servers are down....

Posted by oldguy on January 07, 2009 at 06:08 AM EST #

> Any system, software or hardware, on which any failure results in loss of life is a clear 7x24 mission critical system . This is a system that \*cannot\* be allowed to fail. This concept, while repeated and lip service paid, is clearly not understood by many.

I have no reservations in agreeing to your definition of life critical systems or that many uses of mission critical do not describe life critical systems.

My assumption is that an engine which is built by competent engineers and tested for life critical operational parameters will be functional, without regard for the commercial or free nature of a piece of software that was used in its _design_, not in its operation. "The R Project for Statistical Computing" is a free software environment for statistical computing and graphics. It serves a significantly different (non-life critical/non-operational) function than C in your examples.

> Because of the nature of the work, any bugs or problems in the tools were escalated immediately to the vendor and fixed in a matter of days, if not hours, at the highest priority. That was possible because it was purchased, with further purchased support contracts. The level of support we required is simply not available for Linux.

Note that I did not say that I would be happy flying on a plane with avionics software based on Linux (I would not). I believe that I would be comfortable with custom built open source real time software/hardware solution which had been rigorously tested by third parties as you describe.

> Freeware and open source does not have the kind of rigorous testing and version control that flight systems have. It is literally unaffordable.

> Until the incredibly costly processes, testing, and process controls of my prior experiences are put in place on "open systems" I will gladly "suck the joy out your life" in return for a more rigorous tested and safer jet engine.

We remain in agreement. I did make a "light comment" although my aim was less at trying to position open systems as the next generation of flight (life critical) systems so much as I responded to what sounded like a throw away FUD comment in an article about a free software application.

> Even the MS based London Stock Market was down for 7 hours .... I would love to see Torvald getting a 3am call from some banking CEO because the "freeware-based" stock market servers are down....

Having worked in operations and been on the receiving end of some of those calls I would too.

Posted by Shawn Ferry on January 07, 2009 at 09:09 AM EST #

Thanks Shawn, for taking the time to read & reply!
I guess we are actually in agreement in many ways :-)
I actually embrace open source software for it's diversity in the contributing community (many geniuses are hiding out there) and the very concept of openness and sharing. I feel it's biggest drawbacks remain in "quick response to bugs"; "who's going to pay for fixes"; "achieving consensus"; and the inevitable "who pays for development". If only "cool" projects are developed, then what happens to "badly needed but uncool" projects?

Posted by oldguy on January 08, 2009 at 07:55 AM EST #

Post a Comment:
Comments are closed for this entry.
About

yakshaving

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
Sun Managed Operations