Peak Code

Hubbert's theory on peak oil (and other geological resources) states that the maximum rate of production of the resource occurs when half of the reserve has been extracted. Initially, there is near exponential growth in the rate of extraction as lots of "easy oil" is found and pumped, but after the peak the rate of extraction decreases. The marginal cost of extraction goes up as less desirable source of oil are put into production; this cost should provide economic incentives to consider alternate energy sources.

I think the evolution of programming environments has some similarities to Hubbert's theory. A successful new programming environment enjoys an early inflationary phase where lots and lots of new code is being written in it. As the platform matures, the significant programs that are "easy" to write in that language get written, often code less practical to write elsewhere. As the platform grows, increasing interactions and compatibility constraints make adding new features more challenging. At some point, half of the total lines of code for that platform will be written. The further a platform gets past its peak, the less attractive adding new features to it becomes since there is less and less potential return on investment due to the dwindling number of programs that could benefit from the improvements.

However, there are some important differences between potential programs that can be written and natural resources. First, old programs are not consumed or made unavailable in the same way oil is when refined to fuel, made into plastic, and so on. Second, the constraints on the production of new code are very different from those in isolating a chosen natural resource; the primary limiting reagent to producing new programs is not a physical limit. In chemistry, a reaction takes places between fixed ratios of molecules so if you don't have enough of one kind of reagent molecule, say oxygen, the unavailability of that reagent limits the reaction no matter how much of the other reagents, like gasoline, are available. However, the amount of code that gets written will be capped by human interest or economics before hard physical resource limits come into play. Hubbert's peak applies to finite resources; in contrast, the potential amount of software is effectively infinite for the purposes of this kind of modeling. Additionally, there are difficulties in accurately measuring the rate of program production. These differences mean that it can be impractical to know the precise rate of new program development and even if that rate were known, the life cycle of the platform could follow a very different curve then Hubbert's. Therefore, if there is a peak, is is hard to know whether or not it has occurred and the timing and size of any peak can be influenced by changes in the platform.

As volume drives value, evolving a general-purpose programming platform should try to enable a wide range of applications to be written in a convenient and safe manner, expanding over time. Java's relative simplicity, garbage collection, and rich libraries were early advantages; more recently the language changes of generics and annotations have expanded the set of programs that are tractable to write on the platform. As the Java approaches its teenage years, we'll aim to keep improving the platform to support more kinds of programs in JDK 7 and beyond; perhaps closures will be the next major enhancement, allowing programmers to abstract over another axis.

Thanks to Alex for the conversation which led to this post and for providing feedback on earlier drafts.


Interesting idea. I think you could say that the emergence of AOP was another axis of abstraction that let people expand what they wanted to do in Java. When I first started reading your post my initial thought was that you were talking about code in the space of a particular software project/system, which I think holds up even better as an analogy. Basically, lots of code gets written initially but a lot of it is not well-factored. At some point, you hit a maximum efficiency point where the scaffolding you've built so far helps a lot and you don't have much other code to slow you down. But after that point the weight of the existing system starts to slow down the efficiency with which you can add additional features or transform your frameworks. Eventually, if you want to continue evolving the system that weight will crush you (see: Windows) or you must reinvent yourself, largely by starting over (see: Mac OS, Netscape/Mozilla/Firefox). Or ideally, you can make a system so amazingly built for change that there is no weight to crush you in the first place (see: emacs? maybe RoR?). Maybe I should expand this idea more into a blog...

Posted by Alex Miller on March 02, 2007 at 02:29 AM PST #

Hubbert's original theory is supposed to be applicable both to individual oil field as well as to global oil production. I agree it would be relevant to smaller projects too; although individual projects may have an explicit goal to get past their peak and toward final resolution :-)

Posted by Joe Darcy on March 02, 2007 at 03:16 AM PST #

Post a Comment:
Comments are closed for this entry.



« August 2016

No bookmarks in folder