Fortress Wrapping Up

After working nearly a decade on the design, development, and implementation of the Fortress programming language, the Oracle Labs Programming Language Research Group is now winding down the Fortress project. Ten years is a remarkably long run for an industrial research project (one to three years is much more typical), but we feel that our extended effort has been worthwhile. Many aspects of the Fortress design were novel, and we learned a great deal from building an interpreter and an initial set of libraries. Nevertheless, over the last few years, as we have focused on implementing a compiler targeted to the Java Virtual Machine, we encountered some severe technical challenges having to do with the mismatch between the (rather ambitious) Fortress type system and a virtual machine not designed to support it (that would be every currently available VM, not just JVM). In addressing these challenges, we learned a lot about the implications of the Fortress type system for the implementation of symmetric multimethod dispatch, and have concluded that we are now unlikely to learn more (in a research sense) from completing the implementation of Fortress for JVM. We also note that, over the last ten years, other languages (Chapel, X10, Clojure, and Scala, among others) have explored some of the same issues that Fortress has addressed, and we have very much enjoyed conversations, collaboration, and friendly competition with those who have explored these ideas in alternative contexts.

The Fortress source code remains open-source, and the code repository will remain available for the foreseeable future to those who wish to work on it. The Programming Language Research Group will continue to respond to inquiries about Fortress (and requests for minor bug fixes as we can). We not do not plan to cease work on Fortress suddenly, but will spend the next few months getting the code and language specification into the best shape that we can. We also plan to write several academic papers about various aspects of the Fortress design and implementation technology. Going forward, we will look for opportunities to apply lessons learned and to transfer Fortress-related technology to other projects.

We thank everyone who has expressed an interest in or support for Fortress, especially DARPA (which funded in part some of the early work as part of its High Productivity Computing Program) and its HPCS review committees; former Programming Language Research Group team members and the many interns who contributed to the design and implementation effort; the management of Sun Microsystems Laboratories and Oracle Labs; and those who became part of the open-source effort, downloading the implementation, trying it out, and asking questions that kept us on our toes.

Here are some of the aspects of Fortress with which we are quite pleased, with commentary:

Generators and reducers
This has turned out to be a powerful way to organize collections classes and their use. It is related to the idea of "map-reduce" but is a bit more general. Moreover, comprehension and "big operator" syntax allows programmers to write code that sometimes focuses on a single instance rather than having to adopt the always-global point of view required by the use of higher-order operators such as map and reduce; this is often a useful stylistic alternative.

Implicit parallelism supported by work-stealing
We have found work-stealing to be an effective implementation mechanism for parallelism on shared-memory processor clusters, and especially for a style of programming in which the compiler can implicitly extract parallelism because function arguments and tuple elements are by definition assumed to be computationally independent.

Nested atomic blocks supported by transactional memory
We believe that this is a powerful and expressive alternative to locks for expressing synchronization among threads in a shared-memory environment.

Parametrically polymorphic types that are not erased
There are advantages (we're talking performance here, not just aesthetics) to being able to inquire, dynamically and cheaply, what is the element type of a collection in order to select an appropriate implementation.

Symmetric multimethod dispatch and parametrically polymorphic methods
Avoiding the need for the "visitor pattern" makes code more flexible and easier to extend; moreover, delegating dynamic dispatch on arguments other than the receiver object to the runtime allows greater opportunity for dynamic code optimization (an area that deserves further exploration). We made excellent headway on elucidating the interactions of this feature with polymorphic types and polymorphic methods.

Multiple inheritance, inheritance symmetry, and type exclusion
Like most modern object-oriented programming languages, Fortress supports multiple inheritance among object types. Unlike most other object-oriented programming languages, in Fortress inheritance is completely symmetric. This led to a need for the programmer sometimes to indicate explicitly when two types are intended to be disjoint (that is, no object can belong to both), and constitutes an interesting and unusual aspect of the Fortress type system.

Mathematical syntax
The idea of "typesetting" the code using LaTeX has received mixed reviews, and many potential users find the issue of keyboarding the extended Unicode character set daunting. Nevertheless, even in its more verbose ASCII form, the syntax of Fortress, especially the notion of treating juxtaposition as a user-definable operation, has proved to be very convenient.

Components and APIs
The Fortress component/API system allows control over namespaces in an unusually fine-grained manner; in particular, we figured out how to allow exporting only part of a set of overloaded functions or methods while correctly maintaining their execution semantics and supporting separate compilation.

Here are some aspects we wish we had been able to explore further (and may yet, in some future project):
Dimensions and units
We designed a way to write such expressions as "2 meters per second" and "15.7 kg" and have them fit smoothly into the rest of the type system and mathematical syntax—still a good idea that has yet to reach fruition.

Explicit descriptions of data distribution and processor assignment
Parallel programming languages still lack a good way to represent complex hierarchical hardware resources so as to allow programs to depend parametrically on these resources in even a moderately portable fashion. We had a paper design for this but did no implementation work.

Conditional inheritance and conditional method definition
We wanted to express such ideas as "a list of elements of type T implements a (lexicographic) total order provided that T implements a total order" and "the binary OPLUS operator applies element-wise to vectors provided that OPLUS is a binary operator on the vector elements, and moreover vector OPLUS is associative (or commutative) provided the OPLUS on the elements is associative (or commutative)". We designed "conditional 'where' clauses" as a notation for expressing such ideas, but never got to implementing them.

There is much work yet to be done in designing and implementing programming languages, and we believe that many of the ideas mentioned above will be important, in some form, in future language designs.

We thank you for YOUR interest in Fortress, and hope that you will be similarly interested in our future efforts in other areas.

—Guy Steele, for the Oracle Labs Programming Language Research Group

Comments:

Very sad to hear this. Fortress is the most promising language I know of.

Posted by Olle Kullberg on July 20, 2012 at 04:38 PM EDT #

Hi Guy,

this is somewaht sad news for me. I was very inspired by the work on Fortress. The mathematical notation sounded very reasonable to me, also the functional and extensible properties of the langague is something I appreciated not to mention that the implementation was open source and your group built up a community around the project!

I hope we see more interesting ideas evolving into a next generation langauge. In Peter Seibel's Coders at Work you mentioned that you were very attracted to Haskell. Do you need more symbolic power in the VM to support the type system you mentioned? What will be the next language research project?

Best,
Jakob

Posted by Jakob on July 21, 2012 at 05:54 PM EDT #

Thank you for all the excellent work. Fortress is the most interesting programming language I've ever seen and it have taught me a lot about parallel programming. I believe that In time we will see all of then great ideas in Fortress being implemented in other languages, just as we've seen with Lisp.

Posted by Thorbiörn Fritzon on July 21, 2012 at 07:11 PM EDT #

I'm sad to hear this, but your open approach to acknowledging and documenting pain points of implementing the language is an inspiration. I'm hoping we parallel library developers can learn something from all this too. Thanks for your hard work!

Posted by Mark Hoemmen on July 23, 2012 at 08:06 PM EDT #

Very sorry to hear this, simply because the language looked both principled and elegant, and we need more such languages in the software world.

I infer that the implementation hindrances to building Fortress on the JVM meant that it was no longer interesting to Oracle to fund Fortress development. I eagerly await your next JVM-centric language research projects.

Posted by Rob Jellinghaus on July 23, 2012 at 08:24 PM EDT #

I'm curious about your thoughts on InvokeDynamic. Were you investigating it at all as part of Fortress, and could it help with the impedance mismatch you describe between Fortress's type system and the JVM?

Posted by Tony Arcieri on July 23, 2012 at 09:35 PM EDT #

A truly sorry turn of events. I hope the other projects, X10 and Chapel, find your research useful to enhance their own development efforts.

In particular, the dimension/units system, orthogonal to the type system, is a great step forward in supporting safer computing in the physical realm. As I have seen from examples, it puts no great burden on programmers but, instead, provides a way for programmers to think about what they're doing. It also offers a compiler mechanism the chance to check the programmer in another way.

I hope this research does not go away, but is adopted by other languages!

Posted by Rod Oldehoeft on July 26, 2012 at 10:46 PM EDT #

This announcement was (to me) surprising and a real shame, but I was glad to read a very positive review of Fortress here: http://www.sdtimes.com/blog/post/2012/07/24/Fortress-Finally-Folds.aspx

Congratulations to the team for their work! I'm sure there are many people like me who've been quietly following Fortress over the years, biding our time for it to be ready to be used productively. It seems to be the ideal language for many purposes: avoiding legacy and compromise in every area, simple, strict, clear and so on. (Even a great language like Scala can't tick most of these boxes.) And I say this quite apart from the original purpose of the language, which is scientific computing and high-performance at that. I'd want to use it for everything.

I don't know if this is realistic or not, but let's hope the open-sourced Fortress has a new lease of life so that in time it can be completed.

Posted by guest on August 13, 2012 at 11:32 AM EDT #

Thank for your hard and dedicated work! I would really love to see me, and majority of programmers use this beautiful language.

Maybe it will find another platform (other than JVM, if any?) to get implemented on? I hope it will.

Nevertheless, it was an exciting process to get hands on with interpreter experience, waiting to become a matured language.

Thank you again!
Ceyhun

Posted by Ceyhun Can ÜLKER on October 02, 2012 at 05:32 PM EDT #

Mr Steele - I just saw the video of your talk on future parallelism, and your comments about "moving 90% of the way from Fortran to Haskell" caught my attention, especially with your mention of functional programming and the importance of immutable data for parallel processing. Taken together, your comments sounded almost like a description of Clojure, which shares most of the same goals you described for Fortress.

If you haven't seen it yet, there is a nice demonstration of these ideas in action at: http://blip.tv/clojure/clojure-concurrency-819147

Alan Thompson

Posted by guest on March 12, 2013 at 12:57 PM EDT #

Hi,

Any plans of donating the code to Apache or Eclipse?

Suminda

Posted by Suminda Dharmasena on May 29, 2013 at 07:33 AM EDT #

It is a shame this project is not continued else where.

Posted by Suminda Dharmasena on May 29, 2013 at 07:35 AM EDT #

We have not made any arrangements yet to transfer the code to some other entity. However, the code remains available as open source, and I would be delighted if someone else wants to use it or extend it.

Posted by Guy Steele on May 29, 2013 at 11:04 AM EDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Technical discussion of the Fortress programming language and its applications

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today