Friday Dec 05, 2008

Lively FX

In case you hadn't noticed, Sun yesterday announced Java FX. Its pretty cool stuff, no doubt, but I'd like to direct your attention to a little piece of this system that comes from Sun Labs.

 Projects Lively Lively-Screenshot-SmallThere is a project in labs called the Lively Kernel which is blazing new trails in dynamic web programming. Its a really cool jaunt into self reference and meta-circularity. The entire system runs in a web page. That web page includes a complete development environment which is used to create everything you see on the web page, including the tools used to create/modify itself. In other words, when you bring up the web page, it includes tools like browsers and editors that allow you to change what is on the page... like the browsers and editors etc etc etc. Lots of fun, but mostly incredibly flexible. For the old school among you, think of this as the SmallTalk style of web programming (this may be because Dan Ingalls helped invent both SmallTalk and this). It really makes you think about web programming differently. The project description says:

The Lively Kernel supports desktop-style applications with rich graphics and direct manipulation capabilities, but without the installation or upgrade hassles that conventional desktop applications have. The system is written entirely in the JavaScript programming language, a language supported by all the web browsers, with the intent that the system can run in commercial web browsers without installation or any plug-in components. The system leverages the dynamic characteristics of the JavaScript language to make it possible to create, modify and deploy applications on the fly, using tools built into the system itself. In addition to its application execution capabilities, the Lively Kernel can also function as an integrated development environment (IDE), making the whole system self-sufficient and able to improve and extend itself dynamically.

Those of you who are paying attention will notice that I started out talking about JavaFX, but Lively Kernel is based on JavaScript. OK, so what does this have to do with JavaFX? It just happens that they have taken the entire system and integrated it with JavaFX. This means that from this flexible, "lively" environment, you can manipulate JavaFX programs. Pretty cool stuff.


As far as I know, the JavaFX version is not yet available outside of labs, however, the JavaScript version is. For those of you with very recent versions of Safari, IE or Firefox, you can try a this link to check it out live.

Thursday Jul 10, 2008

Distributed Storage Open Source - Celeste

I am a avid digital photography enthusiast. I have over 100,000 pictures in my library. With my current digital SLR camera, a single raw image can be 14MB, and I can generate them at 5 images per second. I've done a few HD video editing projects, and as you can imagine, those also eat gigabytes of disk space like it's popcorn. Managing that much data causes significant issues for me as the household's resident IT manager. I have to keep it all backed up and available for my family. We have several terabytes of spinning rust in our home. Recently, as we left on a week long vacation, our neighbor asked, "if these California wildfires find our neighborhood while you are gone, is there anything that you want me to save." Given that he probably can't save the piano, it was easy for me to ask him to grab the backup storage device, with years of photos and records on it.

Of course, I also work with sensors, devices who's main purpose in life is to generate data. We are generating data at an incredible rate. All that data needs to be stored, so storage becomes more and more important. That's why Jonnathan is blogging about it and why we are announcing a slew of new storage products. Well, we at labs wouldn't want to be left out. That is why I'm happy to announce the availability of Celeste, an open source research project aimed at reliable, secure storage made of unreliable, non-secure parts, that scales beyond imagination.


Celeste is a very interesting set of technologies. It is a distributed object store. That means that when you store an object in a Celeste system, it is split up into chunks that are stored on various machines spread across a network. The system hands back a handle, which later can be used to reconstruct the object. Because these chunks of data can be replicated or encoded for redundancy, the storage can be very reliable. This also has interesting implications for security, since each participating node knows only about the chunk of data it is asked to store. It does not know who stored it or what larger object that chunk of data belongs to. The system is designed to handle unreliable or even rogue nodes in the system. In addition to checking the reliability of the data as it is retrieved, nodes can keep track of the reputation of other nodes in the system. This means that when you save an object, you can ask to save it someplace VERY reliable or someplace with a good reputation for fast access. The society of nodes in the Celeste system use the reputations that are built up over time to match with the users wishes when storing future objects.


The system is alive. When left alone it will continually monitor itself to make sure that the proper levels of redundancy are maintained as nodes come and go from the system. This has the effect of sloshing data around the system to level storage over time. This has an additional very interesting side-effect. Say you have a data center and you decide to buy one of our cool new storage boxes. If you could plug it in and tell it to start participating the the Celeste system, data would naturally start flowing into the new storage units. If you have some old storage boxes that you would like to retire, perhaps because they just draw too much power for the amount of data they hold, you can just turn them off. When you do this, the Celeste system will notice that some data has disappeared and perhaps the redundancy levels have fallen below the required threshold, so Celeste will start creating redundancy coded chucks of data, most probably in your new storage system. This means that your data will automatically migrate to the new storage hardware just by connecting the new storage and turning off an old one.

The heart of the system is the Distributed Object Locator (DOLR). This is based on a Distributed Hash Table (DHT), which is a very slick piece of technology. It allows a completely masterless set of nodes to work together to find any piece of data in linear time. This system as implemented in Celeste can scale incredibly. It can address yottabytes of information. Yes yottabytes - now you can tell your friends that you learned a new word. kilo, mega, giga, terra, peta, exa, zetta, yotta. A yottabyte is 1,000,000,000,000,000,000,000 (1000\^8) bytes. That should be able to store all my photos with no problems.

Another interesting aspect of the system is that it is designed to be able to handle drastic failures. For example, suppose that you have a Celeste system that consists of 1000s of nodes spread across the United States. If someone digging a trench in St Louis, severs your main OC-193 cable between the east and west sides of the country, the system will continue to operate as two separate Celeste systems. Stores and retrieves can go ahead as normal. Eventually, when the damage is repaired, the system will stitch itself back together including flagging potential conflicts where two independent writes have been carried out on a single object. Additionally, a side effect of the way that the data is looked up in the DOLR is that data naturally migrates closer to where it is being used. This means that if you are accessing a lot of data and then travel half way around the world, the data can follow you to ensure speedy access from anywhere. In interesting side effect of these two behaviors is that one can imagine a scenario where your laptop is participating in a Celeste system. As you use data, it finds its way to your laptop. At the end of the day, you take your laptop home and work disconnected from the network. You read and write data locally on your hard drive. The next day you come in and re connect to the Celeste system and your data is seamlessly reintegrated.

A key differentiator between our system and other distributed storage systems is that Celeste is mutable. These distributed storage systems go to great length to never forget anything, so often they are designed to NEVER forget anything. In many cases this is not appropriate for real world use. Nothing is ever deleted and storage is never recovered, so these systems just grow and grow. We have many customers who wrestle with regulations that require them to keep customer records for a certain length of time and then guarantee that they will get rid of them. Celeste includes some secret sauce that allows it to really forget data. You can delete data, change it, etc. just as you would on a standard file system.

There are many other aspects like interesting security modes and possible modes of disconnected operation. This system is a research tool and not yet part of any product, so don't expect to just plug it in and replace you existing file system, however, if you are interested in this type of research I encourage you to check it out. The project is entirely open source.

You can learn more about the Celeste project here:

Congratulations to Glenn and Glenn for getting this great technology out to the world!

Friday Feb 29, 2008

Congratulations Susan Landau, Woman of Vision

One of my colleagues here in Sun Labs, Susan Landau has just won a significant and cool award. Congratulations Susan!



2008 Woman of Vision Award winner for Social Impact

Susan Landau, Distinguished Engineer, Sun Microsystems Inc., is the Anita Borg Institute 2008 Women of Vision Award winner in the Social Impact category.

Landau’s focus is on the interplay between security and public policy. She has profound impact in at least three areas of computer science: as an extensive commentator and advisor on U.S. wiretapping and encryption policy; as a world renowned expert in computational algebra and number theory (mathematics intimately related to cryptography), and in developing numerous programs to benefit women in computer science. A Sun Microsystems Distinguished Engineer, Landau is a leading scholar in all three areas and publishes widely. Her book, co-authored with Whitfield Diffie, Privacy on the Line: the Politics of Wiretapping and Encryption, attracted immediate international attention and played a significant role in the 2000 loosening of U.S. cryptographic export-control regulations, stimulating the global technology economy and offering protection to consumers in all non-embargoed countries. Her unusual blend of technical expertise, policy insight, industry connections and drive, along with her dedication to the advancement of Women in Computing, make Landau a true Woman of Vision.

More on the 200 Women of Vision Award Winners:

Friday Oct 26, 2007

Peering inside the labs

My entries in this blog have been focusing quite a bit on Sun SPOTs, but that is only one project here in Sun Labs. I plan to discuss some other projects that you all should know about in the near future. In the mean time, for those of you who are curious about what we do in Sun Labs, there is a new web page here that does a nice job of capturing what we are up to, or you can take the virtual tour. Enjoy...

Wednesday May 30, 2007

The Aspen Movie Map

It is my theory that most good ideas for technology come from artists, hobbyists and students.

A friend, Michael Naimark, created an amazing art project at MIT back in 1978-1980 called the Aspen Moviemap. He used a 16mm film camera to take images in and around Aspen Colorado, transferred them to Video Laser Disc (remember those) to allow a viewer to have the experience of wandering through Aspen. At each intersection you could turn in any direction you chose. From any position, you could look in any direction. It was a truly an impressive piece of work back then. Even today, its impressive to see.

We, if imitation is the sincerest form of flattery, Michael should feel pretty good. Yesterday, Google unveiled a new feature to called Street View, which allows you to wander through the streets of Palo Alto, CA and other areas in much the way that Aspen Moviemap did nearly thirty years ago. While Google's implementation is a little rough around the edges, it has the advantage of being network accessible and having the incredible resources of Google behind it, so it has already grown beyond the scale of the Michael's original project. However, the original inspiration is clearly still in there.

Congratulations Michael!

Sunday May 27, 2007

Hanging around with smart people

My name is Roger Meike. I'm Senior Director of Area 51 and Assistant Director of Sun Labs (the research arm of Sun Microsystems). This means that I work with the geniuses who brought you Java and other ground breaking technologies. We are tasked with providing the future vision for an already forward looking company. I can tell you that the researchers in a world class institution like Sun Labs are all top notch. To a person, they are brilliant, committed and skilled. I consider myself to be an intelligent person, but this place certainly can be humbling. Sometimes it's a challenge just keeping up.
I used to work in the music industry on one of the first digital audio workstations. It was there that I learned an old musician's joke that I like. "Drummers are people who hang around with musicians." Since I play keyboards, I found this rather humorous. Here in Sun Labs I feel more like the drummer. Managers in a world class research institution like this are people who like hanging around with smart people.

Join me on this blog as I relay the work of some really smart people and give you a ringside seat to watch real innovation as it happens.


Roger Meike, Senior Director of Area 51 and Director of Operations Sun Labs


« July 2016