Planets, Weapons, Dolls and Fallacies
By stern on Feb 21, 2007
The planet is Neptune, also known as Sun's 10G Ethernet network interface, described in the latest Innovating@Sun podcast by Distinguished Engineers Erik Nordmark, Shimon Muller, and Ashley Saulsbury. It's quite a slick piece of engineering, ensuring that higher level operating system abstractions (like virtual interfaces) have hardware support, and conversely reducing the processor performance tax typically paid for a high-speed network interface.
Crossbow is the weapon, and it's the OpenSolaris project that creates virtual interfaces and virtual networking stacks, allows you to do flow management through those stacks, and is cognizant of the hardware support provided by Neptune.
Ariel Hendel has a great blog entry drawing a parallel (literally) between the Neptune and Crossbow network virtualization and a very deep set of Russian nesting dolls. Ariel makes a number of astute observations about parallel and serial data transfers, and how some of our historical thinking about what is serialized and what runs in parallel data transfer lanes has evolved with I/O and memory interface standards.
The bottom line is that efficient, scalable networking helps reduce the overhead of building distributed applications. Face it: every application today is distributed, unless you're off on a laptop in some airport without network access, in which case you wish you had distributed applications. Everything that's interesting is networked, and when you start to aggregate input, output, and data management streams the 8 fallacies of distributed computing become critical scaling rate-limiting factors. The combination of Neptune and Crossbow don't eliminate physical realities like transport cost or latency, however, their impacts are reduced through network stack processing efficiencies. Demands for, and benefits from, this combination are statistically likely.