In my early days of being a Computer Engineer designing CPUs, I was always fascinated by speed. The speed of switching of transistors, the delay through a gate, the size of its pipeline, multiple arithmetic units, parallel pipelines and then the last thing I got to work on tightly or loosely coupled clusters, the foundation for modern supercomputers.
My engineering days are long gone, but it's memorable not only thinking back to those days but also where CPUs and computing have come. It is too broad a topic to cover in one blog post, but here we will look into what you can do with a $35 computer, the Raspberry Pi. For a while, the record of taking this commodity and inexpensive processor and making a beast out of it was held by Los Alamos National Labs with a 750 Raspberry Pi Cluster.
Fast forward to 2019 Oracle Code One, and we have the world's largest Raspberry Pi Cluster. Below is an edited Q&A I did with Chris as well as a more in-depth video interview:
Q1. What are the main specs that you can share, in terms of performance, size, power?
A1. 1024 Raspberry Pi (actually there’s 1050 but we disabled 21 to get a better number). There are 21 Pi per 2U and there are 21 * 6 * 8 + 42 Raspberry Pi. That makes 4096 1.4ghz processors with 1024 gigs of RAM. We have not done a speed test yet.
Q2. Can you share more about the architecture of the Raspberry Pi cluster in terms of the hardware and software?
A2. We went with a Ubiquity network because it has an amazing graphic interface for setting up the network. We used 32 48 port 10G switches and the gateway. So it’s a very fast network “professional”. Since there are more than 255 devices on the network we had to setup a 255.255.0.0 subnet mask. For those familiar with networking this will be old hat but those of us who just plug in a device into our home network and get 192.168.*.* this get complicated fast.
The server is running Oracle Linux 7.7, network booting every single Raspberry Pi with Oracle Linux 7.7 and running an in-memory file system. Because of that, we couldn’t run Docker because Docker runs an in-memory file system and apparently you can’t run an in-memory file system on an in-memory file system. So we turned to Gluon to build a community-based Java application.
We recently build a Raspberry Pi Mini Super Computer with 84 Raspberry Pi that is running Oracle Linux 8.1 and network booting Oracle Linux 8.1 on the Raspberry Pi, running Java and SETI@home searching for aliens.
Q3. Just for the record can you share the major differences between Raspberry Pi and Arduino as well as Xeon?
A3. Arduino is for embedded projects and you program the microcontroller typically with C/C++. They are really good at doing one thing and only one thing. Raspberry Pi and Xeon are general-purpose processors, ARM or x86. They run an operating system like Linux or Windows and can run multiple programs at the same time, typically have more RAM and a faster processor.
Q4. I am assuming based on its cost/performance that this Raspberry Pi supercomputer doesn’t make it to the Top 500 Supercomputer list, but what kind of applications could it potentially run?
A3. No, it is fairly small on the supercomputer scale, but in the world of Raspberry Pi it is a Raspberry Pi Supercomputer! We have the ability now to run Docker which will make it so much easier to run applications. We’ve run SETI@home, a custom Java application driving a video wall, but it could run anything. This many general-purpose processors, unlike a GPU, lends itself to running simulations. For example, an ant colony where each Pi runs a dozen ants.
Q5.There are off the shelf components you purchased, what are the main ones and what parts did you eg 3D Build?
A4. The USB power supplies, network switches, 90 degree right angle USB cables, aluminum racks, 1 ft network cables and surge protectors were all off the shelf components. The custom components were the actual box made out of aluminum extrusion, the caster mounts at the bottom, all the mounts for fans and you can download and print your own 21 Pi rack .
Q6. Imagine if you could slow things down and see how a problem is broken down and then an answer provided using Augmented Reality, what would it look like?
A5. We have had thoughts of adding AR to the Pi cluster. Right now to see the stats on an individual Pi you have to know the IP or MAC address and look it up in log files or we can visualize it on a big monitor. But sifting through the data is difficult. This is where IoT and AR would be useful. Imagine a tablet with AR when you point the camera at the Pi cluster could overlay the stats of each Pi on top of the Pi. There would be no typing, looking at log files or graphs or tables, just point and there’s the data you need. Those are the kinds of things we do with data here at Oracle.
Q7. Records are inevitably broken. In the world of clusters is there a limit to how many Raspberry Pis you could bring together? Would you hit any limit eg Moore’s or Metcalf’s law in terms of processing power or network capacity?
A6. Space, power, networking and heat are your limiting factors and how you run into them would depend on your requirements. Our main requirement was portability. The rest was simply working the problem and iterating.
Q8. Where is next in your plans or ideas that you would like to explore with the Raspberry Pi Supercomputer?
A7. I would like to explore AR and I would like to see what the community has that would be interesting to run. I am the kind of person that likes to build things that enable other people do amazing things. That’s why I’ve worked on developer tools and APIs most of my career. If we could run something that would be meaningful like search for a cure for cancer I would love that.