BPS visits ORNL

Do you remember our visit to SuperComuputing 2010? Today we have a follow up on its events.

Seeing as how Chinese supercomputer dethroned US as top #1 supercomputer in the world (a position US maintained for many years) and also captured #3 spot with another supercomputer, US started to make the plans to regain the leadership in this area.

Of course this is an area that BPS is interested in as well. So Akira decided it's time to fulfill the SC10 promise and visit ORNL in person.

BPS and James
Warm welcome awaited at the Lab

BPS and James

After finishing with formalities, it was decided to walk around ORNL supercomputer resources and see what's available and
see if perhaps Shirase-sensei has any ideas for possible improvements and otherwise familiarize himself with the gear.

BPS and IBM BlueGene/P
Older and smaller IBM BlueGene/P supercomputer is pretty tiny by today's standards. This sort of tilted design is due to
improved air flow inside the case. One big benefit of IBM supercomputers is the relative small amount of power they need and
heat output. This is a pretty big deal for places with expensive power like California, so a lot of places in California really sticks
with IBM gear for most of their operations even though there are other faster offers available on the market.

On the other hand Tennessee has much cheaper electricity and so those other offerings are much move viable as a result.

BPS and Kraken
Kraken supercomputer is so far second fastest at ORNL, it's a Cray XT5 architecture, there are 4 rows of cabinets like this filled
with processors and memory. The machine actually belongs to University of Tennessee and was the first academic supercomputer
to surpass petaflop barrier. Currently sitting at #8 spot in the list of supercomputers with a peak speed of 1.17 petaflop/s
it consists of 9,408 compute nodes each equipped with a pair of 6-core AMD Opteron CPUs. 147 Tb of system memory.

Behind the Kraken stands another supercomputer, the current #2 supercomputer in the world Jaguar

BPS and Jaguar-PF

BPS and Jaguar-PF
It's a nice graphic on the front of it, so a pity it can no longer be easily seen.

BPS and rows of CrayXT5 cabinets
There are additional 8 rows of cabinets for Jaguar and it belongs to ORNL.
Very much similar in architecture to Kraken (also Cray XT5), Jaguar is just bigger version, it has 18,688 compute nodes, 300Tb memory.
Peak performance is 2.3 petaflop/s.

Both of these supercomputers have reached the peak, though and won't be extended anymore.

A new supercomputer called Gaea is currently being installed, based on Cray XT6 architecture

BPS and Gaea
Only just a few cabinets were installed so far of this system. It sports improved room-neutral water cooling.
Interesting detail about the front panel picture: there is no such place on Earth, it's simulated.

BPS and Gaea cabinet
A couple more Gaea cabinets at the other side of the room.

This one is not going to be big and is primarily targeted at climate research.

Gaea will take the rest of available space on the supercomputing floor it seems.

The new supercomputing monster is still planned, though. It will be named Titan and will boast over 20 petaflop/s
in computational power. A new building is being built for it and installation will begin somewhere at the end of this year.
So perhaps another visit is in order to inspect the new machine too.

Another area where ORNL facility shines in is computer storage. While historically supercomputers have been shipping
with their own storage, when you pack many supercomputers nearby like happens at ORNL, this is a big waste so they
went ahead and created an external storage array that all of their supercomputers (present and future) can use.
Currently it offers over 10 petabytes ( 10,000 terabytes) of usable disk space mostly on DDN arrays.
This storage is also one of the biggest in the world.

BPS and DDN arrays

BPS and DDN arrays
There are many-many rows of cabinets that look like this full of hard drives.

BPS and operator table

BPS and operator table
Time to login to the system and see how it actually performs.
Turned out that Jaguar still performs remarkably well.

Control room where system operators sit

BPS in ops room

BPS in ops room

BPS with laptop
A much nicer place to work at than the supercomputing floor and much quieter. And you can still see the supercomputers through
the big window.

BPS and hand sanitizer
And so the day is over and so is this visit. Lots of useful discussions have been held. And now it's time to wash the hands
and go home. Thank you very much Purell-san.