Quantum Diaries

In a previous post, I mentioned that ATLAS will be collecting enormous amounts of data, approximately 6 Petabytes/year (i.e., 6,000,000 Gigabytes). How in the world are we going to handle it, and how do we make it available to all physicists on ATLAS? I spoke to one of my colleagues at Indiana University, Fred Luehring, who has major responsibilities for the US part of the Grid, to get some details.

First, let me mention some ATLAS jargon; if you wish, you can skip this paragraph for now, and come back to it when you run into strange acronyms.

The raw data that we collect is called ByteStream, basically, it is a stream of 1’s and 0’s, and is approximately 3-6 Megabytes/event. This gets “massaged” into Raw Data Objects (RDO); the only difference is that the data now has a “structure” that can be analyzed with software written in C++; it is now about 1 Megabyte/event. At this point, the ATLAS reconstruction software (written in C++) runs over these RDOs and produces tracks in the inner detector, electron and muon candidates, jets, etc., and outputs two other structured formats, ESD (Event Summary Data), AOD (Analysis Object Data), which contain different levels of detail; they are approximately 500 Kilobytes and 100 Kilobytes/event. As you can tell by the name, most physicists will run on AOD’s; there are other smaller formats, but I will skip them.

In a “normal” year, we expect to collect about 2 billion events. How do we handle all this data? To do this, physicists and computer scientists have been working on the Grid. It is basically a whole lot of computers spread all over the world that are networked with very fast connections; typical data transfer rates are 1-10 Gigabits/sec (in contrast, broadband connections to your home are about a thousand times slower).

You may ask why we need the Grid. Why can’t all the collaborating institutes just buy computers and send them to CERN, and let them set up a gigantic processing center? That is one approach. However, funding agencies don’t like this mode of operation. They would much rather keep the hardware in their respective countries, and build upon existing infrastructure at universities and laboratories, which includes people, hardware, buildings, etc. Another advantage of the Grid approach is the built-in redundancy; if one site goes down, jobs can be steered to others. Also, if a Grid site is appropriately configured, then if ATLAS is not using the computers, other scientists can use them (in an opportunistic manner); although, we keep the system pretty busy. In the US, each LHC experiment has its own grid sites, whereas in Europe, they tend to share them.

(more…)