Thanks to Steve for the inspiration and Rozmin for her help with editing! And, of course, to the LHC team.
Posts Tagged ‘heavy ion physics’
Pioneering Supercomputer QCDOC Retires, Regenerates in ‘Next-Generation’ QCDCQ
Monday, October 31st, 2011
On May 26, 2005, a new supercomputer, a pioneering giant of its time, was unveiled at Brookhaven National Laboratory at a dedication ceremony attended by physicists from around the world. That supercomputer was called QCDOC, for quantum chromodynamics (QCD) on a chip, capable of handling the complex calculations of QCD, the theory that describes the nature and interactions of the basic building blocks of the universe. Now, after a career of state-of-the-art physics calculations, QCDOC has been retired — and will soon be replaced by a new “next generation” machine.
In front of the disassembled parts of the now retired giant supercomputer QCDOC are: (front, from left, holding the motherboard) Norman Christ, Columbia University (CU); Robert Mawhinney, CU; and Taku Izubuchi, RIKEN BNL Research Center (RBRC) and BNL Physics Department High Energy Theory group (HET); with (from left) Heng-Tong Ding, BNL Physics Lattice Gauge Theory group (LGT); Hiroshi Ohno, LGT; Frithjof Karsch, LGT; Taichi Kawanai, RBRC and HET; Nicholas Samios, RBRC; Masakiyo Kitazawa, Osaka University; Michael Creutz, HET/LGT; Swagato Mukherjee, LGT; Christoph Lehner, RBRC; Amarjit Soni, HET; Chulwoo Jung; HET; Tomomi Ishikawa, RHIC and RBRC; Qi Liu, CU; Eigo Shintani, RBRC.
Great Start
QCDOC was a “special purpose” machine dedicated to fundamental physics. Among its prime users were the scientists of the RIKEN BNL Research Center (RBRC) at BNL, for their studies of the physics taking place at Brookhaven’s Relativistic Heavy Ion Collider (RHIC) and other forefront physics. Funded by RIKEN, the Institute of Physical & Chemical Research in Japan, with infrastructure support from DOE’s Office of Science, QCDOC was built by BNL, Columbia University, IBM, the RBRC, and the University of Edinburgh.
Most specifically, QCDOC was designed to use lattice gauge theory — a method of approximating space-time in a lattice and then computing the dynamics of quarks and gluons, the fundamental constituents of matter, by their theoretical positions on the lattice. One of the founders of this approach is BNL’s Michael Creutz.
The $5-million QCDOC, which took three years to design and build, had 10 teraflops (Tflops) of peak computing power and could perform 10 trillion arithmetic calculations per second. When installed, it took pride of place over BNL’s 1998 supercomputer called QCDSP — for quantum chromodynamics on digital signal processors — which had 0.6 Tflops at peak speed. Together, these two supercomputers provided comparable or greater computing resources for lattice QCD than were available anywhere else in the world.
Surpassing Expectations
“We expected great things from QCDOC, and it surpassed all expectations,” said Nicholas Samios, RBRC director. “It has been the workhorse of three groups at BNL — the High Energy Physics Theory Group, the Lattice Gauge Theory Group, and the RBRC Computing Group that together have established BNL as a world leader in the QCD field. With QCDOC installed, we produced a series of ‘firsts’ in calculations that shed light on much of the breakthrough physics at RHIC as it explores the fundamental properties of matter and conditions of the early universe.”
Taku Izubuchi, Physics Department, a leader of the RBRC Computing Group using QCDOC, explained that until 2008 the entire lattice physics program of RBRC was carried out on the QCDOC and QCDSP supercomputers.
“Through the capabilities of these machines, we developed successful techniques to study kaon physics, nucleon physics, hadron electromagnetic effects, and other important problems in low energy QCD,” he said. “Scientists from Japan, the UK, and many U.S. institutions — we have all benefited from the unique combination of QCDOC and QCDSP.
“We also benefited greatly from the excellence of the Brookhaven support staff from the machine’s installation through technical upkeep, as well as in collaborations with the Lab’s Computational Science Center and New York Center for Computational Sciences,” Izubuchi added.
Among the many highlights of the QCDOC computations were the first lattice QCD computation with chiral quarks (quarks treated with a proper separation of mirror-image spin states, important for the light quarks in nature) and the first and only lattice QCD computation including both dynamical chiral quark effects and electromagnetism. Another was the first and only direct computation for a long-standing problem in certain decays of kaon particles that were key to understanding important aspects of the Standard Model of particle physics. For more, see Physical Review D78: 034503 (2009), arXiv:1106.2714.
Building on Past Success: Blue Gene, QCDCQ
The success of these supercomputers at Brookhaven preceded the much larger commercial machines that exploited the same technology but had greater economy of scale. One example is the IBM Blue Gene/P established in 2008 at Argonne National Laboratory. The U.S. QCD group, of which RBRC is a part, used nearly one-quarter of this huge supercomputer for its work.
Now, a new project is under way: RBRC, Columbia University, and BNL, together with the University of Edinburgh, a collaborator in the United Kingdom KQCD group, are developing an even more powerful supercomputer, QCDCQ (QCD with Chiral Quarks). This machine will have 75 to 150 Tflops of sustained speed — key to RBRC’s continuing international leadership in lattice QCD. The researchers will also work with colleagues at the T.J. Watson Laboratory of IBM — the same colleagues who transformed the technology of QCDSP and QCDOC into IBM’s Blue Gene product. So QCDCQ will benefit from advantages of IBM’s next-generation supercomputer, which will be available in prototype form at RBRC and Edinburgh between one and two years before larger installation of commercial versions are available.
When installed, these QCDCQ machines will provide the RBRC and the larger RBRC/UKQCD collaboration with spectacular research opportunities for understanding more of what RHIC shows us about the early universe and exploring new physics at present not accessible to lattice QCD, Izubuchi said.
And Beyond
“There’s another great value of QCDCQ,” said Izubuchi. “Just as QCDOC provided the basis of innovations for QCDCQ, the advances we’ll make using QCDCQ will inspire the next generation of supercomputers even further into the future.”
– Liz Seubert, BNL Media & Communications Office
Right now the LHC is about to start a short run with proton-proton collisions at a center of mass energy 2.76 TeV. This is lower than what we ran last year and is a special request from the heavy ion physicists. So you’ve heard a lot about why the particle physicists want to go to higher energy. But why do we heavy ion physicists want to go to lower energy?
We want a reference for our lead-lead collisions. If nucleus-nucleus collisions were nothing but a bunch of proton-proton collisions, what we measure in lead-lead collisions should be just some constant times what we measure in proton-proton collisions. This is a bit simplistic, but it’s a pretty good start. A lot of our measurements use proton-proton collisions as a reference and look for differences between proton-proton collisions and lead-lead collisions. For instance, in the paper I discussed here we looked at the distribution of particles as a function of their momenta in lead-lead collisions and compared that to what we observed in proton-proton collisions. For this paper we used the data from proton-proton collisions at 900 GeV and at 7 TeV to extrapolate to what we’d expect at 2.76 TeV, the same energy per nucleon as our lead-lead collisions. As discussed here our models for proton-proton collisions are pretty good but they get some of the details wrong – and miss some features like this. Since we depend on models to extrapolate to 2.76 TeV, we have greater uncertainty in our measurements than we would have if we had data at 2.76 TeV. The LHC can go down to 2.76 TeV and what we need 2.76 TeV proton-proton data doesn’t require as many statistics (as many total proton-proton collisions) as what the particle physicists need to look for things like the Higgs. So we’re having a short run with proton-proton collisions at a lower energy because it will significantly help the heavy ion physics program. (We’ll also get some core physics measurements out of the 2.76 TeV proton-proton data, but like the paper I discussed here, these will refine our understanding but not dramatically change our understanding of proton-proton collisions.) I hope you’re as excited as I am!
I am overdue for a blog post because I have been way too busy lately. I got an email from an elementary schooler, Jacob, asking about the QGP so I thought instead of replying privately I’d reply here since it may be of general interest. The questions are from Jacob.
What is QGP going to be used for in the future when it is better controlled?
Right now we don’t think the QGP has any practical applications. We’re studying it because we want to understand the universe in general and nuclear matter in particular. Shortly after the Big Bang, we think that the universe went through a Quark Gluon Plasma phase. By understanding the QGP better, we may understand how the universe expanded better. When we do basic research, we don’t usually know what impact it will have. What we know by looking at history is that basic research eventually leads to benefits to humanity – but we’re very bad at predicting what those benefits will be. When Mendel studied genetics of plants, he never imagined that genetic studies would lead to all of the improvements in medical care we have now. Einstein developed his theory of gravity not so that we could send satellites into space or so that we could all have GPS in our cars or get better TV reception – he was motivated by simple curiosity and a desire to understand our universe better. We are still reaping new benefits from quantum mechanics, developed in the early 20th century – we now have light emitting diodes (LEDs) in traffic lights and flashlights and while LEDs existed when I was your age, they weren’t nearly as widespread, as cheap, or available in so many colors. So it takes a long time to see the benefits of basic research.
So we don’t know what applications this research will have in the future. That said, there are a lot of spin off benefits to this research. In high energy physics, we are always building the fastest and most precise detectors possible. To do this we often have to develop and test new detector technologies. Once we’ve developed the technology, these detectors can be used elsewhere too. Particle detectors are used in hospitals in x-ray and MRI machines. They are used in chemical and biomedical research to study the images of proteins and the structures of solids. They are used in national security for detecting radioactive materials.
Basic research moves the boundary of what is possible. Once we have done that, there are a lot of benefits. But since we’re working on doing things that have never been done and studying things never studied before, we can’t predict exactly how it will be useful. Put another way, if we knew what would happen, we wouldn’t call it an experiment.
What attributes does it have that other matter does not have?
This is a difficult question to answer as worded – it depends on what you mean by “attributes”. When I think of the properties of a particular form of matter, I think about its density, its opacity to different probes (like if you shine light through it does the light come out the other side?)… All forms of matter have a density. So I’m going to answer a slightly different question – what makes a QGP unique? What makes the QGP unique (among the forms of matter we’ve studied in the laboratory) is that the quarks and gluons interact through the strong force. There are four fundamental forces in nature
1. Gravitation
2. Electromagnetism
3. Weak interaction
4. Strong interaction
The first two are the most familiar. Gravity is the reason why you stay on the ground instead of floating through the air. It’s also the reason the Earth orbits the Sun. The electromagnetic force is ultimately responsible for basically every other force you feel or see. When you sit in a chair, the reason you don’t fall through the chair is ultimately due to interactions between your atoms and the atoms of the chair. It’s also behind light and electricity. It’s how your microwave and your TV work. The most familiar thing we can attribute to the electroweak decay is beta decay – a particular kind of decay of a nucleus. The strong force is what holds nuclei together. If we only had the electromagnetic force, the protons in the nucleus would not be bound.
So a QGP is a liquid of quarks and gluons bound together by the strong force. Water molecules, for instance, primarily interact through the electromagnetic force. The properties of water are determined by the way water molecules interact through the electromagnetic force. To understand the QGP, we have to understand how quarks and gluons interact through the strong force. This turns out to be a very difficult computational problem. But by studying the QGP, we can try to calculate what we would expect and then compare what we expect from our theories to what we see in the laboratory.
In addition to that, it is the hottest, densest form of matter ever created in the laboratory. And it appears to have the lowest viscosity of any form of matter ever created in the laboratory. Viscosity is a way of measuring how much a fluid resists flowing. Honey, for instance, is much more viscous than water.
How will QGP affect modern or future physics?
I don’t know exactly. It depends on what we learn. Already we’ve learned a lot about relativistic fluids – where the individual particles in the fluid are traveling close to the speed of light. As I said in the first answer, we don’t know exactly what we’ll learn – because if we did, we wouldn’t call it an experiment. One thing I hope – and maybe you can help me out here – is that we’ll inspire the next generation to go into science, math and engineering.
Also, what state of matter is it? I know that it is called plasma but I’ve also read that it is very similar to both liquid and gas.
A QGP is a new state of matter. We believe it is a liquid – indeed, a liquid that probably has the lowest viscosity of anything we’ve ever measured. We thought it’d be a gas, but it turned out to be a liquid. Here I have a post describing what we know about the QGP and its phase diagram.
I also could not verify what temperature it occurs at because there is so much different information on the internet.
The reason what you find on the internet is somewhat unclear is that the answer is somewhat unclear. First, it doesn’t exist at just one temperature. Think about water. Water can be cold, warm, hot, etc. It depends. There’s a temperature where ice melts and becomes water and below that you can’t have water. That temperature is called the melting point. But then once you have water, you can heat it up and you have to heat it up a lot before it boils and becomes a gas. That also occurs at a special temperature – the boiling point. The problem is, these temperatures depend on pressure and volume. Water boils at a lower temperature at high altitude. Analogously, we have a melting point and a boiling point for the QGP. We think the melting point at the baryochemical potential at RHIC is about 170 MeV – but there’s a fairly large uncertainty in that number. We think we’re well above that at RHIC and we’ll be even further above it at the LHC (but we haven’t yet had enough time to analyze the data at the LHC to say how hot it is). This gets to a crucial issue – we don’t have a thermometer to measure a QGP. If you put a thermometer like the one you have in your house into a vat of QGP (if we could ever create that much of it) it’d melt. So we have to come up with other ways of measuring the temperature. We can look at the energies of particles created in the collision, for instance. But it takes more work than just using a thermometer.
Many thanks to Jacob for the great questions!
The electromagnetic calorimeter is now fully installed but there’s still work to do before we start running. We now have to make sure we’re able to read all of the data. I’ve spent most of the last week in, on, and next to ALICE troubleshooting (along with several of my colleagues working on the calorimeter.) Here I am sitting inside the magnet on top of the support structure next to the front end electronics (the boards that read out the data) for the calorimeter. I’m on the phone with someone upstairs who’s trying to take a pedestal run to see if we’ve fixed a problem reading out data from one of the new supermodules. (A pedestal run is a run you take without proton-proton or lead-lead collisions to see what the background in your detector is. It’s useful for troubleshooting because the detector has to send data.)
Now that we’re getting close to the start of the run, they’re putting the concrete shielding in. In total 30 or 40 tons of concrete blocks sit above ALICE. Here you can see one of the last blocks going in:
And just to go along with the preposition theme, here’s a picture under ALICE (in the magnet but under the TPC, TRD, and TOF):
I introduced you to the ALICE electromagnetic calorimeter (EMCal) a while ago, and told you about some additional training I had so that I could work on the detector after the EMCal is physically installed. Over the winter shut down – right now – the EMCal is being installed inside ALICE.
There are several steps in this process. First the EMCal was assembled, partially calibrated, and tested – this was done in November. There were several stages of testing. We tested that each individual cell works. We tested that each individual electronic card for reading out the data works. We assembled everything exactly how it would be installed inside ALICE and tested it again, making sure that all of the parts (including the wires) worked together. We partially calibrated the detector by taking data on cosmic rays. We’ve had all of the six supermodules we’re adding waiting at CERN until we could get access to ALICE to install them.
Now they’re physically installing the supermodules and our amateur EMCal documentarian, Federico, has taken some videos of the process. (It might help to go back to this post, where I introduce each of the detectors and explain what they do, and this post, where I show you some pictures of each of the detectors. Then maybe you can identify the different parts of ALICE in the video.) Note the action in the videos is very slow because it’s very important not to damage anything while installing the detector.
The first step is to put the supermodule in the EMCal insertion tool. This is a specialized device for installing EMCal supermodules. Here you can see a video of that step:
And then once the supermodule is in the insertion tool, it gets installed in ALICE:
[youtube 0es9Qcdj-H8]
And now some gratuitous cool pictures of the process:
ALICE ready for the installation of EMCal supermodules
Looking up from the cavern. We sit at the top when we take data and the detector is far below us. The EMCal supermodules have to be lowered down.
Checking everything twice to make sure there are no mistakes.
Getting the EMCal insertion tool ready for a supermodule
Getting ready to strap the support onto the supermodule
Sliding the support onto the supermodule
Now it’s in and they’re strapping it onto the crane
Up goes the supermodule…
…and into the insertion tool.
And it’s rotated to the correct angle…
Waiting on the support structure in ALICE to make sure it goes in properly…
Get it in the right position…
Now loosen it from the EMCal tool and in it goes!
And now we do the next one.
Many thanks to Federico for the great pics!
There have been a lot of exciting results lately and I haven’t gotten a chance to write about them because I’ve been too busy. Today I’ll tackle jet quenching, which Seth touched on in one of his posts.
You may have done absorption spectroscopy in a chemistry lab. In absorption spectroscopy, light from a calibrated source passes through a sample and changes in the light after passing through the sample are used to determine the properties of the sample. For example, you may have a liquid that absorbs blue light but lets orange light through. This tells you something about the properties of the liquid. We want something like that for studying the Quark Gluon Plasma (QGP). Perhaps we could try shining light on the QGP to see what it does to the light, how much is absorbed? The problem with that is that the QGP formed in a nucleus-nucleus collision doesn’t live very long – about 10-24 seconds. Trying to aim light at the QGP would be like trying to hit a fighter plane at top speed with a Nerf gun – by the time you aimed, the plane would be long gone.
Fortunately, photons (light) are created in the lead-lead collisions. Since they are produced in the collision, we know they went through the QGP so we can use them and study how they’re affected by the QGP to determine its properties. This is analogous to determining what a store sells by looking at what people have in their shopping bags when they leave the store rather than by going in the store yourself. This is one of the measurements we’ll see at some point. But photons only interact through the electromagnetic force and many of the features of the QGP we’re trying to study come from the interaction of quarks and gluons through the strong force. To study these properties, we need something like a photon, but that interacts through the strong force. We can use quarks and gluons.
There are quarks and gluons in the incoming lead nuclei, and a quark or gluon in one nucleus can scatter off of a quark or gluon in the other nucleus. We’re particularly interested in hard scatterings, where they hit each other and bounce off like billiard balls. This process happens early in the collision, and then the partons travel through the medium, as shown below:
But there’s a complication. We can’t see individual quarks and gluons – they’re always bound in hadrons, states made of two quarks (mesons) or three quarks (baryons), a property called confinement. After the parton gets knocked out of the nucleon, it hadronizes – it breaks up into several mesons and baryons. These are actually what we observe in our detector. For each parton, we have a cone of hadrons called a jet. This is an event display from the STAR experiment showing two jets in a proton-proton collision:
In a proton-proton collision, it’s easy to see jets, but in a heavy ion collision they’re in events like these:
So it’s not as easy to find jets in heavy ion collisions. One thing we can do is look at very fast moving hadrons. These are more likely to have come from jets. This is the subject of the most recent ALICE paper. This is the main result from that figure:
The x-axis is the momentum of the hadron perpendicular to the beam, called the transverse momentum. The y-axis is something called RAA, which is the ratio of the number of hadrons we measure in lead-lead collisions to the number we would expect if a lead-lead collision were just a bunch of nucleon-nucleon collisions. We take what we measure in proton-proton collisions and scale it by the number of proton-proton, proton-neutron, and neutron-neutron collisions we would expect. (Yes, I’m skipping lots of technical details about how that scaling is done.) Another way of putting it is that it’s what we get divided by what we expect. If RAA were exactly 1.0, it’d mean there’s no physics in lead-lead collisions that isn’t in proton-proton collisions. An RAA less than one means we see way fewer particles than we expect. In the figure, the open points are what we measure for peripheral collisions, where the nuclei just barely graze each other. The solid points show what we measure for central – head-on – collisions. The big, obvious feature is the bump which peaks for particles with a transverse momentum of about 2 GeV/c. There’s a lot of physics in there and it’s really interesting but it’s not what I’m talking about today. Look at what it does at higher momenta – above about 5 GeV/c. This is where we trust our theoretical calculations the most. (At lower momenta, there’s much more theoretical uncertainty in what to expect.) We see only about 15% of the number of particles we expect to see. This was already observed at the Relativistic Heavy Ion Collider, but the effect is larger at the LHC.
This happens because the QGP is really, really dense. It’s harder for a parton to go through the QGP than it’ll be to walk through a Target store on the day after Christmas. The parton loses its energy in the QGP. Imagine shooting a bullet into a block of lead – it’d just get stuck.
ATLAS’s recent paper exhibits this more directly. Here’s a lead-lead event where the lead nuclei barely hit each other. Here you can see two jets, like what you’d expect if neither parton got stuck in the QGP:
The φ axis is the angle around the beam pipe in radians, the η axis is a measure the angle between the particle and the beam pipe, and the z axis is the amount of energy observed in the calorimeter. Imagine rolling this plot up into a tube, connecting φ=π to φ=-π and that would show you roughly where the energy is deposited. The peaks are from jets, like in the event display from STAR above. The amount of energy in each peak is about the same – if you added up each block in the peak for both peaks, they’d be about equal. And here’s a lead-lead event where one of the partons got stuck in the medium:
In this plot one of the peaks is missing. One of the jets is quenched – it got absorbed by the QGP. This is the first direct observation of jet quenching in a single event. It’s causing quite a buzz in the field.
ALICE’s first two papers on lead-lead collisions were submitted yesterday, about a week and a half after the first lead-lead collisions.
One paper is a measurement of the charged particle multiplicity. This analogous to the multiplicity measurements in proton-proton collisions, except a more particles are produced in a lead-lead collision. In p+p the models were off by around 10-15%. This is a plot from the lead-lead paper:
The red point shows the ALICE measurement. The x-axis is a measure of the number of particles produced in the collision. (Specifically it is the number of charged particles produced in the collision per unit pseudorapidity for pseudorapidities from -0.5 to 0.5.) The black points are different predictions. Notice is that the predictions vary from 1000-2000 particles. This is a rather large theoretical uncertainty, especially compared to proton-proton collisions. So less than two weeks after our first collisions, we have already gained a much deeper understanding of lead-lead collisions.
The other new paper is a bit more abstract than the number of particles created in the collision. Think of a heavy ion collision as like slamming two ice cubes at each other. If you slammed two ice cubes together fast enough, they’d melt when they hit each other. If you did this in space – where it’s about 3K (about -270 Celcius and -454 Fahrenheit) – the water would immediately freeze again. This is roughly what happens in a lead-lead collision. Nuclei are basically frozen quarks and gluons, and when we collide them fast enough, they melt. But by the time we see the remnants of the collision in our detector, the quarks and gluons have frozen again. However, they went through a phase where they were a liquid. A liquid can flow. We can see evidence that the liquid of quarks and gluons was flowing because we can see in our detector that the particles are all moving in a preferred direction.
This plot compares ALICE’s measurement to earlier measurements:
The x-axis is the collision energy per nucleon (proton or neutron) in the center of mass. The y-axis is a measure of how much the liquid is flowing. [Technical audience: The y-axis is the coefficient of the second term of the Fourier decomposition of the distribution of particles in azimuth with respect to the reaction plane.] Measuring how much the Quark Gluon Plasma is flowing gives us some insight into its viscosity. There are lots of technical details and subtleties in interpreting these data that I’m skipping over. But already – less than two weeks after the first lead-lead collisions – we have two measurements that give us deep insight into the properties of the Quark Gluon Plasma.
You can see live event displays from ALICE here. Perhaps you can identify the different detectors in the event display from the description of the ALICE detector. And here’s an event display with our Time Projection Chamber in:
What you’re looking at is the charged particles we’ve seen in ALICE. I’d tell you roughly how many we see except that I’m afraid that some theorists in our field would mistake that for a measurement. (You know who you are!)
There’s more information about ALICE and more event displays here.
Very early this morning we got the first lead-lead collisions at the LHC! I am all a twitter. This is a very exciting time. I just arrived at CERN today and I am very, very jet-lagged, so I’ll keep this short.
Pictures. What you all want to see is pictures.
Here are some event displays with the first Pb+Pb collisions seen by ALICE. This is an example:
These event displays only show information with the Inner Tracking System (ITS). Our main tracking detector, the Time Projection Chamber (TPC), was off for these collisions. The reason is that the beams were not perfectly stable for the first collisions and we did not want to damage our TPC.
And check out this video of an event display (the original video is here):
[video http://aliceinfo.cern.ch/static/Pictures/pictures_High_Resolution/wwwFirstPbPb/animation9968.avi]
And now that we have lead-lead data, we have a lot of work to do. Expect the first lead-lead paper soon. It will be a multiplicity paper like ALICE’s first few proton-proton papers. We will just measure the number of charged particles in an event. This information alone will tell us a lot about heavy ion collisions – the first estimates for how many particles we should see in an event varied by a factor of 4, from 2000-8000 tracks.
