What is like to be on shift in the ATLAS Control Room? It’s my second night on shift and we’re already off to an interesting start. As I walked through the door I saw that the LHC was ramping and my excitement grew. Soon we would have beams and data! But before I even sat down the beams got dumped, so the LHC had to try again. Since I don’t really have much to do this would be an excellent time to describe what I do on shift and why I don’t really have much to do right now.
Some terminology
There are a few words you need to know to understand what we do here.
Someone who is in the ATLAS Control Room and performing a routine task on shift is known as a Shifter. (There are often people there performing non-routine tasks who are not Shifters. They work with the Shifters to accomplish their tasks.)
Shifters work at a designated Desk. Each Desk has a label on it telling the other Shifters who works there, as well as its own computer system and telephone. There are 16 Desks in total, some which manage part of the detector, and some which manage different kinds of tasks that involve the whole detector.
The LHC provides beams of protons which can be in one of several states. Stable beams means that the beams parameters are very precisely tuned and we can use them to record data. Injection is when the LHC is preparing the beams to become stable. When the beams become unstable they get dumped (time for a toilet flush sound!) and the LHC also has periods of Ramp Up and Ramp Down before and after stable beams.
Data taking is split up into long periods of stable beams called a Run. Each run is split up into short time intervals of about 1 minute called Lumiblocks. A lumiblock is a small period of time with roughly the same kinds of conditions.
The desk
I’m sitting at the Trigger Desk and that means that I’m responsible for making sure the trigger is behaving well. Here is a picture of my desk:
My Trigger Desk
There’s a lot of stuff there! Going from left to right we have the Red Folder Of Answers (RFOA? That would be a difficult acronym!) This holds a copy of nearly all the training material we need to be familiar with, and also some simple notes about how to handle a few common situations. Every desk should have one of these. I’m not just talking about in the ATLAS Control Room, I mean every desk in the world.
Next we have the Ominous Telephone Of Panic. Well, not really. Most of the time this is just used to exchange simple information. But there’s always the possibility that it will ring and there will be an expert on the other end of the line who has some difficult questions. If there’s one thing we don’t like on shift it’s surprises.
Then my laptop. I’d be lost without this. On here I have all the most useful websites and talks I could find that relate to the trigger. It’s quicker and easier to look up information on here, I can’t break anything by accidentally clicking the wrong button, and it’s shinier than the computers that comes with the desk. And of course, this is where I write my blog posts! And next to that is my ID. I need that to get in and out.
Towards the back of the desk we have the monitors that give me all the information I need to be able to be a good Shifter. Each monitor has about four applications open, and each one of those has about 20 tabs. There are literally hundreds of plots and tables that are available, and I can’t look at them all for all of the time! That is what makes this role difficult. To help things, I reserve the left most monitor for the most important plots (how quickly we are recording data) and the right most monitor for essential reference material (the plan for the day, the normal instructions and the instructions for special tasks.) The other two monitors are where most of the work takes place. Looking at plots, reading error messages and “spying” on the other Shifters.
Then we have the two pads of paper. The big one is mine and that’s for notes, doodles and anything else that comes to mind. (I think I have some cocktail recipes in there somewhere!) The smaller pad is for sending short messages to other Shifters in the room. For each set of “prescale keys” I have a different sheet of paper. It’s just a matter of picking the right one.
Then there’s my food. Since we’re not allowed to leave the Control Room for very long we need to bring all our food with us for the next 8 hours. I’ve got some bread and cheese, some fruit, and some yoghurt. Very healthy! Unfortunately I’ve also got a can of coke and some chocolate. Oops. Also note the coffee cup. This is the night shift, after all.
In the background we see the projectors. These are one of the ways the Shifters can share information between each other and see what is happening. The projector immediately in front of me shows the data as they flow through the system (from left to right.) If there’s a problem, those boxes turn red and it’s my job to work out what is happening. Usually when that happens it sorts itself out after about a minute. The other projectors show the trigger rates, the state of the various parts of the machine, the Daily Run Plan, the status of the LHC and then some event displays.
Although you can’t see it, there’s also a monitor to my left that shows the LHC Operation Page and the LHC Page 1. This is essential to the Trigger Shifter as we need to know the state of the LHC for our job.
(And for those who are observant, yes I do have a Higgs boson with me! Who knew it would be so easy to find? And in ATLAS Control Room of all places.)
We can get bored easily on shift...
The role of Trigger Shifter
So what do I actually do? In principle it’s actually quite straightforward. When we are taking data I look through the plots to make sure that they look okay. This is not a particularly well defined task though, since we don’t know exactly what the plots will look like for a given run (if we did then we wouldn’t need a human to compare them.) The advice we get about the plots is usually very good, so most of the time we can be confident that the plots look okay. If they look a little odd and it’s a Day Shift we usually phone the expert on call, whereas if it’s a night shift it’s usually good to get a second opinion before waking up a poor sleepy civilian. If the plot is obviously wrong, then a call is made pretty much immediately (with a few seconds spent getting some vital information about what happened immediately before the problem.) Once I’ve looked at all the plots I take a break and come back to them again a little while later.
The efficiency of the trigger needs to be kept high (we are competing with CMS, after all) so we keep adjusting it. This is done by changing the “Trigger Menu”, which is a list of different kinds of triggers and how often we use them to record data. The different trigger menus are specified using prescale keys, so when the conditions change enough to justify changing the menu, I get a new set of keys (which is just a fancy word for “four digit number”.) I write these down on my little pad and pass them to the Run Control Shifter whose job it is to update the menu. Why is that not the job of the Trigger Shifter? I’m not really sure. I suppose two heads are better than one.
Hurrah, data!
When we have no beams the Trigger Shifter usually has nothing useful to do. I can prepare the prescale keys for the next run, but otherwise I can’t really do much until the next time the LHC ramps up. As it ramps up I need to watch the Bunch Groups (this tells ATLAS how many bunches of protons are moving through the LHC.) If they change, I create new bunch group information. In fact, that’s the most useful part of the Red Folder Of Answers. It’s something I will have to later today for the first time!
From time to time we have special running conditions, and we have special prescale keys for that. Right now I have to keep track of seven different sets of prescale keys (for normal running, for when the machine is in Standby mode, for when we take calibrations and for different special tasks, such as when the LHC tries a new configuration.) Each one is written on its own tiny sheet of paper!
And the other stuff
Apart from the tasks listed above, I have be able to answer general questions about the trigger, such as making comments about its performance, finding out what a specific part of the trigger is doing and working out problems. I also need to be in constant communication with the other Shifters, usually the Shift Leader (the boss!) and the Run Control Shifter who needs to know what changes to make to the trigger menu. My Desk allows me to “spy” on the Run Control Shifter to make sure the trigger settings are correct there. Right now the prescale keys are exactly what we expect. Unless there is a serious breakdown of communication, they should always match. From time to time the Run Control Shifter gets a message popping up on their screen about background events, and then ask the Trigger Shifter to make a decision. The answer is usually “No”.
Right now…
At the moment the LHC is injecting beams. That means that I can’t make any changes. The prescale keys are ready for the next run. I’ve updated myself with all the information for the day. I’ve even written up a summary for the Shift Leader and the next Trigger Shifter about the day’s plans. There are currently some plots flashing on the monitor before me. When the LHC starts to ramp up I need to watch these plots carefully. If they change even by a little bit I’ll need to update the bunch group information. If not, then I can just sit back until we start recording data again.
It’s quite an odd job when you think about it.
