Particle physics is hard work. I don’t just mean that the theory is difficult, I mean that the day to day work is difficult even if you know what you’re doing. The only thing more complicated than working with people is working with computers, and at CERN we have to do a lot of both. It’s even more difficult when you don’t know what you’re doing it, and since we work in research there are plenty of things that nobody will know! In that kind of scenario, having more than a passing knowledge of some area of hardware or computing can give you the title “expert”. The past two weeks have been a non-stop barrage of really hard things happening, and having to make tough choices about how to deal with them. The humidity and heat hasn’t helped, and only made it harder for everyone to concentrate! But today, the skies opened up, the rain poured down and the work finally became manageable again!
In the corridors of CERN, no one can hear you scream!
In the past couple of weeks I’ve had an awkward dealing with the IT admin. I have a series of computing tasks which must work each night for the Trigger Rates group, and if they don’t work, we run into problems with the monitoring of the trigger rates. Blips happen and jobs fail from time to time, but when it happens for more than a few days people get unhappy. Unfortunately these jobs required large resources, and as the software migrated from one version to another this lead to huge problems with allocations of resources. The IT people noticed this and contacted me immediately! What could I do? If I continued to run the jobs the problems with resources would continue, and I’d get into some serious trouble. If I stopped the jobs we’d have problems with the rates monitoring system. That’s not a nice position to be in! After speaking the experts it looks like there’s a solution and today I got an E-mail telling me how to get the jobs to work without annoying anyone. That’s one crisis averted!
If things were looking bad for my service work, what about my analysis? That was looking a bit shaky as well. I needed to perform a study on the data to see how often we see QCD events (These are events that produce huge numbers of quarks and gluons and they dominate the data. Our poor simulations just aren’t vast enough to model these backgrounds properly!) The problem involved using two pieces of software which were written by different people. That’s usually okay, I’d just E-mail the software experts for help and arrange an informal meeting. Unfortunately neither of these experts were available for various reasons, and when they became available I was still handling the trigger rates problem. So I was submitting analysis jobs to the GRID (the huge internationally distributed network of computers designed for handling analysis jobs) and hoping they would work. At around 2am last night, after failing to get things to work for the fifth time I gave up and went to sleep. Then I awoke suddenly at 7am with a revelation! All I had to do was a simple search function to find out where the problem came from, change a few lines and it should work fine. I resubmitted the jobs and went back to bed. Then this afternoon, with a few precious hours left before the deadline I had to get the output, rescale it properly and perform a fit to count the QCD events. This part had never worked for me in the past, and I’d already failed to show results of the study twice. With 10 minutes left before the start of the meeting I changed a line of code, pressed enter and it worked perfectly! After more than two weeks of struggling with GRID jobs, ROOT fits and late nights it all fell into place very quickly.
Science should always be this much fun!
Now spending that time on a study of the QCD events meant that I couldn’t spend it on other areas of the analysis. I would have liked to see if I could get a better efficiency with a different choice of trigger (picking out events with b-jets instead of missing energy.) I was speaking to a colleague and he was under the impression that this was an easy change to make and would give me better results for very little effort. In a sense he was right, I’d just change one line of code and get more events. But the problem comes back to working with other people. My group is just one small part of a larger collaboration and most of our time is spent coordinating our efforts. We perform cross check upon cross check to make sure that we agree on various definitions and values. It’s necessarily tedious, but it makes sure that we get it right. So if I was to change the choice of trigger I’d have to then convince the other groups that this is a good idea, perform all kinds of studies and then perform cross checks with other people. That means weeks of work and the deadlines for the European Physical Society (EPS) are rapidly approaching! On the one hand I want to see our work presented at EPS, but on the other hand I want to spend time improving the analysis. There just isn’t enough time to do both, so I had to make the decision to pursue EPS. It’s frustrating all on its own, but when a colleague doesn’t see the dilemma it’s made all the more worse. After all, physics is fun! We all love to play around with different ideas and see how we can best improve our results. But bureaucracy is a necessary part of getting a respectable publication, so we must practice some restraint and spend time getting it right. It’s a bit like going on a road trip, only to get stuck in traffic! We’re moving forward slowly and safely, but I want to speed up and see the exciting parts.
So now that these stumbling blocks have passed can I relax and enjoy the fruits of my work? Well I can for one evening. Tomorrow I need to get started on the next project- a high profile talk to the Trigger experts on another part of my service work. That involves all the things that make this work more complicated. I need to coordinate responses from lots of people, each with their own agenda and niche knowledge, and then running jobs on computers (without getting in trouble with the IT admin!) Don’t get me wrong, I enjoy working with people and I enjoy it when the computers work properly, but it can quickly lead to information overload, and when it’s for a high profile talk it can also be stressful. On the other hand it indicates that this work is coming to an end and I can focus on something else such as control room shifts (and more blogging!)
It’s hard and at times it’s frustrating, but when it works properly physics is amazing. Days like today make the weeks that precede them worth while. I just hope I get more days like today and less stressful ones this summer! With so many interesting results around the corner I want to be able to enjoy them all and have time to make the most of one of the most exciting years in the history of high energy physics.
