Quantum Diaries

It’s a busy time. First, the LHC was closed up today for the first time this year, allowing the start of machine checkout and then, eventually, circulating beams. The beginning of the 2011 run is in sight, although we won’t have collisions for physics for a while yet. Also, we’re getting close to winter conference season. The Recontres de Moriond meetings are traditionally a venue for the presentation of new experimental physics results, and you can be sure that all of the LHC experiments are readying some interesting stuff for that. I have previously discussed the internal review processes of experiments, which can take a while, so even though the conferences are a few weeks away, a lot of analyses are becoming finalized and starting to be reviewed right now. Whether you are a reviewer or reviewee (or both), it can take a lot of time. (Oh, and then there is the recent discussion of federal budget cuts in Washington, which has us all reading the newspapers pretty closely.) So we don’t lack for things to do.

But, meanwhile, here is something to consider. The ATLAS and CMS experiments are ultimately very similar — they both have similar goals (which are different than for ALICE and LHCb, hence their absence from this discussion), and similar enough capabilities (although differing strengths and weaknesses), and they both record pretty much the same amount of data. So why don’t they publish the same measurements at the same time? Just as an example, the two experiments submitted publications on measurements of rates of W and Z bosons three months apart, with the later one analyzing ten times as much data (and having much more precise results) than the first one. Please note, in an attempt to be neutral, I am not naming names here!! Let’s instead take this as an introduction to a broader question — given that the LHC will continue to pile up data over time, when do you stop and say, “OK, let’s publish with what we’ve got?” How much data is enough?

I’m not going to claim to have all the answers to this question, and for any given measurement there will be a unique set of circumstances. But here are a few possible considerations:

• Is there a break in the action at the LHC? This is a totally pedestrian consideration, but if the LHC isn’t going to run for, say, three months, as is happening right now, for many measurements it might not be worth the wait for more data, so you should just publish with what you’ve got. There are going to be a lot of publications based on the data recorded in 2010. It’s true that in 2011, if the collision rates are as expected, the 2010 data will quickly be superseded, but why wait those few months, especially if you are doing a measurement in which additional statistics might not make a meaningful difference?
• When can I make a scientific statement that has sufficient impact on the world? If you only have enough data to make a measurement that’s, for instance, ten times less accurate than the most accurate measurement of the same quantity that’s currently available, there’s no point in publishing. But if you are at least in the range of comparable to the best measurement (even if not yet the best), it might make sense to publish, because it’s accurate enough to make a difference in the world’s understanding. If you average two measurements of equal precision, then the average will be a factor of 1/sqrt(2) = 1.4 more accurate than either individual measurement. Seems worth it, right?
• Am I worried that someone else is going to beat me to something? Let’s face it, there is some glory to being first, especially if there is something new to report. If you are worried that competitors might get to it first, perhaps you will decide that you have to release your result, even if you know you might do a better job yet, either by recording more data or just having more time to work on it.
• Then again, it’s better to be second than to be wrong. A wrong result would be embarrassing, for sure, so it’s better to do the work necessary to have greater confidence in the result.
• If you really can do a much better job with not much more time or effort, why not just do that? If you do, then your measurement is going to be the one in the history books, even if you weren’t first.
• Do I finally have enough data to report a statistically significant result? Well, this is what we’re all waiting for — at some point some new phenomenon is going to emerge from the data. At first, the statistical strength will be marginal, but as more data are analyzed, the signal will stand out more strongly. You can be sure that once any anomaly is observed, even at a low level, it will be tracked very carefully as additional data are recorded, and as soon as an effect reaches some level of statistical significance, it’s going to be published just as quickly as possible, without delay.

These are just a few of my own musings, dashed off quickly — I invite our readers to offer ideas of their own. (OK, and now I click on the “Publish” button on the right….)