–by T. “Isaac” Meyer, Head, Strategic Planning & Communications
Seems like scientists travel a lot, going to and fro, traversing universities, laboratories, and countries around the world. Is this because scientists are wanderlusts and cannot be contained within one set of borders?
No, not really.
For instance, a chief reason scientists travel is to participate in peer-review committees of their fellow laboratories and research institutions. This isn’t the type of peer review that is used to hone and select papers for publication. This is a peer review where scientists and other experts gather to give tough love to a research project and its leadership.
I wrote this from a small hotel off the U.S. Route 1 in New Jersey, near Princeton University. What brought me here was an invitation to join a group of experts to hear about the progress and plans of the nearby U.S. DOE national laboratory and give shrewd advice about where they needed to straighten up and fly right—and where things are really humming. I came as a “subject-matter expert” in communications and strategic planning; others were international experts in plasma and fusion science; still others were seasoned veteran project leaders with decades of project management and budgeting experience. Perhaps most importantly, some of the people in the group hailed from the lab’s traditional “competitors” and its collaborators. In this way, the host lab got the toughest scrutiny.
That’s the amazing thing. As competitive as scientists are with each other, with one another’s collaborations and experiments, we all know that it’s better if it’s a “fair fight” and a level playing field. The only “losses” should be on the scientific playing field, not because of poor management, accidents, or design flaws. So scientists, especially particle physicists, have a great tradition of inviting groups of their expert colleagues over to review their programs and plans with a fine-toothed comb. Some of the committee members can be quite gruff, but it’s like preparing for a test: you want the hardest questions in advance so that you are totally prepared and totally ready for an official government review. Not only does the laboratory get crucial advice about what is working and what isn’t, but the visiting experts get a chance to get updated on the capabilities and plans of the laboratory. Everybody wins, because everybody contributes something and everybody gets contributed to.
There’s another feature of these reviews that is neat. Not all review committees do it this way because of their specific tasks or mandates, but there is one style of the review and the report that is especially common—based on the so-called “Lehman review” format named after Danny Lehman of the U.S. DOE’s Office of Engineering and Construction Management. (See this article in ITER Newsline to learn more about it.) A Lehman review is considered one of the most grueling and investigative reviews in the business. The report of the review committee is designed to be short and sweet; no time to waste on pleasantries. The report is organized into three sections on the major topics: findings, comments, and recommendations. Findings capture the essential facts that the visiting committee learned during the review. Findings are meant to be just observations and “what’s so.” It forms the basis of the next two sections of the report. A “comment” is exactly that; it is a reasoned judgment or assessment of the review committee based on their substantial experience. A comment might state that a certain finding is a good technique for getting the work done or it might state that a certain finding has inherent risks for achieving the overall project goals. Finally, recommendations are the specific pieces of hard-hitting, specific advice. Recommendations are intended to be very clear: who should do what by when? A recommendation that says, “Somebody should think hard about that situation and then maybe do something,” is rejected. A recommendation is meant to a homework assignment that the host laboratory could understand, complete, and then report back on how it went.
In any event, the peer review I participated in for the Princeton laboratory was phenomenal. The laboratory has come along way over the past decade and they are taking a real leadership position for the future of plasma science and technology in the U.S.
Thank you for sharing the lab with us, and we’ll back in 6 months to check on your “homework!”
