Giovanni Schiaparelli (1835 – 1910) is mainly remembered for his discovery of “canali” on Mars. What a fate, to be remembered only for discovering something that does not exist. I suppose it could be worse; he could be remembered only as the uncle of the fashion designer, Elsa Schiaparelli (1890–1973). But he was not alone in seeing canals on Mars. The first recorded instant of the use of the word “canali” was by Angelo Secchi (1818 – 1878 ) in 1858. The canals were also seen by William Pickering (1818 – 1878) and most famously by Percival Lowell (1855 – 1916). That’s right, the very Lowell whom the Lowell observatories on Mars Hill Road in Arizona are named after. Unfortunately, after about 1910, better telescopes failed to find them and they faded away. Either that or Marvin the Martian filled them in while fleeing from Bugs Bunny. However, they did provided part of the backdrop for H.G. Wells’s War of the World. But it is interesting that the canals were observed by more than one person before being shown to be optical illusions.
Another famous illusion (delusion?) was the n-ray discovered by Professor Blondlot (1844 – 1930) in 1903. These remarkable rays, named after his birth place, Nancy, France, could be refracted by aluminum prisms to show spectral lines. One of their more amazing properties was that they were only observed in France, not England or Germany. About 120 scientists in 300 papers claimed to have observed them (note the infallibility of peer review). But then Prof. Robert Wood (1868 – 1955), at the instigation of the magazine Nature, visited the laboratory. By judiciously and surreptitiously removing and reinserting the aluminum prism, he was able to show that the effect was physiological, not physical. And that was the end n-rays and also of poor Prof. Blondlot’s reputation.
Probably the most infamous example of nonsense masquerading as science is, Homo piltdownensis, otherwise known as the Piltdown man. This was the English answer to the Neanderthal man and the Cro-Magnon man discovered on the continent. A sculptured elephant bone, found nearby, was even jokingly referred to as a cricket bat. Seems appropriate. While there was some scepticism of the find, the powers that be declared it to be a breakthrough and it was only forty years later that someone had the brilliant idea that it might be a fake. Once the signs of faking were looked for, they were easily found. What we see here is an unholy combination of fraud, delusion, and people latching onto something that confirmed their preconceived ideas.
These examples are not unique. Most exciting new results are wrong[1]: polywater, the 17 kev neutrino cold fusion, superheavy element 118, pentaquarks, and the science on almost any evening news cast. Cancer has been cured so often it is a surprise that any cancer cells are left. So, why so many exciting wrong results? First, to be exciting means, almost by definition, that the results have a low prior probability of being correct. The discovery of a slug eating my garden plants is not exiting, annoying but not exciting. It is what we expect to happen. But a triceratops in my garden, that would be exciting and almost certainly specious (pink elephants are another matter). It is the unexpected result that is exciting and gets hyped. One can say over hyped. There is pressure to get exciting results out quickly and widely distributed so you get the credit; a pressure to not check as carefully as one should, a pressure to ensure priority by not checking with one’s peers.
Couple the low prior probability and the desire for fame with the ubiquity of error and you have the backdrop to most exciting new results being wrong. Not all exciting new results are wrong, of course. For example, the discovery of high temperature superconductors (high = liquid nitrogen). This had crackpot written all over it. The highest temperature recorded earlier was 30deg Kelvin. But with high temperature superconductors, that jumped to 90 in 1986 and then shortly afterwards to 127deg kelvin. Surely something was wrong, but it wasn’t and a Nobel Prize was awarded in 1987. The speed at which the Nobel Prize was awarded was also the subject of some disbelief. Why was the result accepted so quickly? Reproducibility. The results were made public and quickly and widely reproduced. It was not just the cronies of the discoverers who could reproduce them.
The lesson here is to distrust every exciting new science result: canals on Mars, n-ray, high temperature superconductors, faster than light propagation of neutrinos (which coincidentally just released some new interesting information), the Higgs bosons and so on. Wait until they have been independently confirmed and then wait until they have been independently confirmed again. There is a pattern with these results that turn out to be wrong. In almost every example given above, the first attempts at reproducing the wrong results succeeded. People are in a hurry to get on the band wagon; they want to be first to reproduce the results. But after the initial excitement fades, sober second thought kicks in. People have time to think about how to do the experiment better, time to be more careful. In the end, it is this third generation of experiments that finally tells the tale. Yah, I know I should not have been sucked in by pentaquarks but they agreed with my preconceived ideas and the second generation experiments did see them in almost the right place. Damn. Oh well, I did get a widely cited paper out of it.
Once burnt, twice shy. So scientists become very leery of the next big thing. Here again, science is different than the law. In the law, there is a presumption of innocence until proven guilty. In other words, the persecution must prove guilt; the suspect does not need to prove innocence. In science, the burden of proof is the other way around. The suspect—in this case, the exciting new result—must prove innocence. It is the duty of the proponents of the exciting new result to demonstrate the validity or usefulness of the new result. It is the duty of his peers to look for holes, because as the above examples indicate, they are frequently there.
Additional posts in this series will appear most Friday afternoons at 3:30 pm Vancouver time. To receive a reminder follow me on Twitter: @musquod.
[1] For more information on the examples, Google is your friend.
