In these essays, I have discussed various aspects of the scientific method based on model building and testing against experiment. One aspect, I have avoided up until now is how the models are constructed. This is a logically distinct process from how models are tested. We have seen that models are tested by requiring them to make predictions that can be tested against observation. We can then rank models on parsimony and their ability to make successful predictions. But how are models constructed in the first place? Francis Bacon (1561 – 1626) and Isaac Newton (1642 – 1727) would have told you that models were deduced from observation by a method called induction. In this approach, the construction and testing of models is seen as the same process, not two distinct processes. But induction is not valid and the making of models is logically independent of their testing. This is a key point Karl Popper (1902 – 1994) made when he introduced the idea of falsification.
Let us turn to a master, Albert Einstein (1879 – 1955), and see what he says:
The supreme task of the physicist is . . . the search for those most general, elementary laws from which the world picture is to be obtained through pure deduction. No logical path leads to these elementary laws; it is instead just the intuition that rests on an empathic understanding of experience. In this state of methodological uncertainty one can think that arbitrarily many, in themselves equally justified systems of theoretical principles were possible; and this opinion is, in principle, certainly correct. But the development of physics has shown that of all the conceivable theoretical constructions a single one has, at any given time, proved itself unconditionally superior to all others. No one who has really gone deeply into the subject will deny that, in practice, the world of perceptions determines the theoretical system unambiguously, even though no logical path leads from the perceptions to the basic principles of the theory.
Very curious: no logical path leads to these elementary laws. Paul Feyerabend (1924 – 1994) would have agreed. He wrote a book entitled, Against Method: Outline of an Anarchistic Theory of Knowledge. In this book, he argued that there is no scientific method, but that knowledge advances through chaos. However, what Bacon and Newton missed, Feyerabend glimpsed through a haze, and Einstein understood, is that model construction is a creative, not algorithmic, process: the intuition that rests on an empathic understanding of experience. Science does not function by deducing models from observations. Rather, we construct models and compare their predictions with what is observed. A falling apple inspired Newton, rising water in a bath inspired Archimedes, a dream inspired Kekulé (the structure of benzene). Or at least, that is how the stories go. For model construction, Feyerabend is correct; anything goes—dreams, divine inspiration, pure luck, and especially hard work. Creativity by its very nature is chaotic and erratic.
We start with observations and ask, “what is the simplest model that could account for these observations?” Once a model is constructed, it is tested by the much more algorithmic or deterministic process of making predictions and checking them against observation. Now, one of the necessary constraints in building scientific models is simplicity. Without simplicity, we can get nowhere: an infinite set of models describe any finite set of measurements. That is why we cannot ask questions such as, “what model do these observations imply?” There are infinitely many such models.
Yet, simplicity comes at a price. For the sake of argument, take simplicity to be defined in terms of Kolmogorov complexity. This is a measure of the computational resources needed to specify the model. There is a theorem that says the Kolmogorov complexity cannot be determined algorithmically. If we accept the above identification of simplicity, it then follows that scientific models cannot be constructed algorithmically from observations. So much for Francis Bacon, Newton, and induction. The identification is probably not exact, but never-the-less sufficiently close to reality to be indicative. Model building cannot be algorithmic, but rather, is creative.
The creative aspect of science is obscured by two things: the analytic aspect and the accumulative aspect. The analytic aspect of testing models tends to obscure the creativity in constructing them. We are so blinded by the dazzling mathematical virtuoso of Newton that we fail to see how creative the development of this three laws were. Aristotle got it all wrong, but Newton got it right—and he did it through creativity, not just math.
The accumulative nature of science gives a sense of inevitability that makes the creativity less obvious. The sense of creativity can also feel lost in the noise of a thousand lesser persons. To quote Bertrand Russell:
In science, men have discovered an activity of the very highest value in which they are no longer, as in art, dependent for progress upon the appearance of continually greater genius, for in science, the successors stand upon the shoulders of their predecessors; where one man of supreme genius has invented a method, a thousand lesser men can apply it. No transcendent ability is required in order to make useful discoveries in science; the edifice of science needs its masons, bricklayers, and common labourers as well as its foremen, master builders, and architects. In art, nothing worth doing can be done without genius; in science even a very moderate capacity can contribute to a supreme achievement.
While we common labourers may not be creative geniuses, the foremen, master builders, and architects are. When it comes to creativity, Isaac Newton, Charles Darwin, and Albert Einstein do not take a back seat to writers like William Shakespeare, Charles Dickens, or James Joyce, nor to painters like Michelangelo, Vincent van Gogh, or Pablo Picasso.
Additional posts in this series will appear most Friday afternoons at 3:30 pm Vancouver time. To receive a reminder follow me on Twitter: @musquod.
