Planning scientific discoveries is not like building a railroad. Extreme accuracy in the analysis of projects can be sometimes counterproductive, similarly to what happens in Darwinian evolution.
The interesting discussion by Doroty Bishop (Nature, 584, 9 (2020)) on how to avoid cognitive biases in the scientific and statistical analysis is technically perfect but still it can be disorienting if one looks at how major scientific discoveries are actually made. In the early eighties I was a young university professor full of enthusiasm and good intentions. I began to get research projects to referee and set my optimal rules for this task pretty much in the line of Dorothy’s view. Intellectual honesty, competence, clarity of the objectives and unbiased statistical analysis were my rules which kept me happy for a few years. But after a while three major discoveries occurred, all Nobel prizes: The Quantum Hall Effect (QHE) by Von Klitzing; The High Tc Superconductivity by Bednorz and Muller and the Scanning Tunneling Microscope (STM) by Binning and Rohrer. All three were, broadly speaking, in my field of expertise and I even knew personally most of the protagonists.
Well, to my great disconcert I realized that, as a potential referee of these projects, my “perfect” system of analysis would have led to the rejection of all of them, and with perfectly good reasons. The original project of von Klitzing had little to do with the magic topological properties of the QHE; Alex Muller’s original idea of how to reach high Tc SC was based on the softening of phonons near a structural instability, but this is not really the mechanism. Even the STM was supposed to be a metallurgical tool to explore the surface of metal with a resolution of 500 atoms. Nobody could have predicted that the tip would rearrange to give information at the level of individual atoms.
So, I realized that my perfect system, honest, unbiased and competent, would have killed all these three major discoveries. The lesson I learned was that a too meticulous analysis based on what you know can be problematic to explore what you don’t know. But in terms of positive hints on how to improve my analysis there was little to learn. After a few years an interesting discussion with Stuart Kauffman on how birds can fly gave me some intellectual relief. The Darwinian development of wings was certainly not motivated by the hope to fly because below a certain size, you certainly don’t fly. The evolutionary line for wings was something else (it seems balancing the running or cooling blood) and only when they reached a certain size this evolutionary line met the (unplanned) evolutionary line of flying. Similarly, in the three discoveries above there was a certain line of research and then, unexpectedly, something else appeared. Actually, in the case of Muller he was really looking for High Tc SC but along a line which was, in some sense, different from the actual one. The merit of these scientists was to go along some line in a consistent and professional way but also to be able to realize of the new discoveries which suddenly appeared.
Note that something similar happened to Columbus traveling to America (he miscalculated the earth radius) and in the properties of the radio by Marconi (he thought radio waves could follow the curvature of the earth). May be after all this is what is called serendipity and it is a natural characteristic of experimental science.
But also, in theory a sort of educated bias, or intuition seems to play a major role. Some time ago I realized that the statistical methods used by cosmologists in analyzing the distribution of visible matter assumed a priori that this must be homogeneous at some scale, so they considered only the question of “when” it becomes homogeneous, but not “if” it becomes homogeneous. Technically this implied turning the amplitude of a power law correlation into a correlation length, which is a capital sin for all those who are familiar with complex and fractal structure. It was a clear bias motivated, however, by different observations. So, we repeated these analyses and found that in all the available samples the distribution of galaxies did not show any homogeneity. In 1996, at a conference in Princeton I had a public debate with Jim Peebles (Nobel 2019) and his group. The hall was so crowded that Phil Anderson was not allowed to enter by the security and I spoke to him only afterwards. He asked me what the comment of Peebles to my arguments was. I replied that Peebles argued that I only showed the data in my favor. Actually, I wanted to add that instead I showed all the available data, but Phil suddenly interrupted me by saying: “Of course, what else should one do?”.
A few years later, around 2000, in Trieste I had another interesting discussion with Phil on the much-debated subject of High Tc SC. Phil had argued since several years that the problem was obviously solved by the properties of strongly correlated electrons, beyond the Fermi Liquid theory. When I asked him, what was in his view the active principle for the increase of Tc from strong correlation, his answer was: “Well, I wish I knew that!”. So, his position was more a prophetic wish rather than a scientific proposition. Was this good or bad for science? Who knows? The fact is that today it is still not clear how strong correlation may increase Tc. Finally, the toast proposed by Phil at the end of each dinner was always: “Against common wisdom!”. Not exactly an unbiased point of view.
In summary scientific discoveries resemble Darwinian evolution in which one looks for new things in a new space. A strategy which is too strictly based on what is known may lead to good incremental progress but hardly to a real breakthrough. Elements like an educated bias, interpreted as intuition or creativity can give an artistic touch and sometimes lead to important results. The evaluation of these elements is intrinsically difficult but for sure they make the scientific game more fun.