“The system is not designed for replications”
The verification of research results by means of replication studies is decisive for safeguarding the reliability of science. However, such replication studies have not so far attracted a great deal of attention from researchers. The Münster Center for Open Science (MüCOS) aims to remedy this and has therefore started up a series of scientific publications entitled “Replication Research”. In this interview with Linus Peikenkamp, psychologist Dr. Lukas Röseler, the Managing Director of MüCOS, explains why replications are indispensable for preserving quality in science and what the role of the new publication is.
It might help one or other of our readers, as an introduction to the topic, if you could describe for us a specific example of a replication …
Well, imagine if you’re cooking pasta and want to calculate the best possible cooking time. You enter the data for each run-through in a diagram. In the case of a reproduction of the procedure, you would again evaluate the data you have in order to check on the result. In the case of a replication, you would again cook pasta over a longer period of time, and under the same conditions, in order to collect new data. In other words: replications are scientific investigations which verify results already published. In the process, either the available data are assessed anew in order to test whether findings are correct or not – and in this case we call this a reproduction – or, alternatively, a study is again carried out, under the same conditions which existed in the original study, in order to test whether the findings can be generalised. We call this process replication.
Would you agree with me that replication studies don’t attract much interest?
Up to a point. It depends on the discipline. In sociopsychology or linguistics, replication studies are widespread. In many subjects, however, researchers rely more on new studies carried out by themselves.
What’s the reason for that?
Many traditional publications like new findings in preference to older studies. Especially as far as young researchers are concerned, this leads to pressure on them to innovate because they want to prove themselves in their scientific community – and rightly enough. It becomes problematic, though, when scientific quality suffers as a result. Moreover, verifying studies can be seen as criticism of the original authors. In brief: the rewards system in science is not designed for replication studies. Over the past few years this has even led to a “replication crisis”. In some subjects, experts assume a replication quota of only 50 percent.
Does this mean, in other words, that 50 percent of scientific findings cannot be confirmed in new studies?
The figure needs to be treated with some caution because in many subjects only about 0.1 percent of the studies have been replicated. Also, the results vary. Depending on the way the success of a replication is calculated, the success rate can be anything between 30 and 60 percent. This shows that there is still work that needs to be done on the replication method.
What does all this mean for trust in science overall?
A low replication quota goes hand in hand with a slight loss of trust in the field of research concerned. However, this loss of trust can be compensated for by efforts on the part of researchers to make scientific processes freely accessible and transparent.
And this presumably is where the new MüCOS series of publications comes in?
Exactly. We publish not only replications but also articles on methods of replication which offer guidance to researchers in their work. Anyone from the field of science who has an interest in the subject can submit an article. The entire review process is public and verifiable.