Skip to content

Research quality and reproducibility

Published: 28 October 2019

Last week, I was in Stockholm at the Federal Assembly of Swedish universities (SUHFs förbundsförsamling). The focus of the day was Research Quality and Reproducibility; probably the most important subject we researchers have to reflect and debate upon.

That our current system, often referred to as a publish or perish system, is neither perfect nor sustainable is well known. Some of the flaws or challenges are the strong focus on the number of articles published rather than the quality of each article as well as quantitative quality measures (citations, impact factor, downloads) rather than qualitative measures (the excellent study design, the transparency, the reproducibility), but also the focus on new significant findings compared to reproducibility.

John P.A. Ioannidis, professor at Stanford, presented some extremely interesting but alarming research results related to reproducibility of quantitative research studies in a number of fields. While 96% of the scientific literature claims significant results while only 50% of all papers are reproducible. In some fields the level of reproducibility is as low as 1%. When asking a research team to replicate their own study and its results, they were often not able to do so.

Threats related to reproducibility 

Some of the threats listed by professor Ioannidis in relation to reproducibility are:
  • Lack of replication
  • Hypothesizing after the results are known
  • Poor design study
  • Low statistical power
  • P-hacking
  • Publication bias
  • Lack of data sharing
He also discussed the five Hong Kong principles for assessing researchers and fostering research integrity: 
  • Assessing researchers on responsible practice from conception to delivery. 
  • Value accuracy and transparent reporting of all research, regardless of the results.
  • Value practices of open science and open research.
  • Value a broad range of research and scholarship, from innovation to replicability such as peer-review – not only publish papers
  • Value a range of other contributions to responsible research and scholarly activity
The conclusion of the day was that our publication culture needs to change; we need to return to a focus on quality, not quantity. The big question is, however, how to manage this. 
Toppbild rektors blogg