That our current system, often referred to as a publish or perish system, is neither perfect nor sustainable is well known. Some of the flaws or challenges are the strong focus on the number of articles published rather than the quality of each article as well as quantitative quality measures (citations, impact factor, downloads) rather than qualitative measures (the excellent study design, the transparency, the reproducibility), but also the focus on new significant findings compared to reproducibility.
John P.A. Ioannidis, professor at Stanford, presented some extremely interesting but alarming research results related to reproducibility of quantitative research studies in a number of fields. While 96% of the scientific literature claims significant results while only 50% of all papers are reproducible. In some fields the level of reproducibility is as low as 1%. When asking a research team to replicate their own study and its results, they were often not able to do so.
Threats related to reproducibility
- Lack of replication
- Hypothesizing after the results are known
- Poor design study
- Low statistical power
- Publication bias
- Lack of data sharing
- Assessing researchers on responsible practice from conception to delivery.
- Value accuracy and transparent reporting of all research, regardless of the results.
- Value practices of open science and open research.
- Value a broad range of research and scholarship, from innovation to replicability such as peer-review – not only publish papers
- Value a range of other contributions to responsible research and scholarly activity