Post

Why base important decisions on One Hit Wonders?

No Comments

Reliable evidence is an answer to a question that was found and verified multiple times in independent data sets. It means that doctors and nurses can make decisions with predictable outcomes. Tens of thousands of clinical trials and thousands of systematic reviews being published every year at great cost. Each is a piece of evidence which is unreliable on its own. What make evidence reliabile is verification through reproduction of the results by independent team on independent data sets.

Recently, our co-founder Prof. Enrico Coiera published a systematic review of trials for clinical decision support systems (CDS). Enrico and team found thousands of such trials but less than 0.03% of them are actually replications of another 0.03% of the studies. This means that 99.4% of the evidence produced at great cost, are unreliable. This wastage is the tragedy of the commons. It not only diverts resources from reliable to unreliable evidence, in some cases the lack of accountability also leads to trial designs that cut corners. In other words, when trialists know that their results will not be verified, they are more likely to design trials that don't withstand verification; studies with incredible results that are dubbed "one hit wonders".

Further reading: An interview with Professor Enrico Coiera about the study

The main inhibition to reproducing research is that both original research and reproduction require about as much effort and resources. Evidentli encourages research reproduction by making reproduction of research between two Piano instances as simpe as copy and paste. There is no need for complicated data sharing agreements because there is no need to share data. Simply, copy a research workflow and run it on your own data. Yes. it's that simple.

Recent Posts

Leave a Reply

Comments, questions or queries? Let us know!"