Blog Tag statistics
What Statistics Can and Can’t Tell Us About Ourselves
Interesting to see the P-value crisis has made it to the New Yorker. It is indeed time to rethink the way we employ the iconic measure of the "success" of an experiment. I have literally had postdocs in the lab come to my office and tell me that a given experiment "failed" because the P-value was 0.051. That is not the appropriate way to approach science. In another case, a postdoc came to me with the results of a multivariate model. The overall model statistics were relatively weak, so he considered the experiment to be non-informative. However, an inspection of the loadings plot showed that all of the identified metabolites came from the same enzymatic pathway. This article reports that "In medicine, a study of forty-nine of the most cited medical publications from 1990 to 2003 found that the conclusions of sixteen per cent were contradicted by subsequent studies." So how do we deal with this brave new world in which P-values no longer light the way? We move forward with common sense, biochemical knowledge, replication, validation and most importantly, the courage to question our findings.
Is science loosing its way??
The rage over reproducibility in science continues. I must admit that I find myself perplexed that this has even become an issue. To me it seems completely and utterly natural that science requires replication. The field of discovery and inquiry demands replication. A single discovery, while exciting, is only that - a single report. If you believe in your science - and your findings, would you not welcome replicaiton? If your findings are replicated - then the field will continue to move forward. If your findings are not replicated - then everyone has to take a step back and figure out what happened. Of course the real problem here is the issue of how we publish our research. The constant push to publish new findings makes the science of replication unglamorous at best - and top journals are certainly not interested in publishing these studies. The journals' push to make our Methods sections as short as possible - or even shoved to the oblivion of Supplemental Material - certainly exascerbates this problem. What could be more important than a detailed presentation of the exact methods used to produce the published result? Yet academic publishing considers the Methods to be almost irrelevant to the paper. It is past time for scientists to demand a better model. We want to be left alone to do perform our science, but we cannot ignore what is happening in the field. Publishing needs a new paradigm - and it is up to scientists to make it happen.
Leave a message