Study Finds 3 out of 4 Climate Science Papers Use Flawed Statistical Analysis

Fascinating.

Significance Tests in Climate Science

Maarten H. P. Ambaum

Department of Meteorology, University of Reading, Reading, United Kingdom

A large fraction of papers in the climate literature includes erroneous uses of significance tests. A Bayesian analysis is presented to highlight the meaning of significance tests and why typical misuse occurs. The significance statistic is not a quantitative measure of how confident one can be of the “reality” of a given result. It is concluded that a significance test very rarely provides useful quantitative information.

We tested a recent, randomly selected issue of the Journal of Climate for at least one instance in each article of misusing a significance test to quantify the validity of some physical hypothesis. The Journal of Climate was not selected because it is prone to include such errors, but because it can safely be considered to be one of the top journals in climate science. In that particular issue, we observed a misuse of significance tests in about three-quarters of the articles; a randomly selected issue published 10 years prior showed such misuse of significance tests in about half of the articles. The examples mentioned above were rephrased from those articles. The two sampled issues perhaps would not pass a traditional significance test, but they do indicate that such errors occur in the best journals with the most careful writing and editing. Indeed, in one of this author’s papers, such erroneous use occurred. …

This paper comes from the same guy I recently cited on space weather being responsible for cloud formation.