The multiple-comparison trap and the Raven’s paradox—perils of using null hypothesis testing in environmental assessment
Environmental Monitoring and Assessment
Links
- More information: Publisher Index Page (via DOI)
- Download citation as: RIS | Dublin Core
Abstract
Detecting and quantifying environmental thresholds is frequently an important step in understanding ecological responses to environmental stressors. We discuss two statistical issues often encountered in threshold detection and quantification when statistical null hypothesis testing is used as a main analytical tool.The hidden multiple-comparison trap (leading to a much higher risk of a false detection) and Raven’s paradox(rendering a "detection" meaningless) are often obscured when statistical hypothesis testing is used as part of a more elaborate model, especially models based on computer-intensive methods. Using two examples, we show that the hidden multiple-comparison trap can be exposed using computer simulation to estimate the probability of making a false detection; Raven’s paradox can be avoided by clearly stating the null and alternative hypotheses using scientific terms to substantiate that the rejection of the null is equivalent to proving that the alternative of interest is true. The hidden multiple-comparison trap implies that a null hypothesis testing based on a computer-intensive method should be used with caution. The implication of Raven’s paradox re-quires that we focus on providing evidence supporting the proposed hypothesis or model, rather than seeking evidence against the frequently irrelevant null hypothesis. These two problems, and many others related to null hypothesis testing, suggest that statistical hypothesis testing should be used only as a component of the body of evidence, perhaps, as the devil’s advocate.
Suggested Citation
Qian, S.S., Cuffney, T.F., 2019, The multiple-comparison trap and the Raven’s paradox—perils of using null hypothesis testing in environmental assessment: Environmental Monitoring and Assessment, v. 190, no. 7, 409, 9 p., https://doi.org/10.1007/s10661-018-6793-1.
ISSN: 1573-2959 (online)
ISSN: 0167-6369 (print)
| Publication type | Article |
|---|---|
| Publication Subtype | Journal Article |
| Title | The multiple-comparison trap and the Raven’s paradox—perils of using null hypothesis testing in environmental assessment |
| Series title | Environmental Monitoring and Assessment |
| DOI | 10.1007/s10661-018-6793-1 |
| Volume | 190 |
| Issue | 7 |
| Publication Date | June 18, 2018 |
| Year Published | 2019 |
| Language | English |
| Publisher | Springer International Publishing AG |
| Contributing office(s) | South Atlantic Water Science Center |
| Description | 409, 9 p. |
| Online Only (Y/N) | N |