Common Mistakes Researchers Make When Interpreting P-Values

P-value misinterpretation is so common that statisticians have published entire papers cataloguing researcher errors. Dissertation students are especially vulnerable: SPSS outputs a number, supervisors expect significance, and the pressure to support hypotheses is intense. The mistakes below appear in student theses, published articles, and conference presentations worldwide. Recognising them in your own work before submission can save revision cycles and viva embarrassment.

Mistake 1: Confusing statistical and practical significance

A p-value tells you whether an effect is unlikely under the null—not whether it matters in practice. A training programme that improves scores by 0.3 points on a 100-point scale may be p < .001 with n = 10,000 but useless for policy. Always report and discuss effect sizes.

Mistake 2: Treating p-values as probability H0 is true

Students often say 'there is a 4% chance the null is true.' That is incorrect. The p-value is the probability of the data given H0, not the probability of H0 given the data. Reversing this conditional probability is a fundamental logical error.

Mistake 3: The .05 cliff and approaching significance

• p = .051 is not meaningfully different from p = .049—both are close to the threshold.
• Labelling p = .08 as approaching significance without pre-specified marginal alpha is post-hoc rationalisation.
• Binary thinking (significant vs not) ignores the continuous nature of evidence.
• Report exact p-values and let readers judge alongside effect sizes.

Mistake 4: Ignoring non-significant results

Hiding non-significant hypotheses, moving them to appendices, or rewriting the discussion to avoid mentioning them damages credibility. Non-significant results are informative—they may indicate low power, weak manipulation, or genuine absence of an effect. Discuss them honestly.

Mistake 5: P-hacking and flexible analysis

• Running many tests and reporting only significant ones.
• Stopping data collection once p < .05.
• Trying different covariates until significance appears.
• Removing outliers selectively to improve p-values.
• Switching from two-tailed to one-tailed after seeing direction of effect.

Mistake 6: Wrong test, wrong p-value

Using a t-test on non-normal data with tiny samples, ignoring violated assumptions, or applying parametric tests to ordinal data produces p-values that are technically output but inferentially invalid. Check assumptions before trusting any p-value SPSS prints.

Mistake 7: Overinterpreting significant correlations

A significant correlation does not imply causation. A significant p-value for r = .18 may be statistically significant with large n but explain only 3% of variance. Report r² and discuss whether the relationship is theoretically meaningful.

Mistake 8: Misreading SPSS output

• Reading the wrong row in Levene's or equal variances tables.
• Using asymptotic p-values when exact tests are available.
• Reporting p = .000 instead of p < .001.
• Confusing significance of the model (F-test) with significance of individual predictors.

Mistake 9: Equating significance with hypothesis confirmation

Rejecting H0 is not the same as confirming your theoretical model. Your alternative hypothesis may be one of many explanations for a significant result. Competing explanations, confounds, and alternative models belong in your discussion.

Mistake 10: Neglecting power and sample size

A non-significant result in an underpowered study tells you little. A significant result in an overpowered study may detect trivial effects. Document sample size justification and, where possible, report post-hoc power or confidence intervals.

How to self-audit your p-value interpretation

1. 1List every hypothesis and its pre-specified test before looking at output.
2. 2Verify each test matches variable types and design.
3. 3Report exact p-values with test statistics and effect sizes.
4. 4Discuss both significant and non-significant findings.
5. 5Ask a colleague to read your results without seeing SPSS output—can they follow your logic?

Professional data analysis support

If test selection, SPSS output interpretation, or results chapter writing is blocking your dissertation timeline, ReportLift data analysis support helps you run valid tests, interpret findings correctly, and report results to examiner and journal standards.