Master statistical reporting vocabulary for technical writing: p-value, confidence intervals, effect size, statistical vs. practical significance, null hypothesis, Type I/II errors, and correct use of 'significant'.
0 / 5 completed
1 / 5
What does a p-value of 0.03 mean in the context of statistical reporting, and how should it be written in a research paper?
p-value reporting conventions: use p = .03 (not p = 0.03 in APA style, no leading zero), or p < .001 for very small values. The p-value answers: 'If the null hypothesis were true, how often would we see data this extreme by chance?' p = .03 means this would happen 3% of the time by chance. Common errors: (1) 'The p-value is the probability the null hypothesis is true' — incorrect. (2) Reporting p < .05 without effect size. (3) Treating p = .049 as categorically different from p = .051.
2 / 5
What is a confidence interval and why is it more informative than a p-value alone in technical reporting?
CI reporting: 'The mean response time improved by 120ms (95% CI [45ms, 195ms]).' This tells you: the best estimate is 120ms, and the plausible range is 45–195ms. A p-value of .03 alone tells you the effect is unlikely to be zero — the CI tells you its practical size. Wide CI (e.g., [-10ms, 250ms]) signals imprecision and overlaps zero. Narrow CI signals precision. Report CIs alongside p-values: 'The difference was statistically significant (p = .03) with a mean improvement of 120ms (95% CI [45, 195]).'
3 / 5
What is the distinction between 'statistical significance' and 'practical significance' (effect size) in technical writing?
Example: A/B test with 1 million users finds a 0.1% conversion rate improvement (p < .001, Cohen's h = 0.002). Statistically significant (large N), but practically negligible (tiny effect). Conversely, a small pilot study with 20 users finds a 30% improvement (p = .08, Cohen's d = 0.9). Not statistically significant (small N), but the effect size is large and worth further investigation. Writing: 'Although statistically significant, the effect was small (d = 0.12), suggesting limited practical importance.' Effect size conventions: Cohen's d: small 0.2, medium 0.5, large 0.8.
4 / 5
What are Type I and Type II errors in hypothesis testing and how should they be discussed in a limitations section?
Limitations language: 'By using alpha = .05, we accept a 5% Type I error rate — approximately 1 in 20 tests would appear significant by chance.' 'Given our sample size of n = 30, our study may have insufficient power (estimated power = 0.61) to detect small effects, increasing the risk of a Type II error.' In multiple comparisons (testing many hypotheses), Type I error accumulates — address with Bonferroni correction or FDR adjustment: 'We applied Bonferroni correction to control the family-wise Type I error rate.'
5 / 5
When is it appropriate to use the word 'significant' in technical and academic writing, and what are common misuses?
Misuses of 'significant': (1) 'There was a significant improvement in user satisfaction' in a qualitative study — no statistics were performed. Use 'marked improvement' or 'substantial improvement.' (2) 'The difference was not significant' meaning 'not important' rather than 'did not reach p < .05.' (3) 'Highly significant' (p = .001) implying a large effect — significance level does not indicate effect size. Correct usage: 'The difference in error rates was statistically significant (p = .02, d = 0.6), representing a practically meaningful reduction.' Always pair statistical significance with effect size.