- H0 / H1 : the null hypothesis of no difference, and its alternative.
- p-value : the probability of observing such a difference if H0 were true. If this probability is low, it casts doubt on H0.
- Alpha risk : the risk of asserting a difference that does not exist. Determined in advance, often 5 %.
- Beta Risk : risk of missing a real difference.
- Power : 1 − beta, the probability of detecting a true difference.
- Parametric Test : assumes a law; more powerful if the assumption holds.
- Nonparametric test : does not assume any law; more robust.
- Paired data : two sets of measurements taken on the same individuals.
- Bilateral / unilateral : «different from» versus «greater than.» You decide on the meaning before looking at the data.
- BOB / WOW : the best and worst pieces, selected to highlight the contrast.
- Equivalence : a conclusion stating that two populations do not differ by more than a predetermined threshold. This is the opposite of a test for a difference, not a failure of that test.
- δ (equivalence limit) : the threshold below which the difference has no practical significance. A professional decision made before reviewing the data.
- Zone of Indifference : the interval ±δ. Equivalence is demonstrated when the confidence interval for the difference y is entirely contained within it.
- TOST : Two One-Sided Tests, the method used two one-tailed tests, one for each end point.
- Non-inferiority / non-superiority : one-way versions, for when only one side of the gap is causing you trouble.
Appendix
Reading time