Power is the probability of detect a discrepancy that actually exists. Its complement is the beta risk : to overlook a real difference.
This page relates four quantities, three of which determine the fourth:
- the sample size ;
- the tested difference : the difference you want to be able to detect; ;
- dispersion: sigma, or the sigma ratio ;
- the risks alpha and beta.
Tests covered:
- Test Z and T Test compared to a theoretical value,
- T Test To compare two means,
- Paired T-test,
- ANOVA for several averages,
- Chi-Square Test and Test F on standard deviations,
- Test 1P and Test 2P on proportions.
- One section covers the 2k design of experiments, depending on the number of factors, the number of rows, and the number of second-order terms.
Example: Calculate the sample size needed to detect a difference of 15% to 5% between two lots: (answer: 111 units in each lot)
💡 This is the page you should open before the test, not after. It answers the question, «How many samples do I need to measure?»—a question whose answer determines everything else. Conducting an undersized experiment guarantees a «not significant» result that proves nothing and will have to be repeated.
⚠️ The implication is unpleasant but useful: detecting a small deviation is costly. Halving the detectable deviation quadruples the required sample size. It’s best to know this before starting the tests.