3.1 Choosing a Test
Ellistat presents the tests in a two-entry matrix : the family of online tests, the type of comparison in columns.
The Five Families, depending on what you're comparing:
| Family | You're comparing | Typical Question |
| Positions | Averages | Do these two settings yield the same average measurement? |
| Ladders | Dispersions | Is this machine more consistent than the other one? |
| Distributions | Entire distribution forms | Do these two sets follow the same pattern? |
| Ranks | Positions, without any legal assumptions | Same question as in *Positions*, but without assuming normality. |
| Frequencies | Proportions | Has the rejection rate changed? |
Types of Comparison, in columns:
- At a theoretical value : Does your sample meet a target, a standard, or a historical value?
- Between two samples, paired or independent ;
- Among several samples, paired or independent.
3.2 The Tests Offered
| Family | Theoretical value | 2 samples | Several samples |
| Positions | Theoretical Z, Theoretical T | Z-test, t-test — Paired t-test | ANOVA — Paired ANOVA |
| Ladders | Chi-Square Test | F Test / Fligner-Killeen | Bartlett, Levene / MAD Permutation |
| Distributions | (based on descriptive statistics) | Cramér-von Mises | Energy |
| Ranks | Sign Test, Wilcoxon Test | Mann-Whitney Test, B to C | Kruskal-Wallis — Paired Friedman |
| Frequencies | Test 1P | Test 2P | Chi-Square Test |
When two tests are separated by a slash, the first one is parametric and the second nonparametric : The F-test and the Fligner-Killeen test address the same question, with the latter not assuming normality.
💡 Z or T?
The Z-test assumes that the population standard deviation is known, which is rarely the case in practice. The T-test estimates it from the sample. When in doubt, use the T-test.
Ellistat automatically selects the appropriate tests based on the type of data you want to compare. No need to remember all the tests—Ellistat provides you with the complete results of the appropriate tests. For a continuous variable, it provides tests for differences in location, scale, and dispersion! You can select just one of these categories.
3.3 Parametric vs. Nonparametric Tests
Parametric tests assume a distribution, usually the normal distribution. They are more powerful when this assumption holds: given the same data, they detect smaller differences.
Nonparametric tests make no assumptions about the distribution. They are more robust but a little less sensitive.
- Normal distribution, no outliers: parametric ;
- Questionable laws, a small workforce, and extreme values: nonparametric.
The menu displays the two side by side precisely to make this choice clear.
3.4 Validity requirements, verified for you
This is a highlight of the menu, and one that’s often underutilized: Ellistat automatically checks the assumptions of the selected test and displays the verdict.
The three badges can be:
- The conditions have been met
- The conditions are being met despite a deviation from the ideal
- At least one condition is not met
| Condition | What Is Verified |
| Normal Law | Normality assumption accepted or rejected. For a paired test, it concerns the differencebetween samples, not on each sample. |
| Equivalence of Variances | The null hypothesis of equal variances is accepted or rejected. Some tests of the mean are sensitive to this. |
| Outliers | The presence or absence of outliers, which can, on their own, create or mask a difference. |
When a test has no condition to verify, Ellistat indicates this: «There is no hypothesis to verify for this test.» This is the case for nonparametric tests.
⚠️ A parametric test for which the assumptions are not met yields a p-value which does not have the advertised value. The natural response: switch to the non-parametric equivalent of the same line, or address the cause, remove a justified outlier, or transform the data.
💡 Never remove an outlier just because it’s inconvenient. An outlier is a information : data entry error, production issue, out-of-process part. It's better to understand it than to delete it.
3.5 Read the result
The results show the assumptions H0 and H1 stated explicitly for your case, the statistics of the test, the p-value, and the conclusion along with its result: Rejection of H0 or Non-rejection of H0.
The accompanying graph goes with the test: box-and-whisker plots for comparing means, and bar charts for proportions.
💡 Always look at the graph and the p-value together. The graph shows by how much and in what sense, the p-value indicates if that's to be believed. One without the other leaves half of the conclusion out.
3.6 Multiple samples
A significant ANOVA or Kruskal-Wallis test indicates that «these groups are not all equal.» It does not say not which one is different.
To this end, Ellistat provides a conclusion in pairs, which compares the groups two by two.
⚠️ Comparing all pairs increases the number of tests—and thus the number of false positives: with ten groups, we perform forty-five comparisons, and with a risk of 5 % for each, we find an average of two spurious differences. That is why you should use the provided pairwise comparison feature rather than performing t-tests manually one after another.
3.7 The Matching Trap
Data is paired when both series focus on the the same individuals : the same part before and after treatment, the same operator using two methods, the same part measured by two instruments.
The paired test eliminates between-subject variability and becomes much more powerful.
⚠️ Treating paired data as independent is the most common and costly mistake: the variability between samples masks the effect being sought, leading to the conclusion that «there is no difference» when in fact there is a clear difference. Conversely, classifying data sets as paired when they are not creates differences.
💡 The test indirectly reminds you of this: for a paired test, normality is checked on the difference. If Ellistat mentions the difference between samples, it means you're indeed dealing with a paired sample.