This test is the same as the Tukey HSD Test (see Unplanned Comparisons), except that we test the difference between rank sums and use the following standard error
where k = the number of groups and m = the size of each of the group samples. The group sample sizes must all be equal. The statistic has a studentized range distribution (see Studentized Range Distribution). The critical values for this distribution are presented in the Studentized Range q Table based on the values of α, k (the number of groups) and df = ∞. If q > qcrit then the two means are significantly different.
This test is equivalent to Rmax − Rmin > qcrit ⋅ s.e.
Picking the largest pairwise difference in means allows us to control the experiment-wise for all possible pairwise contrasts; in fact, this test keeps the experiment-wise α = .05 for the largest pairwise contrast, and is conservative for all other comparisons.
Real Statistics Data Analysis Tool: The Real Statistics Resource Pack provides a data analysis tool to perform the Nemenyi test, as shown in Example C.
Example 1: Conduct the Nemenyi Test for the data in range B3:D11 of Figure 2 to determine which groups are significantly different.
Press Ctrl-m, double click on Analysis of Variance option and select Single Factor Anova. When a dialog box similar to that shown in Figure 1 appears, enter B3:D11 in the Input Range, check Column headings included with data, select the Kruskal-Wallis and Nemenyi KW options and click on OK.
Figure 1 – Selecting Kruskal-Wallis and Nemenyi Tests
The output is shown in Figure 2.
Figure 2– Nemenyi Test
The Kruskal-Wallis Test (the middle part of Figure 2) shows there is a significant difference between the three groups (cell J12). Since the three groups are equal in size we use the Nemenyi test to determine which groups are significantly different.
The template for the Nemenyi test is generated by the Real Statistics data analysis tool, as shown on the right side of Figure 2). You begin by inserting a 1 and -1 in cells M3 and M4 to compare the New and Old groups. The difference between the rank sums of these two groups is 76.5 (cell N6), which is greater than 66.28, the value of x-crit (cell P9), we conclude there is a significant difference bwteen the New and Old groups.
We can compare the New and Control group in the same way (removing the -1 from cell M4 and inserting -1 in cell M5) and see that there is no significance between these groups. Similarly there is no significant difference between Old and Control.
Some key formulas from Figure 2 are shown in Figure 3.
Figure 3 – Selected formulas from Figure 2
This test is similar to the above test and can be viewed as a version of Nemenyi’s test when the sample sizes are unequal.
Dunn’s test uses the statistic
and the standard error is
If there are a lot of ties, an improved version of the standard error is given by
where f is as in the ties correction for the Kruskal-Wallis test. This test is equivalent to
Here the term in parentheses is a Bonferroni-like correction.
Real Statistics Data Analysis Tool: The Real Statistics Resource Pack provides a data analysis tool to perform Dunn’s test, as shown in Example 2.
Example 2: Find all significant differences between the blemish creams of Example 1 of Kruskal-Wallis Test at the 95% significant level.
We repeat the data from Example 1 of Kruskal-Wallis Test in range B3:D13 of Figure 4. As we saw from the Kruskal-Wallis analysis, there is a significant difference between the three groups. Unlike in Example 1, this time we use Dunn’s test since the group sizes are different.
To perform this test, we proceed as in Example 1, except that we choose the Dunn KW option instead of the Nemenyi KW option. When we press the OK button the result shown on the right side of Figure 4 is displayed.
Figure 4 – Dunn’s Test
We see that there is a significant difference between the New and Old creams. If we change the contrast coefficients in range G5:G7, we see that there is no significant difference between New and Control and between Old and Control.
Schaich-Hamerle Test is similar to Dunn’s test, but it uses the chi-square distribution instead of the Studentized q range distribution. Once again, pairwise differences of the average ranks
are compared with the critical value
Here χ2α,k-1 is the critical value of the chi-square distribution for the given alpha and k – 1 degrees of freedom. The difference between the ith and jth groups is significant if dcrit < d.
Example 3: Find all significant differences between the blemish creams of Example 2 at the 95% significant level.
We summarize the results of the above analysis for the 3 pairwise comparisons in Figure 5.
Figure 5 – Schaich-Hamerle test for Example 3
Here χ2α,k-1 (in cell B36) = CHIINV(B35, B34–1) = 5.9915. For the comparison of the new and old creams, d = D29–D30 = 10.25 and dcrit = SQRT(B36*B33*(B33+1)/12*(1/B29+1/B30)) = 8.9267, and similarly for the other two comparisons.
The only significant comparison at the 95% significance level is between the new cream and the old cream where p < .05 since d > dcrit.
After a significant Kruskal-Wallis test, we can compare a control group with each of the other groups, in a manner similar to that used in Dunnett’s test after a one-way ANOVA. The test is also similar to the Nemenyi test, except that this time we use the Dunnett’s Table.
Real Statistics Data Analysis Tool: The Real Statistics Resource Pack provides a data analysis tool to perform Steel’s test, as shown in Example 4.
Example 4: Determine whether there is a significant difference between the Control and the New and Old groups for the data in Example 1.
After choosing the Steel test option from the One Factor ANOVA dialog box (see Figure 1) and filling in the contrast coefficients, we obtain the results shown in Figure 6.
Figure 6 – Steel’s Test after KW Test
We see there is no significant difference between the Old and Control groups. If we move the -1 contrast coefficient to cell G5, we would see there is no significant difference between the New and Control groups.
Contrasts can be used after a Kruskal-Wallis test as for one-way ANOVA. A contrast C is defined based on the contrast coefficients by
Taking ties into account the formula for the standard error becomes
The square of the contrast C2 is then tested using a chi-square distribution with n−1 degrees of freedom.
Example 5: Determine whether there is a significant difference between the Control and the average of the New and Old groups for the data in Example 2.
This time we choose the Contrasts KW option from the One Factor ANOVA data analysis tool. We see from Figure 7 that there is no significant difference.
Figure 7 – Contrasts after KW Test