Another approach to handling ANOVA type analyses when the assumptions are violated is to use resampling, as described in Resampling Procedures.
Example 1: Repeat Example 1 of Kruskal-Wallis using bootstrapping (the data is repeated in Figure 1).
Figure 1 – Sample data
The sample data contains 27 data elements: 10 New, 9 Old and 8 Control. As can be seen in Kruskal-Wallis, the data violates the homogeneity of variance assumption, and so we can’t be sure whether ANOVA will yield valid results. We therefore use the Resampling data analysis tool as follows.
Press Ctrl-m and double-click on the Resampling data analysis tool from the menu. Next fill in the dialog box that appears as shown in Figure 2 and click on the OK button.
Figure 2 – Resampling dialog box
The output is shown in Figure 3.
Figure 3 – Bootstrapping test for ANOVA
The data analysis tool first calculates the F-stat for sample data. This can be done using the Excel or Real Statistics One-sample ANOVA data analysis tool or via the ANOVA1 function. For Example 1, F-stat = ANOVA1(A4:C13) = 2.109681.
The data analysis tool now creates a new sample of size 27 (the size of the orginal sample) by randomly drawing 27 elements from the the original sample with replacement and places the first 10 in the New group, the next 9 in the Old group and the remaining elements in the Control group. It now calculates the F-stat for this new sample. This is repeated 10,000 times (since Iterations is set to 10,000 in Figure 2).
For each iteration, the data analysis tool determines whether the bootstrap F-stat is larger than 2.109681 (the F-stat for the original sample). The p-value for the test is equal to the count of bootstrap F-stats > 2.109681 divided by 10,000. As we can see from Figure 3, for Example 1, p-value = .1452 (cell P26). Based on α = .05, this means that we cannot reject the null hypothesis that the three groups have equal means.
Observation: The analysis can also be done using randomization. The approach is identical to that described above, except that the samples of size 27 are done without replacement (e.g. by using the SHUFFLE function instead of the RANDOMIZE function).
Alternatively, the sampling can be performed on the residuals (i.e. the raw data minus the appropriate group mean) instead of the raw data, using either bootstrapping or randomization. For Example 1 this can be done by selecting the ANOVA (via errors) option in the Resampling dialog box as shown in Figure 2. The output from the randomization version of the test is shown in Figure 4.
Figure 4 – Randomization test on residuals for ANOVA