ANOVA for main and interaction effects
Example 1: A new drug is tested on a random sample of insomniacs: 7 young people (20-40 yrs), 7 middle aged people (40-60 yrs) and 7 older people (60+ yrs). The number of minutes each person sleeps per night is recorded for 5 successive nights in the left side of Figure 1. Determine whether the drug is effective.
Figure 1 – Data plus additional information for Example 1
In addition to the raw data in range B4:F24, the figure displays key counts: Age (cell J5) = # of levels in the Age factor = 3, Days (cell K5) = # of levels in Days factor = 5 and Subject (cell L5) = # of subjects in each Age group = 7.
Marginal means of all the intersections of Age × Days are given in range J9:N11, along with their sums. E.g. cell J9 provides the mean number of minutes slept by all the young subjects on day 1 via the formula =AVERAGE(B4:B10) and cell K12 provides the mean for all the subjects on day 2 via the formula =AVERAGE(K9:K11). Finally the figure also provides a chart of Age × Days.
We now can create the ANOVA analysis using the table in Figure 2.
The implementation of Figure 2 in Excel is given in Figure 3. The SS terms of the first five items are derived using DEVSQ of the appropriate elements, while the other four are obtained by subtraction as described in Figure 2.
Figure 3 – Data elements from Figure 2
From this table we easily get the ANOVA report displayed in Figure 4.
Figure 4 – ANOVA for Example 1
Note that Subjects within Age is the error term for the between subjects analysis and Days x Subjects within Age is the error term for the within subjects analysis.
From Figure 4, we conclude there is a significant difference in Age group means (between-subjects analysis); i.e. the three age groups show a significant difference in the number of hours of sleep (averaged over the 5 days).
The figure also shows a significant difference in the Days means as well as the means of the interactions between Age groups and Days (within-subjects analysis). Thus there is a significant difference between the number of hours of sleep obtained by day (averaging over all three age groups) and there is a significant difference between the average hours of sleep by day and age group.
Assumptions for between-subjects analysis
The assumptions that need to be met for the analysis described in Figure 4 to be valid fall into two categories: assumptions for the between-subjects analysis and those for the within-subjects analysis.
The assumptions for the between-subjects analysis are the familiar ones from Two Factor ANOVA with Replication, namely
- All samples are drawn from normally distributed populations
- All populations have a common variance
- All samples were drawn independently from each other
- Within each sample, the observations were sampled randomly and independently of each other
To test for normality, for each Age group we need to test the mean values across Days for each subject, i.e. range G4:G10 from Figure 1 for Young, range G11:G17 for Middle and G18:G24 for Old. We show the corresponding box plot in Figure 5.
Figure 5 – Boxplots for within-subjects factor
With such small samples the results are quite limited, but there is no dramatic departure from symmetry.
The key assumption for the between-subject F test to be valid is that the variance of the subject means within any one level of Age is the same as the variance of the subject means within any other level in Age. This can be tested by Levene’s test, although ANOVA is quite robust against reasonable violations of this assumption. Since the groups are independent we don’t need to test for sphericity. This follows since if the variances are homogeneous, then the covariance matrices will be equal and compound symmetry will hold since the independence assumption implies that all the off-diagonal covariances are zero.
Figure 6 compares the variance of the subject means for the Young, Middle and Old samples, and shows via the Levene’s test there is no significant difference between these variances, and so the homogeneity of variances assumption is satisfied.
Figure 6 – Levene’s test for between-subjects factor
Each item in the range BU93:BW99 is the mean across the 5 days of the corresponding elements in the raw data. E.g. cell BU93 contains the mean of the first Young subject, i.e. the value of cell G4 (see Figure 1). The cells in range BU100:BW100 contains the means of the three Age groups. The cells in the range BY93:CA99 contain the deviation of each subject mean from the corresponding Age group mean; e.g. cell BY93 contains the formula =ABS(BU93-BU100). Levene’s test is carried out by performing a one-way ANOVA on the data in the range BY93:CA99. Since the p-value (in cell CH101) = .50 > .05 = α, we conclude there is no significant difference between the Age group variances.
Assumptions for within-subjects analysis
For the within-subject tests to be valid the following assumptions need to be satisfied:
- All samples are drawn from normally distributed populations
- The Days × Subjects population covariance matrices at each Age level are approximately equal, which in Example 1 means that ΣYoung ≈ ΣMiddle ≈ ΣOld
- Σ satisfies the sphericity assumption
We need to test each of the 15 Days × Age samples for normality (or at least symmetry). From the box plot we see in Figure 7 that this assumption is reasonably well satisfied.
Figure 7 – Box plots of Days × Age samples
Since we don’t have access to the population covariance matrices, we test the equality of the sample covariance matrices. Box’s M test can be used for this combined with Levene’s test for homogeneity of variances. Figure 8 shows the result of Levene’s test for Day 1.
Figure 8 – Levene’s test for Day 1 within-subjects factor
We can see that that variances are 5118, 4378 and 938, but the p-value of .09 > .05 shows that the variances are equal enough for our purposes. Levene’s test on the other days show that they too satisfy the homogeneity of variance assumption.
Box’s test for equality of covariance matrices yields the following result: M = 34.81, (30, 1027) = .67, p-value = .91 > .05 = α, and so we can conclude there is no significant differences between the covariance matrices. Click here for the details about how to use Box’s Test for the Equivalence of Covariance Matrices.
As usual instead of checking for sphericity, we will simply adjust the degrees of freedom to compensate for lack of sphericity by using Greenhouse and Geisser’s epsilon:
where k = the number of days (k = 5 in our example) and S is the sample covariance matrix. Since we are assuming that ΣYoung ≈ ΣMiddle ≈ ΣOld, it is sufficient to use the pooled sample covariance matrix (i.e. the average of the three group level covariance matrices). In our example, the three sample covariance matrices and the pooled matrix are shown in Figure 9.
Figure 9 – Covariance matrices for Example 1
The value of is calculated from the pooled covariance matrix exactly as was done in Figure 3 of Sphericity. See Figure 10.
Figure 10 – Calculation of GG epsilon and HF epsilon for Example 1
The HF epsilon is calculated by the formula used in Sphericity, namely
Here k = # of days = 5 and n = # subjects ∙ # Age groups = 7 ∙ 3 = 21. Thus, GG epsilon = .80 and HF epsilon = .998. Since GG epsilon > .75, we should use the HF epsilon correction factor. Since this is almost 1, it is safe to assume sphericity. In any case, the revised version of the omnibus ANOVA test from Figure 4 for the within-subjects factor is shown in Figure 11.
Figure 11 – Within-subjects ANOVA corrected for sphericity
More detailed within-subjects analysis
Although the GG epsilon and HF epsilon is very useful in addressing sphericity issues, it is still better not to use MSE (= 2241.748) as the error term in detailed analyses. In fact it is best to analyze simple effects on an individual Age group basis. This can be done by running a one-way repeated measures ANOVA for each Age group (or by skipping ANOVA and going directly to contrasts).
Figure 12 shows the analyses for each of the age groups (as described in ANOVA with Repeated Measures with One Within Subjects Factor).
Figure 12 – Within-subjects simple effects ANOVA for Example 1
We must also compute a sphericity correction for each of these analyses based on the covariance matrices in Figure 9. E.g. the correction factors for the Young simple effect are shown in Figure 13.
Figure 13 – Calculation of GG and HF epsilon for Young effect
Figure 14 displays the GG and HF epsilon values for each age group.
Figure 14 – GG and HF epsilon for each age effect
Applying the appropriate correction factor to the within-subjects ANOVA in Figure 12 (GG epsilon for Young and Middle and HF epsilon for Old) we get the revised output shown in Figure 15.
Figure 15 – Corrected within-subjects ANOVA
Note that to perform the simple effect tests, adjustments for sphericity are necessary, but equality of the covariance matrices is not necessary.
More detailed between-subjects analysis
For the between-subjects simple effects since we only use one Days level at a time we don’t need to worry about sphericity, and as a result we can and should use a pooled error term.
Let’s begin by looking at the simple effects of Age vs. Day 1 and Age vs. Day 5. Essentially we compute one-way ANOVA with independent variables but with a pooled error term (see Figure 16).
Figure 16 – ANOVA for Age × Day 1 and Age × Day 5
As we have seen previously, we build the two simple effects ANOVAs from the table in the top part of Figure 16. Key formulas for the Age × Day 1 analysis in that table are shown in Figure 17 (referring to cells in Figure 1). The formulas for Age × Day 5 are similar.
Figure 17 – Key formulas from Figure 16
is the pooled error term and is used for both the Age × Day 1 and Age × Day 5 analyses. Unfortunately, the value calculated for F in Figure 16 is not really distributed as F(2, 90) as indicated. The problem is that the error term is the sum of two possibly heterogeneous sources of error. To address this we need to recalculate df in the same manner as we did in Theorem 1 of Two Sample t Test with Unequal Variances.
Using the subscript 1 for the first error term (Subj w/i Age) and the subscript 2 for the second error term (Days × Subj w/i Age), we calculate the following revised value of df.
Figure 18 shows the revised version of Figure 16.
Figure 18 – Corrected ANOVA for Age × Day 1 and Age × Day 5
This tells us there is a significant difference between the three Age groups on Day 1, but no significant difference on Day 5.