**ANOVA for main and interaction effects**

**Example 1**: A new drug is tested on a random sample of insomniacs: 7 young people (20-40 yrs), 7 middle aged people (40-60 yrs) and 7 older people (60+ yrs). The number of minutes each person sleeps per night is recorded for 5 successive nights in the left side of Figure 1. Determine whether the drug is effective.

**Figure 1 – Data plus additional information for Example 1**

In addition to the raw data in range B4:F24, the figure displays key counts: Age (cell J5) = # of levels in the Age factor = 3, Days (cell K5) = # of levels in Days factor = 5 and Subject (cell L5) = # of subjects in each Age group = 7.

Marginal means of all the intersections of Age × Days are given in range J9:N11, along with their sums. E.g. cell J9 provides the mean number of minutes slept by all the young subjects on day 1 via the formula =AVERAGE(B4:B10) and cell K12 provides the mean for all the subjects on day 2 via the formula =AVERAGE(K9:K11). Finally the figure also provides a chart of Age × Days.

We now can create the ANOVA analysis using the table in Figure 2.

**Figure 2 – Sources of variation for Example 1**

The implementation of Figure 2 in Excel is given in Figure 3. The *SS* terms of the first five items are derived using DEVSQ of the appropriate elements, while the other four are obtained by subtraction as described in Figure 2.

**Figure 3 – Data elements from Figure 2**

From this table we easily get the ANOVA report displayed in Figure 4.

**Figure 4 – ANOVA for Example 1**

Note that Subjects within Age is the error term for the between subjects analysis and Days x Subjects within Age is the error term for the within subjects analysis.

From Figure 4, we conclude there is a significant difference in Age group means (between-subjects analysis); i.e. the three age groups show a significant difference in the number of hours of sleep (averaged over the 5 days).

The figure also shows a significant difference in the Days means as well as the means of the interactions between Age groups and Days (within-subjects analysis). Thus there is a significant difference between the number of hours of sleep obtained by day (averaging over all three age groups) and there is a significant difference between the average hours of sleep by day and age group.

### Assumptions for between-subjects analysis

The assumptions that need to be met for the analysis described in Figure 4 to be valid fall into two categories: assumptions for the between-subjects analysis and those for the within-subjects analysis.

The assumptions for the between-subjects analysis are the familiar ones from Two Factor ANOVA with Replication, namely

- All samples are drawn from normally distributed populations
- All populations have a common variance
- All samples were drawn independently from each other
- Within each sample, the observations were sampled randomly and independently of each other

To test for normality, for each Age group we need to test the mean values across Days for each subject, i.e. range G4:G10 from Figure 1 for Young, range G11:G17 for Middle and G18:G24 for Old. We show the corresponding box plot in Figure 5.

**Figure 5 – Boxplots for within-subjects factor**

With such small samples the results are quite limited, but there is no dramatic departure from symmetry.

The key assumption for the between-subject *F* test to be valid is that the variance of the subject means within any one level of Age is the same as the variance of the subject means within any other level in Age. This can be tested by Levene’s test, although ANOVA is quite robust against reasonable violations of this assumption. Since the groups are independent we don’t need to test for sphericity. This follows since if the variances are homogeneous, then the covariance matrices will be equal and compound symmetry will hold since the independence assumption implies that all the off-diagonal covariances are zero.

Figure 6 compares the variance of the subject means for the Young, Middle and Old samples, and shows via the Levene’s test there is no significant difference between these variances, and so the homogeneity of variances assumption is satisfied.

**Figure 6 – Levene’s test for between-subjects factor**

Each item in the range BU93:BW99 is the mean across the 5 days of the corresponding elements in the raw data. E.g. cell BU93 contains the mean of the first Young subject, i.e. the value of cell G4 (see Figure 1). The cells in range BU100:BW100 contains the means of the three Age groups. The cells in the range BY93:CA99 contain the deviation of each subject mean from the corresponding Age group mean; e.g. cell BY93 contains the formula =ABS(BU93-BU100). Levene’s test is carried out by performing a one-way ANOVA on the data in the range BY93:CA99. Since the p-value (in cell CH101) = .50 > .05 = *α*, we conclude there is no significant difference between the Age group variances.

**Assumptions for within-subjects analysis**

For the within-subject tests to be valid the following assumptions need to be satisfied:

- All samples are drawn from normally distributed populations
- The Days × Subjects population covariance matrices at each Age level are approximately equal, which in Example 1 means that
*Σ*_{Young}≈ Σ_{Middle}≈ Σ_{Old} *Σ*satisfies the sphericity assumption

We need to test each of the 15 Days × Age samples for normality (or at least symmetry). From the box plot we see in Figure 7 that this assumption is reasonably well satisfied.

**Figure 7 – Box plots of Days × Age samples**

Since we don’t have access to the population covariance matrices, we test the equality of the sample covariance matrices. Box’s *M* test can be used for this combined with Levene’s test for homogeneity of variances. Figure 8 shows the result of Levene’s test for Day 1.

**Figure 8 – Levene’s test for Day 1 within-subjects factor**

We can see that that variances are 5118, 4378 and 938, but the p-value of .09 > .05 shows that the variances are equal enough for our purposes. Levene’s test on the other days show that they too satisfy the homogeneity of variance assumption.

Box’s test for equality of covariance matrices yields the following result: *M* = 34.81, (30, 1027) = .67, p-value = .91 > .05 = *α*, and so we can conclude there is no significant differences between the covariance matrices. Click here for the details about how to use Box’s Test for the Equivalence of Covariance Matrices.

As usual instead of checking for sphericity, we will simply adjust the degrees of freedom to compensate for lack of sphericity by using Greenhouse and Geisser’s epsilon:

where *k* = the number of days (*k* = 5 in our example) and S is the sample covariance matrix. Since we are assuming that *Σ _{Young} ≈ Σ_{Middle} ≈ Σ_{Old}*, it is sufficient to use the pooled sample covariance matrix (i.e. the average of the three group level covariance matrices). In our example, the three sample covariance matrices and the pooled matrix are shown in Figure 9.

**Figure 9 – Covariance matrices for Example 1**

The value of is calculated from the pooled covariance matrix exactly as was done in Figure 3 of Sphericity. See Figure 10.

**Figure 10 – Calculation of GG epsilon and HF epsilon for Example 1**

The HF epsilon is calculated by the formula used in Sphericity, namely

Here *k* = # of days = 5 and *n* = # subjects ∙ # Age groups = 7 ∙ 3 = 21. Thus, GG epsilon = .80 and HF epsilon = .998. Since GG epsilon > .75, we should use the HF epsilon correction factor. Since this is almost 1, it is safe to assume sphericity. In any case, the revised version of the omnibus ANOVA test from Figure 4 for the within-subjects factor is shown in Figure 11.

**Figure 11 – Within-subjects ANOVA corrected for sphericity**

**More detailed within-subjects analysis**

Although the GG epsilon and HF epsilon is very useful in addressing sphericity issues, it is still better not to use *MS _{E}* (= 2241.748) as the error term in detailed analyses. In fact it is best to analyze simple effects on an individual Age group basis. This can be done by running a one-way repeated measures ANOVA for each Age group (or by skipping ANOVA and going directly to contrasts).

Figure 12 shows the analyses for each of the age groups (as described in ANOVA with Repeated Measures with One Within Subjects Factor).

**Figure 12 – Within-subjects simple effects ANOVA for Example 1**

We must also compute a sphericity correction for each of these analyses based on the covariance matrices in Figure 9. E.g. the correction factors for the Young simple effect are shown in Figure 13.

**Figure 13 – Calculation of GG and HF epsilon for Young effect**

Figure 14 displays the GG and HF epsilon values for each age group.

**Figure 14 – Epsilon for each age effect**

Applying the appropriate correction factor to the within-subjects ANOVA in Figure 12 (GG epsilon for Young and Middle and HF epsilon for Old) we get the revised output shown in Figure 15.

**Figure 15 – Corrected within-subjects ANOVA**

Note that to perform the simple effect tests, adjustments for sphericity are necessary, but equality of the covariance matrices is not necessary.

**More detailed between-subjects analysis**

For the between-subjects simple effects since we only use one Days level at a time we don’t need to worry about sphericity, and as a result we can and should use a pooled error term.

Let’s begin by looking at the simple effects of Age vs. Day 1 and Age vs. Day 5. Essentially we compute one-way ANOVA with independent variables but with a pooled error term (see Figure 16).

**Figure 16 – ANOVA for Age × Day 1 and Age × Day 5**

As we have seen previously, we build the two simple effects ANOVAs from the table in the top part of Figure 16. Key formulas for the Age × Day 1 analysis in that table are shown in Figure 17 (referring to cells in Figure 1). The formulas for Age × Day 5 are similar.

**Figure 17 – Key formulas from Figure 16**

Now

is the pooled error term and is used for both the Age × Day 1 and Age × Day 5 analyses. Unfortunately, the value calculated for *F* in Figure 16 is not really distributed as *F*(2, 90) as indicated. The problem is that the error term is the sum of two possibly heterogeneous sources of error. To address this we need to recalculate *df* in the same manner as we did in Theorem 1 of Two Sample t Test with Unequal Variances.

Using the subscript 1 for the first error term (Subj w/i Age) and the subscript 2 for the second error term (Days × Subj w/i Age), we calculate the following revised value of *df*.

Figure 18 shows the revised version of Figure 16.

**Figure 18 – Corrected ANOVA for Age × Day 1 and Age × Day 5**

This tells us there is a significant difference between the three Age groups on Day 1, but no significant difference on Day 5.

**Real Statistics Capabilities**

The GGEpsilon and HFEpsilon functions described in Sphericity can also be used in the case of Two Factor ANOVA with one fixed factor and one repeated measures factor. For example, referring to Figure 1, for Example 1, we have GGEpsilon(B4:F24,3) = .80 and HFEpsilon(B4:F24,3,7) = .998.

**Real Statistics Data Analysis Tool**: The Real Statistics Resource Pack provides the **Two Factor Repeated Measures** data analysis tool.

For Example 1, enter **Ctrl-m** and double click on **Analysis of Variance** (as shown Figure 0 of Anova Confidence Interval). Next select **Repeated Measures: mixed **from the dialog box that appears.** **Now fill in the fields in the dialog box that appears as shown in Figure 19.

**Figure 19 – Two Factor Repeated Measures ANOVA dialog box**

The output is shown in Figure 20.

**Figure 20 – Two Factor Repeated Measures ANOVA**

Hi Charles,

This website is a great resource.

I am trying to work through the ANOVA analysis table in Figure 2.

I can only calculate the same value for S6 with [bm * DEVSQ(age group means)] and S7 with [am * DEVSQ(days means)]. The a and b values are reversed from your equations in Figure 3. Is this a typo?

Thanks!

Cynthia,

You are correct. This is a typo. I have just corrected the formulas on the referenced webpage.

Thanks very much for catching this mistake.

Charles

Hi, Charles,

Thanks for your website! You are making my MS thesis data analysis 100x easier. I will have to learn R or SAS or etc. eventually, but for now this is a life-saver.

Question for you: How were you able to generate the box plots (i.e., Figure 5 and Figure 7)? I downloaded the Example Workbooks and have no problem seeing which tables/data were used to generate the plots, however, it is not clear which function in Excel you use to create them. For example, if you are in Excel and click on the Insert Tab -> Charts, which chart do you use? Or, is there another way to generate the plot that I am overlooking?

I am not sure if my question is clear. I’ll be happy to try and clarify if you need.

Cheers,

Phyllis

Phyllis,

I am pleased that you have benefited from the website.

To learn how to create box plots in Excel, please refer to the following webpages:

Box Plots

Special charting capabilities

Charles

Thank you, I will check that out.

Another question if you have the time: I’m running through the step-by-step and I cannot figure out which ANOVA test you ran to generate the ANOVA table in Figure 4.

I see that the calculations in Figure 2 were run for this analysis, but it is not clear to me which ANOVA test you ran on Real Statistics, and why. Will you please clarify?

Thanks,

Phyllis

Disregard, I figured it out! Thanks again,

Phyllis

hi Charles

not sure why my query is being deleted. I need to know how to do within and between subject anova if my samples are not the same size (unbalanced model).

control grp (n = 33) vs intervention grp (n = 24) which both groups underwent pretest and posttest.

If I have a few DVs, is there a MANOVA that handles such design too?

Not to worry. Your query has not been deleted. I have not yet had the time to post your comment and provide a response.

Charles

hi, can you explain how to do similar mixed repeated anova for unbalanced model? i’ve a control group (N = 33) and intervention group (N = 24) in a pre-post-test setting. tried to arrange the data according to what I did for two-way anova (regression) but i got errors for mixed repeated anova. will the answers be the same if i do unpaired t and paired t instead then?

I understand the following to be the problem that you are addressing. Please let me know if I have understood this correctly.

You have two independent groups: a control group and an intervention group. For each group you have data for two (dependent) events: pre- and post-test. You have 33 pre-test data elements for the control group, 33 post-test data elements for the control group, 24 pre-test data elements for the intervention group and 24 post-test data elements for the intervention group.

If this is correct, then one approach is to create a balanced model, by randomly removing 9 pairs of data from the control group. The disadvantage of this approach is that the sample size, and therefore the power of the test, will be reduced, but in this case you can use the capabilities described on the website.

I am about to start updating and upgrading the regression capabilities described on the website and implemented in the Real Statistics Resource Pack. I will look into the issue of unbalanced mixed repeated measures Anova.

You should be able to perform a “post-hoc” t test between the control and intervention groups for pre-test (or post-test) using the t test with independent samples (the sample sizes don’t need to be equal). You should also be able to perform a “post-hoc” paired t test for the control group (or the intervention group).

Charles

hi charles

yes, you’re right about my current research resign.

can i know which of your functions should i choose to have the randomly elimination of 9 extras so that i can have a balanced model comparison?

i recalled having to see some functions which can actually transpose or reorganise data so that the control group of n=33 can be all used to reduce to a revised control group n=24 without eliminating a random 9. do you think you can help with this?

for the post-hoc tests, am i right to say that i can only do so if i uses a ANOVA test with more than 2 set of dependent data?

can you enlighten what will i lose by doing a unpaired t (pre vs post of control then of intervention) and a paired t (control vs intervention at pre then at post)?

belle

Belle,

Q. can i know which of your functions should i choose to have the randomly elimination of 9 extras so that i can have a balanced model comparison?

A. Suppose your data is in range A1:A33. Enter the formula =RAND() in cell B1, highlight the range B1:B33 and press Ctrl-D. This will put random numbers in cells B1 through B33. Now put the formula =SMALL(B1:B33,9) in cell B35. This cell will contain the 9th smallest of the random numbers. Now place the formula =IF(B1< =$B$35,"",A1) in cell C1, highlight the range C1:C33 and press Ctrl-D. Range C1:C3 will now contain the original data except that the values in 9 random cells will be replaced by blanks. Q. i recalled having to see some functions which can actually transpose or reorganise data so that the control group of n=33 can be all used to reduce to a revised control group n=24 without eliminating a random 9. do you think you can help with this? A. There are various types of Imputation approaches to add 9 elements to the control group, although there are shortcomings with each of these approaches. See Handling Missing Data.

Q. for the post-hoc tests, am i right to say that i can only do so if i uses a ANOVA test with more than 2 set of dependent data?

A. If the ANOVA test only has two levels for some factor, there is no point in performing a post-hoc test on that factor (since th ANOVA test has already done it).

Q. can you enlighten what will i lose by doing a unpaired t (pre vs post of control then of intervention) and a paired t (control vs intervention at pre then at post)?

A. Sorry, but I don’t understand this question.

Charles

hi Charles

can I know if the unbalance model for two factor repeated anova being created? do you have an updated excel software since this qs was posted to you at jul 2015?

Sorry, but I have been busy with other things. Fortunately I am finishing these now, and will return to completing the next release of the software. I hope to have enough time to include the feature you are looking for in the next release. If there isn’t enough time, it should be in the release after that.

Charles

hi Charles, can I know if it is available now for unbalanced model?

It is not yet available, but it is one of the next things I will be working on.

Charles