Contrasts for Two Factor ANOVA

We now show how to use the Real Statistics Two Factor ANOVA Follow-up data analysis tool to create contrasts for two factor ANOVA. There are two main advantages of this approach over one factor analyses:

  • The two factor ANOVA model accounts for more of the variance, and so the error term MSE is smaller
  • Information about the interaction between factors is available

Caution: You should only calculate the contrasts that you are interested in. Testing too many contrasts raises the familywise error rate to an unacceptable level. You can control the familywise error rate, but this risks a serious loss of power.

Example 1: A research team has investigated whether mosquitos bite specific parts of the body (arm, neck, foot) more often than others and whether there are differences depending on the climate of the room (cold, dry, humid). Five people were studied for each combination, with the number of bites recorded in the table shown in Figure 1.

Two-way ANOVA example

Figure 1 – Data for Example 1

We conduct a two-way ANOVA using the Real Statistics data analysis tool with the output shown in Figure 2.

ANOVA2 data analysis

Figure 2 – ANOVA output for Example 1

From the ANOVA report we see there are significant differences between the climate and interaction means, but not between the body parts.

Using the data in Figure 2, determine whether there are significant differences between the number of bites (a) in cold and humid climates, (b) to the arm and feet, (c) to the arm and foot in humid climates and (d) to the foot in cold and humid climates.

Cold vs. Humid: To determine whether there is a significant difference between the number of bites in cold and humid climates, press Ctrl-m, double click on the Analysis of Variance option and select the Two Factor ANOVA Follow-up option (as shown in Figure 1 of Real Statistics Support for Two Factor ANOVA). After pressing the OK button, another dialog box appears. Fill in this dialog box as shown in Figure 3 and press the OK button.

Contrasts ANOVA2 dialog box

Figure 3 – Two Factor ANOVA Follow-up dialog box

The output is shown in Figure 4 (except that initially the contrast coefficients in the shaded range are all blank).

Contrasts main effects

Figure 4 – Comparison of cold vs. humid

Since we are interested in Cold vs. Humid and the mean for Humid is larger than that for Cold, we place a 1 in cell AC6 and a -1 in cell AC4.

Here the standard error (cell AB10) is based on the value of MSW = 56.3778 (cell O10 of Figure 2). Since p-value = .3.83E-05 < .05 = α we conclude there is a significant difference between the number of bites in cold and humid climates, thereby addressing Example 1(b).

Arm vs. Feet: As we saw in Figure 2 there is no significant difference between the number of bites by parts of the body. Thus, it is not necessary to perform a contrast for arm vs. feet.

Arm vs. Feet in Humid climates: We again use the Two Factor ANOVA Follow up data analysis tool, inserting G13:I15 in the Input Range and selecting the Contrasts – no correction option (see Figure 3). For this contrast, we need to select the Interaction option.

We place the -1 in the Arm × Humid cell and the 1 in the Foot × Humid cell. The output is shown in Figure 5.

Contrasts interaction effect

Figure 5 – Comparison of arms vs. feet in humid climates

Since p-value = .06 > .05 = α, we conclude there isn’t a significant difference between the number of bites on the arm and feet in humid climates, thereby addressing Example 1(c).

Observation: As you can see from Figure 5, interaction contrasts are based on comparing the interaction means. There are two types of interaction comparisons: confounded and unconfounded. E.g. we can compare the means of Cold-Arm (cell AC60) with Dry-Foot (cell AE61), but even if there is a significant difference, we can’t tell whether this is due to a difference in the climate or a difference in the part of the body which the mosquitos bit. This is an example of a confounded comparison. Confounded comparisons differ in both factors. We won’t try to analyze these.

Feet in Cold vs. Humid climates We again use the Interaction option. As we can see in Figure 6, this time there is a significant difference.

ANOVA 2 interaction contrast

Figure 6 – Comparison of feet in cold vs. humid climates

Observation: Note that we needed to perform three contrasts in Example 1. In order to reduce the familywise error, we can Bonferroni’s correction factor and employ an alpha value of .05/3 = .01667 instead of .05. The conclusions arrived in this case don’t change.

Observation: If we use the Contrasts – Bonferroni correction option (see Figure 3) in the Two Factor ANOVA Follow-up data analysis tool, then the value of alpha is modified assuming the maximum number of orthogonal contrasts, which for the Rows option is equal to the number of row factor levels minus one. For an analysis with three row factor levels, an alpha of .05 becomes alpha = .05/2.  Similarly when the Column option is used, alpha is modified to alpha divided by the number of column factor levels minus one. The situation is similar when the Contrasts – Dunn-Sidak correction option is used.

If the Contrasts – Bonferroni correction and Interaction options are selected then the value of alpha is divided by the product of the number of row and column factor levels minus one. For an analysis with 3 rows and 2 columns, then converts an alpha value of .05 into a value of .01.

Example 2: Using the data in Example 1, determine whether there are significant differences between the number of bites in the arm or neck in cold and humid climates.

All the tests used in Example 1 are pairwise contrasts. This time, we need to use a more complicated contrast. We again choose the Interaction option, but because we will use 4 contrast coefficients, as shown on the left side of Figure 7, we need to specify the # Contrasts field to be 4 instead of the default value of 2 (see Figure 3).

Multiple contrasts ANOVA 2

Figure 7 – More complicated comparison

We see from the right side of Figure 7 that there is no significant difference.

9 Responses to Contrasts for Two Factor ANOVA

  1. Rebecca says:

    Dear Doctor Zaiontz,

    I would like to conduct a planned test, but a question: could the analysis be done when token numbers are not the same? I mean, for example, 24 tokens for males and 28 for females. Many thanks!


    • Charles says:

      The samples don’t need to be the same size to use contrasts.
      For two factor Anova with unequal sample sizes start by performing the two factor ANOVA using the Regression option. Then use this output for the contrasts as described on the referenced webpage.

      • Rebecca says:

        Hi Doctor Zaiontz,

        After a non significant two way ANOVA (gender x generation), I ran the planned cmparison and Tukey tests to compare the group means of 10 university male and 10 university females , but the result was not significant. I tried Mann-Whitney test and then finally got the result I expected. But can I report the result of non parametric test after a non significant two-way anova? What would you suggest? Many thanks!

        • Charles says:

          Generally, if you got non-significant result for the two-way ANOVA, you don’t need to (and should not) do any other tests.
          That you found a significant result from Mann-Whitney is not too surprising since after so many tests, the real alpha is much higher than the original .05 (experiment-wise error).

  2. Felix says:

    Hello Charles:
    You do easy to understand the hard things. Thank you so much!!!
    Best regards

  3. Colin says:

    Where is the Interaction supplemental data analysis tool ? And how do you make the Main effect and simple effect test? It seems you didn’t use the supplemental data analysis tool but entered formulas in the cells.

    • Charles says:

      Hello Colin,

      The Interaction data analysis tool was removed a few releases ago when I added the Two Factor Anova data analysis tool. The Two Factor Anova data analysis tool generates the interaction table (as part of the Descriptive Statistics produced by that tool). I have now updated the website to reflect this change. Thanks for identifying another oversight on my part.

      The main and simple tests (as well as contrasts) can be performed using the formulas indicted, but they can also be conducted using the Two Factor Anova and One Factor Anova data analysis tools provided by the Real Statistics Resource Pack.


      • Colin says:


        When I used the two factor ANOVA data analysis tool Excel only gave me a “COUNT table”, a “MEAN table”, a “VARIANCE table” and a “Two Factor Anova table”. I am using the 2.8 version.

Leave a Reply

Your email address will not be published. Required fields are marked *