Box’s M Test Basic Concepts

Box’s test is used to determine whether two or more covariance matrices are equal. Bartlett’s test for homogeneity of variance presented in Homogeneity of Variances is derived from Box’s test. One caution: Box’s test is sensitive to departures from normality. If the samples come from non-normal distributions, then Box’s test may simply be testing for non-normality.

Suppose that we have m independent populations and we want to test the null hypothesis that the population covariance matrices are all equal, i.e.

         H0Σ1 = Σ2 =⋯= Σm

Now suppose that S1, …, Sm are sample covariance matrices from the m populations where each Sj is based on nj independent observations each consisting of k × 1 column vector (or alternatively a 1 × k row vector).

Now define S as the pooled covariance matrix

image9166

where n = \sum\nolimits_{j=1}^m n_j define the following:

image9161image9162

Thenimage3699

where
image9164

The null hypothesis (of equal covariance matrices) is rejected when M(1 – c ) > χ2-crit (or p-value < α).

This estimate works pretty well provided nj > 20, m ≤ 5 and k ≤ 5. A better estimate can be obtained using the F distribution by defining the following:

image9165

image9158

image9159

image9160

If c2 > c2 define F = F+, while if c2 < c2 define F = F. Then F ~ F(df, df2). The null hypothesis is rejected if Fcrit.

Observation: If any of the Sj is not invertible then |Sj| = 0, and so ln|Sj| will be undefined. Thus M will be undefined and the test will fail.

Example 1: Determine whether the covariance matrices for Young, Middle and Old are equal in Example 1 of ANOVA with Repeated Measures with One Between Subjects Factor and One Within Subjects Factor.

Pooled covariance matrix Excel

Figure 1 – Covariance matrices for Example 1

The sample covariance matrices for Young, Middle and Old are calculated (see Figure 1) using the COV supplemental array function from the data in Figure 1 of ANOVA with Repeated Measures with One Between Subjects Factor and One Within Subjects Factor. Since the nj (j = 1, 2, 3) are all equal, the pooled covariance is simply the average of the Young, Middle and Old covariance matrices.

The calculations required for Box’s test are given in Figure 2.

Box's M test Excel

Figure 2 – Box’s test for Example 1

m = number of matrices = 3 (Young, Middle, Old), k = the size of each covariance matrix = 5 (each matrix is 5 × 5), n1 = n2 = n3 = number of subjects in each sample = 7 and so n = n1 + n2 + n3 = 21. In columns Q, R, S and V, nn = n1 – 1 = 6 for Young, nn = n2 – 1 = 6 for Middle, nn = n3 – 1 = 6 for Old and nn = n – m = 18 for Pooled. The other entries are as described above.

Generally we use a significant level of α = .001 for this test. From Figure 2 we see that M = 34.81 and both the chi-square test and the F test are not significant. We therefore have no reason to reject the null hypothesis that the three covariance matrices are equal.

24 Responses to Box’s M Test Basic Concepts

  1. Sirinna says:

    Hi,Charles
    I have received a warning message:
    Box’s Test of Equality of Covariance Matrices is not computed because there are fewer than two nonsingular cell covariance matrices.
    What wrong and how can i deal with it ?
    best,
    Sirinna

    • Charles says:

      Sirinna,
      This sounds like an error message from SPSS. I am not familiar with the error messages from SPSS, SAS, etc. The website focuses on Excel.
      Charles

  2. LanaLo says:

    Hello. I am very sorry but I don’t really understand a lot. I am doing Mixed-design ANOVA and my normality assumption is not met. And I have Levene’s test: p = 0,71 which means there is homogeneity of variances, right?
    Do I have to check this Box’s test? SPSS automaticlly calculated it and p = 0,013. What does that mean? Is this data important for me?

    Thank you very much!
    Sincerly

    • Charles says:

      LanaLo,
      Yes, your interpretations of Levene’s and Box’s tests are correct. If you are conducting an ANOVA you really don’t need to use Box’s test; Levene’s test should be sufficient. Also note that Box’s test is very sensitive to normality; since your data is not normal it is not surprising that you can get a a significant Box’s test even though the homogeneity of variance is actually met.
      Charles

  3. Danish says:

    Hello Charles,
    I am conducting a research on advertising effectiveness, when studying gender interaction on the dependent variables (during MANOVA), i found that my box’s m test was significant. i have checked the normality of my data and found it normal. also, i have large and unequal sample sizes, so is it allowable in my case to proceed with manova even if box’s m was significant? further, i have also read that pillai’s criterion should be used if box”s m is significant instead of wiki’s but i was was unable to find any reference. please advice me on the matter and if possible provide me with a reference.
    Regards

    • Charles says:

      Danish,

      You can proceed with MANOVA, but it is important to report the reservations you have based on Box’s M test.

      Regarding which criteria to use, see Field, A. (2009) Discovering statistics using SPSS. 3rd Ed. SAGE. Other sources are
      Olson, C.L. (1976) On choosing a test statistic in multivariate analysis of variance. Psychological Bulletin, 83, 579-586
      Olson, C.L. (1979) Practical considerations in choosing a MANOVA test statistic. Psychological Bulletin, 86, 1350-1352
      Steven, J.P. (1980) Power of the multivariate analysis of variance tests. Psychological Bulletin, 88, 728-737.

      Charles

  4. Artek says:

    Hello Charles,

    I have a design where there is one intergroup factor (2 levels) and one repeated measures factor (3 measures). I used to think, that for assumptions you must check both sphericity (Mauchley’s) and homogeneity of variances (Levene’s test). However, I was recently told that I could use M Box test for this. Which approach do you recommend?

    Also – having read above that M Box is sensitive to non-normal data, can one try M Box, and if significant, check sphericity + homogeneity instead?

    What about checking correlation of means*st.devs only – seems crude, is it used?

    • Charles says:

      hello Artek,
      Since Box’s M test is sensitive to non-normality, I tend not to use it. If you see that the data is normal, then I guess it is ok to use.
      Usually I check for homogeneity of variances. I do this by comparing the sample variances (or using Levene’s test if I have any doubt). I then rely on the GG and HF sphericity correction factors. I have never tried to use the method you describe in your last paragraph.
      Charles

  5. Barry Williams says:

    Charles
    Charles
    I have conducted Shapiro-Wilk normality tests (passed) on data then used in a 2-Way Anova. I am aware that the study is underpowered, with only 7 participants each of three training groups. Box’s Test is contravened (p<0.05) with a significance of 0.034. Does this invalidate the subsequent results/analysis? Or should I be using p<0.01 for this test. If the latter should I not be using a similar p value for other tests (e.g. Levene's)?

    Barry

    • Charles says:

      Barry,
      If you are performing 2-way Anova, I would use Levene’s test (instead of Box’s test) to check homogeneity of variances. In any case, the Box’s Test result is close to .05 and so is not too concerning.
      Charles

      • Hi, could you please help, im doing my dissertation and ive performed a 2 factor manova but my box test is showing .000 sig? what shall i do? should i no longer use this test. Im trying to test the difference between divorced/intact families and the duration of both martial status.

        • Charles says:

          Charlie,

          Make sure that the problem isn’t due to outliers. You should check for univariate outliers and multivariate outliers (using Mahalanobis distances). If so, you could rerun the test without these outliers. In this case you should still report the existence of the outliers.

          Box’s M test is quite sensitive to violations of normality. If you believe the test is giving a significant result because the data is not multivariate normally distributed, then results from MANOVA might still be valid (although it will likely be difficult to determine whether the problem is normality of homogeneity of covariance matrices).

          Don’t use MANOVA, but use separate ANOVA tests instead. Not ideal, but it might be the best you can do.

          Charles

  6. Julie Bendix says:

    Hi Charles
    What does it mean when the Box’s M test is violated? Can you then perform Discriminant analysis anyways and just be aware of the it when you interpret your results?
    Best,
    Julie

    • Charles says:

      Julie,

      Violation of this test could mean that any results obtained from the test that depends in the assumption of equal covariance matrices is invalid. It really depends on by how much the covariance matrices are different.

      A particular problem with Box’s M test is that a significant result may be due to a violation of normality and not equal covariance matrices. The specific test that you are using may be pretty robust to violations of normality but not heterogeneous covariance matrices (in fact this is usually the case).

      Charles

  7. fatima says:

    Hi Charles

    I don’t know how I can interpret Box’s test….. when it is significant, does it mean that our groups are not homogeneous? am I right?

    • Charles says:

      Fatima,
      Yes, you are correct. A significant result means that we reject the null hypothesis that the groups have equal covariance matrices.
      Charles

  8. Tim says:

    Does this work for scalar observations (i.e. k = 1)?

    Also, if I want to use a similar test to check for sameness of several regression coefficients, could I do it? Here’s what I want to do:

    I have several models: “y = alpha + beta*x”, each based on n scalar observations. I want to use the above test, plugging in (k = 1) and replacing your S_j with alpha_j or beta_j (separate tests). Would that allow me to test the null hypothesis that all the true alpha’s or beta’s are the same?

    • Charles says:

      Tim,
      When there is only one independent variable Box’s M test is known as Bartlett’s test.
      I have no reason to believe that you can use this test that the alpha or beta values are the same.
      Charles

  9. Dam says:

    this is nice but what is the relationship b/n sample proportion Box’s test!

  10. Liz says:

    Hi there, I have received an error message:
    Box’s test of equality of covariance is not computed because there are less than two nonempty cells.
    Levene’s test of equality of error variances is not computed because there are less than two nonempty cells.
    Can someone explain what this means, and what I should do please?
    Liz

    • Charles says:

      Liz,
      It just means that you have too little data for these tests to be used. If you believe that you do indeed have more than two data elements then this probably means that you haven’t entered the arguments to these functions correctly.
      Charles

  11. Todd says:

    In the “F-minus” equation, should the denominator read “(a-minus – df2)” instead of “(a-minus – M)”? The former agrees with the results in Fig.2 and the latter does not.

    • Charles says:

      Todd,
      It looks like the mistake is in Fig 2. I will fix the website and issue new versions of the software and the multivariate examples workbook.
      Thanks for catching this error.
      Charles

Leave a Reply

Your email address will not be published. Required fields are marked *