McNemar’s Test

McNemar’s Test is a matched pair test used to determine whether there is a significant change in nominal data before and after an event. We begin with an example.

Example 1: In the BBC program The Doha Debates 100 people were surveyed regarding their opinion about capital punishment. 30 were in favor and 70 against. They then listened to a debate about the subject and the survey was repeated. This time 35 voted in favor and 65 against. 11 changed their mind from against to in favor and 6 changed their mind from in favor to against. Did the debate affect people’s opinion?

Let A = number of people who switched from in favor to against = 6

Let B = number of people who switched from against to in favor = 11

If the null hypothesis that the debate had no affect were true then A = B. the test statistic AB has chi-square distribution with 1 degree of freedom. To allow for Yate’s continuity correction, we simply use

image977

But the critical value of the chi-square distribution is CHIINV(.05, 1) = 3.84. Since 1.19 < 3.84 we can’t reject the null hypothesis, which indicates that we can’t say whether the debate affected the outcome.

Observation: If A + B < 25, as in Example 1, then the McNemar’s test shouldn’t be used. Instead a one-tailed binomial test of the smaller of A and B should be performed with p = .5 and n = A + B. In the case of Example 1, the probability of getting 6 or fewer successes out of 17 is given by

         BINOMDIST(6,.5,17,TRUE) = 0.166 > .05 = α

Thus we cannot reject the null hypothesis, and so conclude the debate did not significantly affect the outcome. The binomial test is equivalent to the sign test.

Example 2: In another installment of The Doha Debates, 1000 people were surveyed and 705 were in favor and 295 against. After they listened to the debate 663 voted in favor and 337 against. 73 changed their mind from against to in favor and 115 changed their mind from in favor to against. Did the debate affect people’s opinion?

In Figure 1 we rerun the tests with A = 115 and B = 73, and see that this time the debate had a significant impact on the outcome.

McNemar's test Excel

Figure 1 – McNemar’s Test

Observation: McNemar’s Test can be used with paired samples where the dependent variable is dichotomous. Where there are more than two samples (groups) Cochran’s Q test can be used. See Cochran’s Q Test for more information as well as for Real Statistics functions and a data analysis tool that can also be used to support McNemar’s Test.

6 Responses to McNemar’s Test

  1. AJB says:

    How were a and b calculated?

    • Charles says:

      A and B are not calculated. They are measured (i.e. they are the input values). Note that in Example 1 B-A = 11-6 = 5 which is the difference between the number voting in favor before and after, i.e 35-30 = 5.
      Charles

  2. Andy says:

    Could you clarify this for me please? You say that if A + B < 25, then you should use the binomial distribution to calculate the p-value. However, in Example 2, A + B is larger than 25 (i.e., 117 + 73) but you still use the binomial distribution. Should this be the chi-sq distribution? I understand that the resulting p-values will be very similar in this case, but was wondering if I missed something. Thanks.

    • Charles says:

      Andy,
      In Example 2, I showed both the McNemar test (based on chi-square) as well as the binomial test. Since A + B >= 25, you should use the McNemar test and ignore the binomial test.
      Charles

  3. Midori says:

    I want to determine if there is a relationship between “having had cholesterol measured (yes/no)” and “whether the person is a registered pharmacist (yes/no)”, is this a paired or unpaired data? would McNemar test be appropriated?
    Thank you.

    • Charles says:

      If I understand the question correctly, it seems like a 2 x 2 chi-square test of independence.
      Perhaps it would help if you explain what the data looks like.
      Charles

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>