McNemar’s Test

McNemar’s Test is a matched pair test used to determine whether there is a significant change in nominal data before and after an event. We begin with an example.

Example 1: In the BBC program The Doha Debates 100 people were surveyed regarding their opinion about capital punishment. 30 were in favor and 70 against. They then listened to a debate about the subject and the survey was repeated. This time 35 voted in favor and 65 against. 11 changed their mind from against to in favor and 6 changed their mind from in favor to against. Did the debate affect people’s opinion?

Let A = number of people who switched from in favor to against = 6

Let B = number of people who switched from against to in favor = 11

If the null hypothesis that the debate had no affect were true then A = B. the test statistic AB has chi-square distribution with 1 degree of freedom. To allow for Yate’s continuity correction, we simply use


But the critical value of the chi-square distribution is CHIINV(.05, 1) = 3.84. Since 1.19 < 3.84 we can’t reject the null hypothesis, which indicates that we can’t say whether the debate affected the outcome.

Observation: If A + B < 25, as in Example 1, then the McNemar’s test shouldn’t be used. Instead a one-tailed binomial test of the smaller of A and B should be performed with p = .5 and n = A + B. In the case of Example 1, the probability of getting 6 or fewer successes out of 17 is given by

         BINOMDIST(6,.5,17,TRUE) = 0.166 > .05 = α

Thus we cannot reject the null hypothesis, and so conclude the debate did not significantly affect the outcome. The binomial test is equivalent to the sign test.

Example 2: In another installment of The Doha Debates, 1000 people were surveyed and 705 were in favor and 295 against. After they listened to the debate 663 voted in favor and 337 against. 73 changed their mind from against to in favor and 115 changed their mind from in favor to against. Did the debate affect people’s opinion?

In Figure 1 we rerun the tests with A = 115 and B = 73, and see that this time the debate had a significant impact on the outcome.

McNemar's test Excel

Figure 1 – McNemar’s Test

Observation: McNemar’s Test can be used with paired samples where the dependent variable is dichotomous. Where there are more than two samples (groups) Cochran’s Q test can be used. See Cochran’s Q Test for more information as well as for Real Statistics functions and a data analysis tool that can also be used to support McNemar’s Test.

17 Responses to McNemar’s Test

  1. Dane says:

    This is very close to what I need; however, my null-hypothesis is that the results are different. In your examples this would correspond to opinions changing; therefore, rejecting the null hypothesis would amount to statistically concluding that the opinions didn’t change (contrary to expectation).

    What test should be used for this opposite null hypothesis with binary ‘yes’ or ‘no’ type data?

    • Charles says:

      You should be able to use the usual test, but a significant result result would support the alternative hypothesis. You should also look at the power of the test.

  2. Diana says:

    Dear Charles,

    I also am struggling to decide what the best statistical test would be for my data. I have a set of students who listened to my lecture. I did a pre-test and post-test multiple and true-false questions. I asked the same questions before and after the lecture. I recorded Correct answers with 0 and Incorrect answers with 1. I want to see whether my lecture had an impact on their understanding. Each student answered the pre- and the post-test questions.

  3. Dr Pity please excuse me that Excel is likely taking to the contrary, please excuse me error
    CHISQ_INV 3.841458821 = (0.95, 1)

  4. Dr Hello, I wish you a Merry Christmas and a lot of prosperity for the coming times. Dr’m very grateful to their shared knowledge and your page is like carrying the R software Excel, and put it in simple and understandable terms. Dr in Example 1 McNemar think is a mistake because CHIINV (05, 1) = 0.0039 making significant changes of opinion arise. Please if I am wrong correct me.
    Thank you.

  5. TNHM says:

    Dear Charles,

    I am struggling to decide what the best statistical test would be for my data. I have a set of patients who have all had two imaging tests – computed tomography (CT) and MRI. The data from CT is ordinal as I have completed a radiological score (1 to 4). The data from MRI is categorical – present vs. not present. I would like to test the null hypothesis there is no relationship between CT scores and MRI presence/non-presence.

    I have concluded that data must be paired as it is from the same patient and both CT and MRI were completed at the same time. I think a McNemar’s should be suitable but with four possible CT scores would Cochran’s Q test be better?

    I am really grateful for your help.

    • Charles says:

      Cochran’s Q test requires that all the data be dichotomous. In your example this is not the case since you use scores of 1, 2, 3, 4.

      It seems to me that the appropriate way to look at the problem as a two sample t test. Sample 1 consists of all the CT values where the corresponding MRI value is present and sample 2 consists of all the CT values where the corresponding MRI value is not present. Since it is likely that the normality assumption for the t test is not met, you probably need to use the Mann-Whitney test instead.


  6. LiuWen says:

    Hi Charles,

    I’m now start learning non-parametric test from your website:)
    One thing confused me is “the test statistic A – B has chi-square distribution with 1 degree of freedom.” , why is “one degree of freedom”?
    Do appreciated if you give some hints:)


    • Charles says:

      Although I don’t have a precise answer for you, I have two hints:
      1. It is probably related to the fact that if x has a standard normal distribution then x^2 has a chi-square distribution with 1 degree of freedom and for large enough samples the binomial distribution can be approximated by the normal distribution.
      2. McNemar’s test is a simple form of Cochran’s test

  7. Midori says:

    I want to determine if there is a relationship between “having had cholesterol measured (yes/no)” and “whether the person is a registered pharmacist (yes/no)”, is this a paired or unpaired data? would McNemar test be appropriated?
    Thank you.

    • Charles says:

      If I understand the question correctly, it seems like a 2 x 2 chi-square test of independence.
      Perhaps it would help if you explain what the data looks like.

  8. Andy says:

    Could you clarify this for me please? You say that if A + B < 25, then you should use the binomial distribution to calculate the p-value. However, in Example 2, A + B is larger than 25 (i.e., 117 + 73) but you still use the binomial distribution. Should this be the chi-sq distribution? I understand that the resulting p-values will be very similar in this case, but was wondering if I missed something. Thanks.

    • Charles says:

      In Example 2, I showed both the McNemar test (based on chi-square) as well as the binomial test. Since A + B >= 25, you should use the McNemar test and ignore the binomial test.

  9. AJB says:

    How were a and b calculated?

    • Charles says:

      A and B are not calculated. They are measured (i.e. they are the input values). Note that in Example 1 B-A = 11-6 = 5 which is the difference between the number voting in favor before and after, i.e 35-30 = 5.

Leave a Reply

Your email address will not be published. Required fields are marked *