McNemar’s Test is a matched pair test used to determine whether there is a significant change in nominal data before and after an event. We begin with an example.
Example 1: In the BBC program The Doha Debates 100 people were surveyed regarding their opinion about capital punishment. 30 were in favor and 70 against. They then listened to a debate about the subject and the survey was repeated. This time 35 voted in favor and 65 against. 11 changed their mind from against to in favor and 6 changed their mind from in favor to against. Did the debate affect people’s opinion?
Let A = number of people who switched from in favor to against = 6
Let B = number of people who switched from against to in favor = 11
If the null hypothesis that the debate had no affect were true then A = B. the test statistic A – B has chi-square distribution with 1 degree of freedom. To allow for Yate’s continuity correction, we simply use
But the critical value of the chi-square distribution is CHIINV(.05, 1) = 3.84. Since 1.19 < 3.84 we can’t reject the null hypothesis, which indicates that we can’t say whether the debate affected the outcome.
Observation: If A + B < 25, as in Example 1, then the McNemar’s test shouldn’t be used. Instead a one-tailed binomial test of the smaller of A and B should be performed with p = .5 and = A + B. In the case of Example 1, the probability of getting 6 or fewer successes out of 17 is given by
BINOMDIST(6,.5,17,TRUE) = 0.166 > .05 = α
Thus we cannot reject the null hypothesis, and so conclude the debate did not significantly affect the outcome. The binomial test is equivalent to the sign test.
Example 2: In another installment of The Doha Debates, 1000 people were surveyed and 705 were in favor and 295 against. After they listened to the debate 663 voted in favor and 337 against. 73 changed their mind from against to in favor and 115 changed their mind from in favor to against. Did the debate affect people’s opinion?
In Figure 1 we rerun the tests with A = 115 and B = 73, and see that this time the debate had a significant impact on the outcome.
Figure 1 – McNemar’s Test