McNemar test — Worked example

Example setup

Suppose we test 100 items with two classifiers A and B and record paired outcomes (correct/incorrect).

McNemar uses only discordant pairs:

• b = # items where A correct, B incorrect
• c = # items where A incorrect, B correct

Example: b=12, c=5

Step-by-step

1. Compute b+c = 17 (discordant total).
2. Continuity-corrected chi-square: (|b−c|−1)²/(b+c) = (|12−5|−1)²/17 = 6²/17 ≈ 2.1176.
3. p-value = P(Chi²₁ ≥ 2.1176).
4. Exact p-value: binomial test on min(b,c)=5 with n=17 and p=0.5 (two-sided).
5. Decision: compare p to alpha (e.g. 0.05).

Try it in the tool

Open with b=12, c=5

FAQ

What data does McNemar require?

Paired nominal outcomes (same items evaluated by two methods). It’s not for independent samples.

Why are only b and c used?

Concordant pairs (both correct or both incorrect) don’t inform which method is better in a paired comparison.

Should I use exact or chi-square p-value?

Exact is safer for small b+c. For large discordant totals, the chi-square approximation is usually fine.