We review three different measures of effect size: Phi φ, Cramer’s V and the Odds Ratio.
For the goodness of fit in 2 × 2 contingency tables, phi, which is equivalent to the correlation coefficient r (see Correlation), is a measure of effect size. Phi is defined by
where n = the number of observations. A value of .1 is considered a small effect, .3 a medium effect and .5 a large effect.
This is the effect size measure (labelled as w) that is used in power calculations even for contingency tables that are not 2 × 2 (see Power of Chi-square Tests).
Cramer’s V is an extension of the above approach, and is calculated as
where df* = min(r – 1, c – 1) and r = number of rows and c = number of columns in the contingency table. Per Cohen, you use the guidelines for phi divided by the square root of df*. Thus, the guidelines are:
Figure 1 – Effect sizes for Cramer’s V
For a 2 × 2 contingency table, we can also define the odds ratio measure of effect size as in the following example.
Example 1: Calculate the odds ratio for the data in Example 2 of Independence Testing.
Figure 2 – Odds ratio effect size
As we saw in Example 2 of Independence Testing, there is a significant difference between those taking therapy 1 and those taking therapy 2. In fact, 26.19% of the people who took therapy 1 were not cured, while 47.22% of those who took therapy 2 were not cured. This shows that those taking therapy 2 were 1.80 times as likely as those taking therapy 1 to remain uncured. This is a meaningful measure of effect size, called the risk ratio or relative risk.
A related measure of effect size is the odds ratio. The odds of a person who took therapy 1 remaining uncured is 11 to 31 or .3548. The odds of a person who took therapy 2 is 51 to 57 or .8947. This means that the odds of remaining uncured is 2.52 times greater for therapy 2 than therapy 1. The ratio 2.52 is the odds ratio.
Fisher Exact Test
When Fisher’s exact test is employed, the odds ratio can be used as described above. Since an approximate chi-square statistic can be calculated from the p-value using the CHISQ.INV function, phi and Cramer’s V can also be calculated as described above. Alternatively, you can use the following Real Statistics array function.
Real Statistics Function: The following array function is provided in the Real Statistics Resource Pack:
FISHER_TEST(R1, lab): returns a column array containing the following values: p-value for the two-tailed Fisher’s exact test, sample size, df, chi-square statistic, Cramer’s V and phi (labelled w). If lab = TRUE (default FALSE), then an extra column of labels is added.
This function can be used for range R1 where the FISHERTEST can be used. E.g. the output from the array formula =FISHER_TEST(B4:C6,TRUE) is shown in Figure 3.
Figure 3 – Effect sizes for Fisher’s exact test