The same approach used to calculate a confidence interval for the population mean (or the difference between population means) can be employed to create a confidence interval for a noncentrality parameter, and in turn Cohen’s effect size.
We demonstrate two techniques for finding a confidence interval for Cohen’s effect size of the one-sample t test. The approach for paired and independent t tests is similar.
Example 1: Find the 95% confidence interval for the effect size for Example 2 of One Sample t Test.
We duplicate the data from the example in Figure 1. The figure also contains the results of the one-sample t test on this data based on the null hypothesis that the population mean is 78.
Figure 1 – One sample t test
We see that Cohen’s effect size d is 0.569318 (cell L7). Since the sample mean of 68.4 (cell G7) is less than the hypothesized population mean of 78 (cell J9), we could consider Cohen’s effect size to be -0.569318, but we will continue to assume that the effect size is the positive value.
To find the 95% confidence interval for d, we first find a 95% confidence interval for the noncentrality parameter δ. One of the endpoints of this interval is the value of δ such that NT_DIST(t, df, δ, TRUE) = .025 where t = 3.66393 (the absolute value of cell J7) and df = 39 (cell K7). We now show how to find the value of δ using the Goal Seek tool.
Figure 2 – Calculating δ using Goal Seek
The formulas in Figure 2 reference the cells in Figure 1. After the OK button in the Goal Seek dialog box is pressed, the worksheet values change to those shown in Figure 3.
Figure 3 – Results from Goal Seek
The value of δ calculated is 5.779139 (cell V8). Since δ = d = d, the corresponding value of d = 0.913762 (cell V9).
In a similar fashion we calculate the other endpoint of the confidence interval by finding the value of (and subsequently of d) such that NT_DIST(t, df, δ, TRUE) = .975. Again we do this via the Goal Seek tool to get a value for of .151 with a corresponding value for d of .23875.
Thus the 95% confidence interval of d is (.23875, .91376), which is a fairly wide range for d = .5693.
Observation: The confidence interval for the effect size can also be calculated using the NT_NCP function. Figure 4 shows how this is done for Example 1.
Figure 4 – Calculating d and δ using NT_NCP
This time we see that the endpoint of the 95% confidence interval for d corresponding to .025 is .911483 (cell V18). If we plug .975 into cell V16 we get .240791 for the other endpoint, which yields a confidence interval of (.240791, .911483).
Note that not only is it easier to calculate the confidence interval using the NT_NCP function, but the results are more accurate. This can be seen from the fact that for the Goal Seek calculation NT_DIST(V5,V6,V8,TRUE) = .024235, which is not quite .025, while for the approach using NT_NCP we see that NT_DIST(V14,V15,V17,TRUE) = .025.
Observation: Once we have a confidence interval for Cohen’s d (or the noncentrality parameter), we can use this interval to find a confidence interval for power.
Example 2: Find the 95% confidence interval for the power of Example 1.
The calculations are shown in Figure 5. The upper part of the figure shows the calculation of the 95% confidence interval for d in a more concise form. The last three rows calculate statistical power based on the three values of d.
Figure 5 – Calculating confidence intervals for effect size and power
Note that the alpha in cell AA8 is based on the fact that we want a 95% confidence interval, while the alpha in cell AA12 is based on the significance level desired for the t test (and power calculation).
We see that although we calculate 94.66% power, a 95% confidence interval for power is (31.79%, 99.99%). This indicates that we need to be quite cautious about how we use the 94.66% power figure, since the actual power can be as low as 31.79% with 95% confidence.