As described in Null Hypothesis Testing, beta (β) is the acceptable level of type II error, i.e. the probability that the null hypothesis is not rejected even though it is false and power is 1 – β. We now show how to estimate the power of test.
Example 1: Suppose bolts are being manufactured using a process so that it is known that the length of the bolts follows a normal distribution with standard deviation 12 mm. The manufacturer wants to check that the mean length of their bolts is 60 mm, and so takes a sample of 110 bolts and uses a one tail test with α = .05 (i.e. H0: µ ≤ 60). What is the probability of a type II error if the actual mean length is 62.5?
Since n = 110 and σ = 12, the standard error = = 1.144. Let x = the length of the bolt. The null hypothesis is rejected provided the sample mean is greater than the critical value of x, which is NORMINV(1 – α, μ, s.e.) = NORMINV(.95, 60, 1.144) = 61.88.
Now suppose that the actual mean is 62.5. The situation is illustrated in Figure 8.8, where the curve on the left represents the normal curve being tested with mean μ0 = 60 and the normal curve on the right represents the real distribution with mean μ1 = 62.5.
We have β = NORMDIST(61.88,62.5,1.144,TRUE) = .295, and so power = 1 – β = .705.
We can repeat this calculation for values of μ1 ≥ 62.5 to obtain the table and graph of the power values in Figure 2.
Example 2: For the data in Example 1, answer the following questions:
- What is the power of the test for detecting a standardized effect of size .2?
- What effect size (and mean) can be detected with power .80?
- What sample size is required to detect an effect of size .2 with power .80?
a) As described in Standardized Effect Size, we use the following measure of effect size:
Thus μ1 = 60 + (.2)(12) = 62.4. As in Example 1,
and so β = NORMDIST(61.88, 62.4, 1.1144, TRUE) = .325, and so power = 1 – β = .675.
We summarize these calculations in the following worksheet:
b) We use Excel’s Goal Seek capability to answer the second question. Using the worksheet in Figure 3, we now select Data > Data Tools | What-If Analysis. In the dialog box that appears (see Figure 4) enter the following values
We are requesting that Excel find the value of cell B9 (the effect size) that produces a value of .8 for cell B12 (the power). Here the first entry must point to a cell which contains a formula. The second entry must be a value and the third entry must point to a cell which contains a value (possibly blank) and not a formula. After clicking on OK, a Goal Seek Status dialog box appears and the worksheet from Figure 3 changes to that in Figure 5.
Note that the values of a number of cells have changed to reflect the value necessary to obtain power of .80. In particular, we see that the Effect size (cell B9) contains the value 0.23691. You must click on OK in the Goal Seek Status box to lock in these new values (or Cancel to return to the original worksheet values).
c) We again use Excel’s Goal Seek capability to answer the third question. Using the worksheet in Figure 3 (making sure that the effect size in cell B9 is set to .2), we now enter the following values in the dialog box that appears (see Figure 6):
After clicking on OK, the worksheet changes to that in Figure 7.
In particular, note that the sample size value in cell B6 changes to 154.486. Thus the required sample size is 155.
Observation: An alternative way of answering Example 2 (a) is as described in Figure 8.
Observation: An alternative way of answering Example 2 (c) is as described in Figure 9. Note that this approach avoids the need for the Goal Seek capability.