# Statistical Power and Sample Size

As described in Null Hypothesis Testing, beta (β) is the acceptable level of type II error, i.e. the probability that the null hypothesis is not rejected even though it is false and power is 1 – β. We now show how to estimate the power of test.

Example 1: Suppose bolts are being manufactured using a process so that it is known that the length of the bolts follows a normal distribution with standard deviation 12 mm. The manufacturer wants to check that the mean length of their bolts is 60 mm, and so takes a sample of 110 bolts and uses a one tail test with α = .05 (i.e. H0: µ ≤ 60). What is the probability of a type II error if the actual mean length is 62.5?

Since n = 110 and σ = 12, the standard error = $\frac{\sigma}{\sqrt{n}} = \frac{12}{\sqrt{110}}$ = 1.144. Let x = the length of the bolt. The null hypothesis is rejected provided the sample mean is greater than the critical value of x, which is NORMINV(1 – α, μ, s.e.) = NORMINV(.95, 60, 1.144) = 61.88.

Now suppose that the actual mean is 62.5. The situation is illustrated in Figure 8.8, where the curve on the left represents the normal curve being tested with mean μ0 = 60 and the normal curve on the right represents the real distribution with mean μ1 = 62.5.

Figure 1 – Statistical power

Since

We have β = NORMDIST(61.88,62.5,1.144,TRUE) = .295, and so power = 1 – β = .705.

We can repeat this calculation for values of μ1 ≥ 62.5 to obtain the table and graph of the power values in Figure 2.

Figure 2 – Power curve for Example 1

Example 2: For the data in Example 1, answer the following questions:

1. What is the power of the test for detecting a standardized effect of size .2?
2. What effect size (and mean) can be detected with power .80?
3. What sample size is required to detect an effect of size .2 with power .80?

a) As described in Standardized Effect Size, we use the following measure of effect size:

Thus μ= 60 + (.2)(12) = 62.4. As in Example 1,

and so β = NORMDIST(61.88, 62.4, 1.1144, TRUE) = .325, and so power = 1 – β = .675.

We summarize these calculations in the following worksheet:

Figure 3 – Determining power based on effect and sample size

b) We use Excel’s Goal Seek capability to answer the second question. Using the worksheet in Figure 3, we now select Data > Data Tools | What-If Analysis. In the dialog box that appears (see Figure 4) enter the following values

Figure 4 – Goal Seek dialog box

We are requesting that Excel find the value of cell B9 (the effect size) that produces a value of .8 for cell B12 (the power). Here the first entry must point to a cell which contains a formula. The second entry must be a value and the third entry must point to a cell which contains a value (possibly blank) and not a formula. After clicking on OK, a Goal Seek Status dialog box appears and the worksheet from Figure 3 changes to that in Figure 5.

Figure 5 – Determining detectable effect size for specified power

Note that the values of a number of cells have changed to reflect the value necessary to obtain power of .80. In particular, we see that the Effect size (cell B9) contains the value 0.23691. You must click on OK in the Goal Seek Status box to lock in these new values (or Cancel to return to the original worksheet values).

c) We again use Excel’s Goal Seek capability to answer the third question. Using the worksheet in Figure 3 (making sure that the effect size in cell B9 is set to .2), we now enter the following values in the dialog box that appears (see Figure 6):

Figure 6 – Using Goal Seek to determine sample size requirements

After clicking on OK, the worksheet changes to that in Figure 7.

Figure 7 – Sample size requirement for Example 2

In particular, note that the sample size value in cell B6 changes to 154.486. Thus the required sample size is 155.

Observation: An alternative way of answering Example 2 (a) is as described in Figure 8.

Figure 8 – Determining power for a given effect size

Observation: An alternative way of answering Example 2 (c) is as described in Figure 9. Note that this approach avoids the need for the Goal Seek capability.

Figure 9 – Determining sample size for a given effect size

### 18 Responses to Statistical Power and Sample Size

1. Vendula says:

Hello Charles,
why is in the first formula, when you calculate lenght for alpha=0,05 used SEM and not SD? According normal distribution, the 95% of data are within mean +- 2 SD, so it should be =norm.inv(0.95,60,12).

• Charles says:

I am not exactly sure which is the first formula that you are referring to, but if it is the effect size formula, then Cohen’s d uses the standard deviation and not the standard error. d does not depend on the sample size.
Charles

• Vendula says:

I was speaking about example 1, when you calculate alpha a and beta, you used SE =1.44 not SD =12

• Charles says:

Vendula,
Yes, for this problem, the appropriate value for the standard deviation for a sample of size 110 is 1.44, which is the standard error for the sample. 12 represents the standard deviation of the population.
Charles

2. Jonathan Bechtel says:

Hi Charles,

I’m interested in how you’d compute beta for observed values that aren’t greater than Xcrit.

For example, if the observed value was 60.5 (less than Xcrit) would the beta be equal to NORMDIST(61.88, 60.5, 1.144, TRUE) = 0.886148, and the beta would be higher the smaller the number gets.

Also, if you were doing a right tail test and the observed value was less than Xcrit, such as NORMDIST(58.12, 58, 1.144, TRUE) = 0.5412.

Thanks

Hello,

If I have a sample with a mean of 1000 and SEM (standar error) of 60 and other sample with a mean of 800 and SEM – 70, how would I calculate the statistical power between these two samples?

Thank you

If you could explain how to to solve it using both excel and spss it would be perfect!! thank you

• Charles says:

I don’t use SPSS and so won’t comment about SPSS. See response to your other comment regarding Excel.
Charles

• Charles says:

You also need to know the sample size. See the following webpage for details:
Power of t test
Charles

4. Angela says:

Hello Charles,

I need assistance with how to plug in the numbers for the Statistical Power and Sample Size option. I will be running a logistic regression. I have all the data, but am unsure as to what I input.

Any insight you have would be great! Thank you.

• Charles says:

Angela, sorry but the Statistical Power and Sample Size data analysis tool supports linear regression but does not yet support logistic regression.
Charles

Hi Charles

I am doing an evaluation research survey. Kindly tell me how to decide the sample size for rural and urban area, with formula for a study on immunization coverage with the previous coverage evaluation survey indicates a rural coverage percentage at 50 % and urban 68 %. Is it ok to do it with the formula n = 4 pq /L?

• Charles says:

I haven’t enough information to answer your question. Which statistical test are you using? What does pq/L abbreviate?
Charles

6. George Domfe says:

I am just about conducting a survey in Ghana on the informal sector workers. The Ghanaian economy is about 84 % informal and over 14 million Ghanaians are currently working. How do I get the right sample size (using power sampling) for the whole country? Thanks.

• Charles says:

George,
The sample size required depends on the type of statistical test that you are going to use. You need to identify the test that you will use (or that you are considering using) before you can estimate the sample size.
Charles

7. Paul says:

How to amend formula when μ0 ˃ μ1 ? It looks to me as there will be no difference, which subtract from what, since from critical value point of view μ1+z*σ = μ0+z*σ. Thus, one should simply swap them.
Do I understand correctly? I would be glad for help.