Statistical Power

The following four measures are interrelated:

  • sample size
  • effect size (which is related to variance)
  • alpha
  • power (which is related to beta)

In particular, for a given sample size and effect size, the lower the alpha the higher the beta (i.e. the lower the power) and vice versa.

Usually we are interested in the following two consequences of the interaction of the above four parameters:

Calculate the minimum sample size (a priori): Prior to conducting an experiment, it is important to determine the minimum sample size necessary to achieve the desired power based on an assumed alpha and estimated effect size. The effect size is based on similar experiments, theoretical considerations or as a last resort commonly used guidelines (e.g. those of Cohen described in Standardized Effect Size).

Calculate the power of a test (post hoc): After conducting a specific statistical test with a selected alpha and sample size, you can determine the effect size, and then calculate the power of the test. If this value is at least .80 (or some other target value), then you can be confident that the power of the test is sufficient to determine the effects, but if the power is lower than this value then you need to consider repeating the experiment with a larger sample if this is practical.

The table in Figure 1 gives some idea of how the minimum sample size varies based on the parameters (a priori analysis). The example is for the two-tailed Student’s t-test for a single sample (see One Sample t Test) where d = .2 represents a small effect, d = .5 represents a medium effect and d = .8 represents a large effect.

Sample size requirements

Figure 1 – Sample size requirements

E.g. the first line of the table shows that a sample of size 199 or more is needed to detect an effect of size d = 0.2 with power 1 – β = 0.80 and α = 0.05.

The Real Statistics Resource Pack provides a number of worksheet functions for carrying out both a priori and post hoc tests in Excel. We illustrate these via numerous examples throughout this website (see, for example, Power and Sample Size, Statistical Power of the t-TestsPower for One-way ANOVA, etc.).

The Real Statistics Resource Pack also provides a data analysis tool which calculates power and sample size requirements for ten statistical tests. Click here to access a description of this data analysis tool.

If you need power and sample size capabilities beyond those provided by the Real Statistics Resource Pack there are software programs that can do this for you. One such program, G*Power, is available for free online [FE].

Leave a Reply

Your email address will not be published. Required fields are marked *