Statistical Power of the t tests

Power for one-sample test

If we have a sample of size n and we reject the one sample null hypothesis that μ = μ0, then the power of the one-tailed t test is equal to 1 − β where

image9246

and the noncentrality parameter takes the value δ = d\sqrt{n} where d is the Cohen’s effect size

image7309

and μ and σ are the population mean and standard deviation.

If the test is a two-tailed test then

image9247

Note that the degrees of freedom is df = n − 1.

Example 1: Calculate the power for a one sample, two-tailed t test with null hypothesis H0μ = 5 to detect an effect of size of d = .4 using a sample of size of n = 20.

The result is shown in Figure 1.

Power one-sample t

Figure 1 – Power of a one sample t test

Here we used the supplemental function NT_DIST. The Real Statistics Resource Pack also supplies the following function to calculate the power of a one-sample t test.

Real Statistics Function: The following function is provided in the Real Statistics Resource Pack:

T1_POWER(d, n, tails, α, m) = the power of a one sample t test when d = Cohen’s effect size, n = the sample size, tails = # of tails: 1 or 2 (default), α = alpha (default = .05) and m = upper limit of the infinite sum (1−170, default = 80).

For Example 1, T1_POWER(.4, 20) = 0.396994. Note that the power of the one-tailed test yields the value T1_POWER(.4, 20, 1) = 0.531814, which as expected is higher than the two-tailed test.

Power for paired-sample test

The paired sample test is identical to the one sample t-test on the difference between the pairs. If the two random variables are x1, with mean μ1 and x2, with mean μ2, and the standard deviation of x1 − x2 is σ, then power is calculated as in the one-sample case where the noncentrality parameter takes the value δ = d\sqrt{n} and d is the Cohen’s effect size:

image7311

Example 2: Calculate the power for a paired sample, two-tailed t test to detect an effect of size of d = .4 using a sample of size n = 20.

The answer is the same as that for Example 1, namely 39.7%

Example 3: Calculate the power for a paired sample, two-tailed t-test where we have two samples of size 20 and we know that the mean and standard deviation of the first sample are 10 and 8, the mean and standard deviation of the second sample are 15 and 3 and the correlation coefficient between the two samples is .6.

The power is 89% as shown in Figure 2.

Power paired sample t

Figure 2 – Power of a paired sample t test

Based on the definition of correlation and Property 6b of Correlation Basic Concepts

image9119 image9120

For Example 3, this means that

image9121 image9122 image9123

We can now calculate the effect size d as follows:

image9124

Power for independent-samples test

If we have two independent samples of size n, and we reject the two sample null hypothesis that μ1 = μ2, then the power of the one-tailed test is equal to 1 − β where

df = 2n − 2 and the noncentrality parameter takes the value δ = d\sqrt{n/2} where d is Cohen’s effect size

image7311

assuming that the two populations have the same standard deviation σ (homogeneity of variances).

If the test is a two-tailed test then

If the two samples have difference sizes, say n1 and n2, then the degrees of freedom are, as usual n1 + n2 − 2, but the noncentrality parameter takes the value δ = \sqrt{n/2}d where n is the harmonic mean between n1 and n2 (see Measures of Central Tendency).

Example 4: Calculate the power for a two sample, two-tailed t test with null hypothesis μ1 = μ2 to detect an effect of size d = .4 using two independent samples of size 10 and 20.

The power is 16.9% as shown in Figure 3.

Power two sample t

Figure 3 – Power of a two sample t test

As for the one-sample case, we can use the following supplemental function to obtain the same result.

Real Statistics Function: The following function is provided in the Real Statistics Resource Pack:

T2_POWER(d, n1, n2, tails, α, m) = the power of a two sample t test when d = Cohen’s effect size, n1 and n2 = the sample sizes (if n2 is omitted or set to 0, then n2 is considered to be equal to n1), tails = # of tails: 1 or 2 (default), α = alpha (default = .05) and m = upper limit of the infinite sum (1−170, default = 120).

For Example 4, T2_POWER(.4, 10, 20) = 0.169497.

10 Responses to Statistical Power of the t tests

  1. iris says:

    Hi,

    Thank you for the site.

  2. Robert Kazmierczak says:

    Dr. Zaiontz,

    I am working my way through the Real-Statistics web site and am finding the site interesting and informative.

    I have encountered a slight technical glitch. In the section on Student’s t-Ditribution, under Statistical Power of the t-Tests, two images are not displaying (image7308 and image7310). The image numbers are shown, but not the images. All the other images on the page and in the previous sections on Basics and Distributions display properly.

    I do not know if the problem is at the web site end or at my computer end. I have Windows XP, and I have tried viewing the page with both Chrome and Mozilla Firefox, with the same result.

    I have one request of a different nature. Would you consider adding a section on Experimental Design? I think it would be a good fit and in the spirit of the rest of the web site.

    Thank you for providing the web site, and for any help you can provide in viewing these images,

    Yours truly,
    Robert Kazmierczak

    • Charles says:

      Robert,

      Thanks for identifying that two images were missing from the referenced webpage. I have now added these images.

      I agree with your suggestion of adding a webpage on Experimental Design. Given other commitments this won’t happen right away, but I will add such a webpage as soon as I can.

      Charles

  3. William Agurto says:

    Charles:

    I don´t understand why I have to correct the Cohen’s d (effect size) and n (sample size) to get the power for a paired sample t-test. In your example #2 (Figure 2) you use the initial values n=40 and d=.4. But you correct them later: n=20 (say that n_new=20), and calculate a new Cohen’s d (say that Cohen’s d_new=.752071) using a “ro” variable which meaning I don’t understand.
    Could you please explain why I have to correct the initial value of Cohen’s d (Cohen’s d_new= f (Cohen’s d)) and the initial value of n (n_new=n/2)? And what is “ro”? Is ro=1-d? Why I have to use those formulas for correct Cohen’s d?

    Thank you.

    William Agurto.

    • William Agurto says:

      Charles:

      Your example #1 also confuse me: why do you correct the initial value of n? Initial value is n=40; the new value (for calculations) is n_new=20.

      Thank you.

      William Agurto.

      • Charles says:

        William,
        The initial value of 40 is wrong. It should be 20. Thanks for catching this mistake, I have now corrected it on the website.
        Charles

    • Charles says:

      William,
      Sorry for the confusion. Two examples got conflated and some of the information was not included. I will correct this tomorrow. Once again thanks for catching this mistake.
      Charles

    • Charles says:

      William,
      I have now corrected the example on the webpage. Hopefully it is easier to understand now.
      Charles

Leave a Reply

Your email address will not be published. Required fields are marked *