Real Statistics Power Data Analysis Tool

Real Statistics Data Analysis Tool: The Real Statistics Resource Pack supplies the Statistical Power and Sample Size data analysis tool to determine the power which results from a statistical test for a specified effect size, sample size and alpha, as well as the sample size required to achieve a specified effect size, power and alpha.

To use this tool, press Ctrl-m and select Statistical Power and Sample Size from the resulting menu. The dialog box in Figure 1 will then appear.

Power Sample Size RequirementsFigure 1 – Statistical Power and Sample Size dialog box

You can now select one of the specified twelve tests and either the Power or Sample Size options. E.g. if you choose the two-sample t test and the Power options, you will be presented with the dialog box shown on the left side of Figure 2.

power-dialog-box-inputpower-dialog-box-output

Figure 2 – Power: two-sample t test dialog box

Next fill in the Input fields, as shown on the right side of Figure 2 (retaining the default values of the # Tails, Alpha and Sum Count fields). After pressing the OK button you will see the results shown in Output area in the dialog box on the right side of the figure.

We see that the power of this test is 48.2% (we explain the other output fields elsewhere). To increase the power of the test for the given effect size, you would normally need to change from a two-tailed to a one-tailed test, increase the sample size(s), increase the alpha value or make some combination of these.

For this example if you change the # Tails to 2, you would increase Power to 60.8%. If you keep to a two-tailed test, you would need to increase alpha to .28 to obtain power of at least 80%, which is usually not very acceptable. Alternatively you could increase the sample sizes. E.g. to achieve a power of 80% you would need to increase the sample sizes to 176.

This can be determined by using the Sample Size option shown in Figure 1. You can do this by first pressing the Cancel button (on the dialog box in Figure 2) or the x in the upper right-hand corner of the dialog box. When the dialog box shown in Figure 1 reappears select the Sample Size option.

The dialog box shown on the left side of Figure 3 now appears. You need to insert .3 for the Effect Size and press the OK button. The result is shown on the right side of Figure 3, confirming that samples sizes of 176 are required. Note that the actual power achieved is a little more than 80%.

Sample size dialog box

Sample size dialog output

Figure 3 – Sample size: two-sample t test dialog box

Note that the Sample size ratio is set to 1, which means that the ratio n2/n1 = 1 (where n1 = size of sample 1 and n2 = size of sample 2), i.e. the samples are equal. If we wanted the size of the second sample to be twice that of the first sample we would set Sample size ratio to 2.

We could also specify that sample 2 has a specific size, say 100, by using a negative value for Sample size ratio, -100 in this case. Note that if we fix n2, then to achieve power of 80% for the above example, we would need a larger sample 1. In particular, we would need sample 1 to have size 695.

15 Responses to Real Statistics Power Data Analysis Tool

  1. Shabbir says:

    Dear Dr. Charles,
    I have very little statistical knowledge and I have failed in performing one of the task asked to do by my supervisor in thesis. I am BBA student. This thesis is a requirement of BBA degree.
    In my research study, I have 3 factors where each factor has 10 questions. I am asked to provide a normal distribution graph with 95% confidence interval where I have to use mean and standard deviation of each question within a factor and the factor as a whole. So, I need to put 11 mean and standard deviation in a graph. Thus, 3 graph for 3 factors.
    I am providing descriptive statistics of one factor named ” Job Motivation” and questions within this factor.
    Descriptive Statistics
    N Range Minimum Maximum Sum Mean Std. Deviation Variance
    Job security 50 4 1 5 209 4.18 1.044 1.089
    salary 50 4 1 5 188 3.76 1.117 1.247
    clarity 50 3 2 5 214 4.28 .701 .491
    interesting 50 3 2 5 226 4.52 .814 .663
    appreciation 50 3 2 5 210 4.20 .808 .653
    finish within time 50 4 1 5 121 2.42 1.430 2.044
    learning opportunity 50 4 1 5 188 3.76 1.170 1.370
    recognize work 50 4 1 5 211 4.22 .864 .747
    atmosphere 50 4 1 5 158 3.16 1.283 1.647
    growth 50 4 1 5 196 3.92 1.027 1.055
    Job Motivation 50 2.75 2.12 4.88 189.88 3.7975 .56463 .319
    Valid N (listwise) 50

    Please help me pass through this problem.
    Thank you

  2. Piero says:

    Dear Dr. Charles,
    I would like to have your comments about this experimental design.

    I have to test the efficacy of a treatment on a single subject.
    The device that measures the outcome variable performs a large number of measurements in a short period of time, but it returns only the mean and the standard deviation of all measurements, and the number of measurements on which this mean has been computed. So I have:

    (mean1, sd1, n1) for condition A (pre-treatment)
    (mean2, sd2, n2) for condition B (after treatment)

    I understand that I can only perform a t-test between the two conditions to verify the null hypothesis that mean1 is statistically different from mean2;
    but I don’t know which is the correct way to compute the power of the test.
    I can set the measurement device to have n1=n2 to simplify computations.

    Thank you very much for your help!

    Piero

    • Charles says:

      Piero,
      If I understand correctly, the only data that you have are the following six statistics:
      (mean1, sd1, n1) for condition A (pre-treatment)
      (mean2, sd2, n2) for condition B (after treatment)
      Is this correct? Did you use the paired t test?
      Charles

      • Piero says:

        Dear Charles,

        yes these are the only data I have.
        So I use the classical formula for t-test:

        t = (mean1 – mean2)/sqrt((sqr(sd1)+sqr(sd2))/n)

        by supposing n1=n2=n .
        I don’t know how to apply a paired t-test to these data, by considering that I have only one subject and only these statistics that I reported.

        Thank you again
        Best Regards
        Piero

        • Piero says:

          Dear Charles,

          please, could you give me some help about how to compute the statistical power for the particular test reported in my messages before?
          I searched on my books and on the web for similar cases, but I am still quite confused, in effect I am even not sure if it is really possible to perform such computation!

          Thank you very much for your help

          Best Regards
          Piero

  3. Basundhara says:

    some how i am not being able to use this tool. the fig 1 is appearing on the screen but after that it does not work . please help.

    • Charles says:

      Basundhara,
      Please describe the problem in more detail.
      Are you able to use other tools, but not this tool? Which version of Excel and Windows are you using? What do you see when you enter the formula =VER()
      Charles

  4. Sael says:

    Hi

    I performed a one way ANOVA and I did not get significant result.
    Omega Square is 0.021
    I need to calculate the required power using cohen’s F for the one way ANOVA to detect a significant result.
    Using statistical power and sample size in real statistics tool, I did not quite understand (SUM COUNT) it set on 40, what does this number represent?

    • Charles says:

      The value of the noncentral F distribution is based on an infinite sum. 40 represents the number of terms in the sum that are used to calculate the non central F distribution value.
      Charles

  5. Ori says:

    Hi,

    I’m trying to preform a power calculation for two samples with a normal distribution and I’m not sure what value should I enter in the Alpha field.

    From my calculation when preforming an hypothesis test (N0 mean 1 = mean 2, N1 mean 1 – mean 2 > 0), my critical Z score for a 95% confidence level for a one tail test is 1.64, and my actual Z score of the difference between the means is 1.38, meaning I do not reject the null hypothesis.
    The P value for 1.38 is 8.24% of committing a type I error.

    Do I need to enter 0.05 for the alpha or should I enter 0.0824?

    English is not my native language so I hope I managed to translate the statistical terms correctly.

    Thanks!
    Ori

Leave a Reply

Your email address will not be published. Required fields are marked *