Shapiro-Wilk Original Test

We present the original approach to the performing the Shapiro-Wilk Test. This approach is limited to samples between 3 and 50 elements. By clicking here you can also review a revised approach using the algorithm of J. P. Royston which can handle samples with up to 5,000 (or even more).

The basic approach used in the Shapiro-Wilk (SW) test for normality is as follows:

  • Rearrange the data in ascending order so that x1 ≤ … ≤ xn.
  • Calculate SS as follows:

image3595

  • If n is even, let m =  n/2, while if n is odd let m = (n–1)/2
  • Calculate b as follows, taking the ai weights from the Table 1 (based on the value of n) in the Shapiro-Wilk Table. Note that if n is odd, the median data value is not used in the calculation of b.

image3598

  • Calculate the test statistic W = b2 ⁄ SS
  • Find the value in the Table 2 of the Shapiro-Wilk Table (for a given value of n) that is closest to W, interpolating if necessary. This is the p-value for the test.

For example, suppose W = .975 and n = 10. This means that the p-value for the test is somewhere between .90 and .95.

Example 1: A random sample of 12 people is taken from a large population. The ages of the people in the sample are given in column A of the worksheet in Figure 1. Is this data normally distributed?

Shapiro-Wilk test Excel

Figure 1 – Shapiro-Wilk test for Example 1

We begin by sorting the data in column A using Data > Sort & Filter|Sort or the QSORT supplemental function, putting the results in column B. We next look up the a coordinate values for n = 12 (the sample size) in Table 1 of the Shapiro-Wilk Table, putting these values in column E. Corresponding to each of these 6 coordinates a1,…,a6, we calculate the values x12 – x1, …, x7 – x6, where xi is the ith data element in sorted order. E.g. since x1 = 35 and x12 = 86, we place the difference 86 – 35 = 51 in cell H5 (the same row as the cell containing a1). Column I contains the product of the coordinate and difference values. E.g. cell I5 contains the formula =E5*H5. The sum of these values is b = 44.1641, which is found in cell I11 (and again in cell E14).

We next calculate SS as DEVSQ(B4:B15) = 2008.667. Thus W = b2SS = 44.1641^2/2008.667 = .971026. We now look for .971026 when n = 12 in Table 2 of the Shapiro-Wilk Table and find that the p-value lies between .50 and .90. The W value for .5 is .943 and the W value for .9 is .973. Interpolating .971026 between these value, we arrive at p-value = .873681. Since p-value = .87 > .05 = α, we retain the null hypothesis that the data are normally distributed.

Example 2: Using the SW test, determine whether the data in Example 1 of Graphical Tests for Normality and Symmetry are normally distributed.

Shapiro-Wilk normality Excel

Figure 2 – Shapiro-Wilk test for Example 2

As we can see from the analysis in Figure 2, p-value = .0419 < .05 = α, and so we reject the null hypothesis and conclude with 95% confidence that  that the data are not normally distributed, which is quite different from the results using the KS test that we found in Example 2 of Kolmogorov-Smironov Test.

Real Statistics Function: The Real Statistics Resource Pack contains the following supplemental functions where R1 consists only of numeric data without headings:

SHAPIRO(R1, False) = the Shapiro-Wilk test statistic W for the data in the range R1

SWTEST(R1, False) = p-value of the Shapiro-Wilk test on the data in R1

SWCoeff(n, j, False) = the jth coefficient for samples of size n

SWCoeff(R1, C1, False) = the coefficient corresponding to cell C1 within sorted range R1

SWPROB(n, W) = p-value of the Shapiro-Wilk test for a sample of size n for test statistic W

The functions SHAPIRO and SWTEST ignore all empty and non-numeric cells. The range R1 in SWCoeff(R1, C1, False) should not contain any empty or non-numeric cells.

For example, for Example 1 of Chi-square Test for Normality, we have SHAPIRO(A4:A15, False) = .874 and SWTEST(A4:A15, False) = SWPROB(15,.874) = .0419 (referring to the worksheet in Figure 2 of Chi-square Test for Normality).

Note that SHAPIRO(R1, True), SWTEST(R1, True), SWCoeff(n, j, True) and SWCoeff(R1, C1, True) refer to the results using the Royston algorithm, as described in Shapiro-Wilk Expanded Test.

For compatibility with the Royston version of SWCoeff, when j ≤ n/2 then SWCoeff(n, j, False) = the negative of the value of the jth coefficient for samples of size n found in the Shapiro-Wilk Table. When j = (n+1)/2, SWCoeff(n, j, False) = 0 and when  j > (n+1)/2, SWCoeff(n, j, False) = -SWCoeff(n, n–j+1, False).

22 Responses to Shapiro-Wilk Original Test

  1. javiera says:

    Your information is really helpfull, thank you for that!
    I have a question, pretty elemental, but I need an answer. In example number 1 I’m following you until the interpolation, how did you do that? I mean, table number 2 doesn’t follow a lineal function, so i tried to put it into a logaritmic function and it didn’t work.
    I would apreciate your answer, I really need it.

    Thank you.

    • admin says:

      Hi Javiera,
      It is a good question. I simply used a linear interpolation. As you pointed out, the table doesn’t represent a linear function, but the results will usually be good enough. I will probably use a more sophisticated approach in the future, but for now I wanted to keep it simple.
      Charles

  2. Touseef says:

    Dear Admin,

    Thanks for the information you provided about SW test. As i am not having the statistic background so i have a small question as when the random samples increase up to n=500 suppose, how would we get the weights “a” as the table only provides “n” up to 50 only?
    I would appriciate if you could provide the answer.

    • admin says:

      Hi Touseef,
      I got the “a” weights from the original paper of Shapiro and Wilk in 1965. In that paper they only provided weights up to n = 50. If it is truly a random sample, then by the Central Limit Theorem for large values of n (usually n > 50 is more than sufficient), the sample will be approximately normally distributed and so doesn’t need to be tested for normality (again provided the sample is truly randomly selected).
      Charles

  3. Molly says:

    Hi,
    Thank you so much for posting this! It was really helpful and easy to understand. My only question is in question one: how do you interpolate the W values? Is there an equation you used?

    • admin says:

      Hi Molly,

      Currently, I don’t do anything particularly sophisticated. I simply perform a linear interpolation for the W values. Even if the value is not precise, it is much higher than the p-value of .05, and so we cannot reject that the data is normally distributed.

      Because so many people have asked for the Shapiro-Wik test for samples larger than 50, yesterday I added a new version of the SW test which doesn’t use interpolation and supports sample sizes of at least 5,000. This is available in the current version of the Real Statistics Resource Pack (R1.7.1).

      Charles

  4. Dmitry says:

    I do not understand the answer to Touseef. The CLT says that the sampling distribution of the mean is approx. Normal for large random sample. It says nothing about the distribution of the sample values. If you are sampling from a non-Normal population, the sample distribution will not be Normal no matter how large the sample is, right?

    • Charles says:

      Dmitry,
      The CLT says that the sample distribution will indeed be approximately normal for large enough samples even if the population distribution is not normal.
      Charles

  5. Louis says:

    I’ve done a few tests using your RS resources pack this and I’m afraid to say I’m detecting a bug of sorts, SWTEST(R1) doesn’t always return the same as SWPROB(n,W) – the latter giving the correct result. I’m not sure if you actually checked this for different values when testing the algorithm? Really not trying to be ungrateful, it’s a brilliant add-in, but I just noticed that in “SHAPIRO(A4:A15) = .874 and SWTEST(A4:A15) = SWPROB(15,.874) = .0419″ the range A4:A15 wouldn’t provide a value of 15 for n unless I’m mistaken?

    Again, thanks for all your work on the site/excel package!

    • Louis says:

      my example data, starting with a label in A1 :
      —————————-
      sample1: 2.8078385
      sample2: 6.22198918
      sample3: 100
      sample4: 58.555133
      sample5: 9.0669786
      sample6: 2.2813688
      sample7: 0.6727113
      W: =SHAPIRO(B2:B8)=0.7118325
      incorrect p-value: =SWTEST(B2:B8)=0.782674
      (from what I can see) correct p-value: =SWPROB(7,B32)=0.005

      From what I can see in the p table for n=7, W=0.7118… lies between p=0 and p=0.01, i.e. p=0.005 is feasible but 0.782674 ain’t.

      I think I’m just going to use SWPROB for now!

      • Charles says:

        Dear Louis,

        Thanks for finding this error. There was an error in my implementation of the Royston algorithm for performing the Shapiro-Wilk test for normality for samples between 4 and 11 elements. I believe that I have now fixed this in the latest release of the Real Statistics Resource Pack which I just put on the website. If you download and install this version (release 1.7.3), you should find that SWTEST(B2:B8) = .004981. The value using the original SW algorithm is calculated by SWTEST(B2:B8,False) = .005.

        Please note that the following Shapiro-Wilk functions have both Royston and original SW versions: SHAPIRO(R1,b), SWTEST(R1,b), SWCoeff(n,j,b). If b is True or is omitted then the Royston algorithm is used. If b is False then the original SW algorithm is used. The SWCoeff(n,j,False) version is new. I just added it to the software. It gives the a coefficients found in the SW Coefficient Table on webpage http://www.real-statistics.com/statistics-tables/shapiro-wilk-table/. I will be updating the website shortly to explain this new function.

        Caution should be used when employing either version of the SW Test for very small samples (less than 15 or 20) since the results are not completely accurate.

        Thanks again for identifying the error and sorry for any inconvenience it caused.

        Charles

    • Charles says:

      Hello Louis,

      For Example 2 on the webpage http://www.real-statistics.com/tests-normality-and-symmetry/statistical-tests-normality-symmetry/shapiro-wilk-test/, we have the following results SHAPIRO(A4:A18, False) = .874 and SWTEST(A4:A18, False) = SWPROB(15,.874) = .0419. The range used is A4:A18 and not A4:A15, and so a sample size of 15 is correct.

      Please note that to use the original Shapiro-Wilk algorithm you need to specify False as the second parameter. The website was not clear about this. I have now revised the website to make this clearer. If you leave out the second parameter, you will get SHAPIRO(A4:A18) = .874 and SWTEST(A4:A18) = 0387, which are the results using the Royston algorithm. Obviously in this case there wasn’t a great difference.

      Charles

  6. Michael Hardy says:

    Table 2 gives “the probability that the data comes from a normal distribution”??

    Could it be that what Table 2 gives is the p-value? It is commonplace to be incessantly explaining to people who don’t understand math that the p-value is NOT the same as the probability of the null hypothesis. This is called “the prosecutor’s fallacy”; Google that term!

    Given the null hypothesis of the defendant’s innocence, the probability of a test statistic at least as extreme as the one observed is 1 in 1000. So the prosecutor’s fallacy is the mistaken idea that that means there’s a 1 in 1000 chance of innocence. Consider: Suppose the culprit is one member of a population of 500,000. If something happens 1 time out of 1000 to each person, then there are 500 people in that population for whom the p-value is at least as extreme as what was observed. That means the defendant is one of the 500 people who could be the culprit.

    • Charles says:

      Michael,
      You are absolutely correct. I have now made the corrections in the appropriate places on the webpage. Thanks for identifying the error.
      Charles

  7. Richard says:

    I believe there are two minor errors in the two paragraphs following Figure 1. First, about halfway through the paragraph following Figure 1, the text states “. . . we calculate the values x12 – x11, . . . ” I believe x11 is a typo and the correct text should read “. . the values x12 – x1, . . . ”

    Second, I believe there is an error in the cell range given in first sentence, of second paragraph, following Figure 1. Specifically, the text currently reads “We next calculate SS as DEVSQ(B5:B16) = . . . ” The correct cell range should be ” . . . as DEVSQ(B4:B15) = . . . ”

    Thanks for the great web page! Its has been very helpful to me.

    • Charles says:

      Richard,
      You are correct on both counts. Thanks for finding these errors. I have now updated the webpage with your corrections. I am pleased to read that you found the webpage helpful. I hope that you will continue to use the site.
      Charles

  8. olu says:

    Hi guys,

    I understand t tests can only be performed on two groups of samples with uniform distribution. What can i use if one group is uniform while the other isn’t?

    • Charles says:

      Olu,
      The t test requires normally distributed samples (not uniform), but it is pretty forgiving and will usually work well with reasonably symmetric data. If one group is not acceptable then you should use a non-parametric test. E.g. if the two samples are independent then you can use Mann-Whitney.
      Charles

  9. Colin says:

    Sir

    SWCoeff(n, j, False) gives us a negative number. LOL. You may change it in the next version of Real Statistics, or we can just multiply -1 to fix the problem.

    Colin

  10. Colin says:

    Sir

    SWCoeff(n, j, False) gives us a negative number. LOL. You may change it in the next version of Real Statistics, or we can just multiply -1 to fix the problem. I am using the latest Real Statistics add-in.

    Colin

    • Charles says:

      Colin,
      I set SWCoeff(n,j,False) to be negative for consistency with the Royston version, namely SWCoeff(n,j,True). For j < (n+1)/2 SWCoeff(n,j,False) is negative, while for j > (n+1)/2 it is positive.
      Charles

  11. Colin says:

    Sir

    Sorry! Forgave me. I found the explanation at the end of this blog.

    Colin

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>