Sign Test

The sign test is a primitive test which can be applied when the conditions for the single sample t-test are not met. The test itself is very simple: perform a binomial test (or use the normal distribution approximation when the sample is sufficiently large) on the signs as indicated in the following example.

Example 1: A company claims that they offer a therapy to reduce memory loss for senile patients. To test this claim they take a sample of 15 patients and test each patient’s percentage of memory loss, with the results given in Figure 1 (range A3:B18). Determine whether the therapy is effective compared with the expected median memory loss over the same period of time of 20%.

Sign test Excel

Figure 1 – Sign test for Example 1

As can be seen from the histogram and QQ plot, the data is not normally distributed and so we decide not to use the usual parametric tests (t-test). Instead we use the sign test with the null hypothesis:

H0: population median ≥ 20

To perform the test we count the number of data elements > 20 and the number of data elements < 20. We drop the data elements with value exactly 20 from the sample. In column C of Figure 1 we put a +1 if the data element is > 20, a -1 if the data element is < 20 and 0 if the data element is = 20.

The number N+ of data elements > 20 (cell B21) is given by the formula =COUNTIF(C4:C18,1). Similarly, the number N- of data elements < 20 (cell B22) is given by the formula =COUNTIF(C4:C18,-1). The revised sample size (cell 23) is given by the formula =B21+B22.

If the null hypothesis is true then the probability that a data element is > 20 is .5, and so we need to test the probability that actually 4 out of 14 data elements are less than the median given that the probability on any trial is .5, i.e.

p-value = BINOMDIST(4, 14, .5, TRUE) = .0898 > .05 = α

Since the p-value > α, (one-tailed test) we can’t reject the null hypothesis, and so cannot conclude with 95% confidence that the median amount of memory loss using the therapy is less than the usual 20% median memory loss.

Note that we have used a one-tail test. If we had used a two-tail test instead then we would have to double the p-value calculated above. Also note that in performing a two-tail test you should perform the test using the smaller of N+ and N-, which for this example is N+ = 4 (since N- = 10 is larger).

Real Statistics Excel Function: The Real Statistics Pack provides the following function:

SignTest(R1, med, tails) = the p-value for the sign test where R1 contains the sample data, med = the hypothesized median and tails = the # of tails: 1 (default) or 2.

This function ignores any empty or non-numeric cells.

Observation: Generally Wilcoxon’s signed-ranks test will be used instead of the simple sign test when the conditions for the t-test are not met since it will give better results since not just the signs but also the ranking of the data are taken into account.

23 Responses to Sign Test

  1. pawan choudhary says:

    Pl send solution of below Q on pawan62003@yahoo.com.

    3. A physical instructor claims that a particular exercise when done continuously for 7 days, reduces body weight at least by 3.5kg. Five overweight girls did the exercise for 7 days and their body weights were found as under:

    Girls 1 2 3 4 5

    Weight before exercise 70 72 75 71 78

    Weight after exercise 66 70 72 66 72

    Making use of the sign test, verify the claim at α = 0.05 that the exercise reduces weight by at least 3.5kg.

  2. Kate says:

    Hi Charles,

    Apologies, but I am new to statistics, so I appreciate your help. I recently administered a pre- and post-lesson survey – -responses ranged along a 5-point likert scale, from strongly disagree to strongly agree. I numbered responses, so that strongly agree =5, and strongly disagree=1. I would like to statistically confirm that there was a change after the intervention, however, although the changes are consistent, they are modest — typically from neutral to agree, or neutral to strongly agree, and they are not normally distributed, i.e. no one started lower than neutral and no one ended up lower than neutral. do you have any recommendations for a good test?

    • Charles says:

      Kate,
      This sounds like the typical paired t test. You need to check whether the differences between the scores after minus before are approximately normally distributed or at least symmetrical. If not you can use the paired version of the Wicoxon Signed Ranks test instead.
      Charles

  3. Vivek says:

    Hi Charles,
    I am looking for a solution to the problem below, but unable to clearly state the null hypothesis, can you please help with the soultion

    A physical instructor claims that a particular exercise when done continuously for 7 days,reduces body weight at least by 3.5kg. Five overweight girls did the exercise for 7 days and their body weights were found as under:
    Girls 1 2 3 4 5
    Weight before exercise 70 72 75 71 78
    Weight after exercise 66 70 72 66 72
    Making use of the sign test, verify the claim at αlpha = 0.05 that the exercise reduces weight by at least 3.5kg.

    • Charles says:

      H0: mean of weight after – mean before – 3.5 < 0, which of course you are trying to disprove. Charles

      • Vivek says:

        Thanks Charles.
        In the given problem, weight is found to be reduced by 4, 2, 3, 5, & 6 Kgs in case of 5 girls. This gives 2 (-) and 3 (+) signs if compared to the claimed reduction of 3.5 kgs.

        Can I state the null hypothesis as
        H0: Weight reduced is >= 3.5 Kg
        &
        Ha: Weight reduced < 3.5 Kg
        now for n=5 and p=0.5 how to accept or reject the H0 ? this is confusing me ….

  4. Uttam says:

    Dear Charles- Thanks again for statistical interpretation on excel. Earlier also i have gained immense knowledge from this website. Help me understand one thing here: i want to set up a goal for a cycle time metric for which i have the population data and have taken a sample out of that. As the data is non-normal, can i go for one-sample sign test to set up a statistically validated target for my project? Will that test be same one as the sign test explained above. If my baseline is 74.5 days CT, what target shall i take up? Can you show with an example? Many Thanks in advance

    • Charles says:

      Uttam,
      Sorry, but I don’t completely understand your question. What null hypothesis are you trying to test? Often (but not always) when the sign test can be used, the Wilcoxon signed ranks test gives better results. Did you consider using this test?
      Charles

  5. Abdul Fattah Bin Haji Osman says:

    Sir,

    Normally, They also include H1: Alternative. Why it isn’t?

    • Charles says:

      The alternative hypothesis is simply the complement of the null hypothesis and so it isn’t always explicitly stated. For the first problem on the referenced webpage, H0: population median ≥ 20 and so H1: H0: population median < 20. Charles

  6. Mo says:

    Dear Charles,
    Thank you so much, you provide a really extensive and informative size of information.
    I conducted an event study on M&As, and calculated t tests, but after normality test I decided to conduct a nonparametric testing.
    As you know, in event studies we end up having a table of abnormal returns for all companies (AR) separately, average abnormal returns of all companies (AAR) and cumulative AR (CAAR), as follows:
    AR AR AR AR AAR CAAR
    0.1 0.2 0.3 0.01 …….. ,,,,,,,,,

    and as a summary we would have
    (-X, 0) t (-X, +X) t (-X, +1)
    -0.71% -0.243 -1.82% -0.350 -0.74%

    after reviewing the literature, I want to conduct a GRANK testing in excel.
    what is the formula to be applied and at which levels (estimation and event windows?) first table or on cumulative data of second window?
    If I want to apply other tests like; Sign Test, Mood’s Median Test, Wilcoxon Rank Sum Test, Mann-Whitney Test, Wilcoxon Signed-Ranks, McNemar’s Test, how can they be applied in the case of event study? what are the 2 samples we are testing against.

    your support is highly appreciated.

    Best wishes

    • Charles says:

      Mo,
      I am quite interested in supporting event studies in my website and software, but haven’t had the time as yet to get to this. I plan to do so, but it will take some time yet.
      Charles

  7. Alie says:

    Hi Charles,
    Could you please help me to discuss TWO INDEPENDENT SIGN TEST? I was looking for different sites but I am wondering I could not find a sample easy to understand or the sign test discussion was for one sample only or for correlation.

    Thanks!

  8. Hao Chen says:

    I wonder how to draw the QQ-plot,in Microsoft Excel or other Statistical Software,please inform

  9. Colin says:

    Sir

    I cannot understand why “p-value = BINOMDIST(4, 14, .5, TRUE) = .0898” is “probability that actually 4 out of 14 data elements are less than the median given that the probability on any trial is .5” .
    Could you please elaborate it?

    Colin

    • Charles says:

      Hi Colin,
      Under the null hypothesis that the population mean is 20, we expect that half of any sample is under 20 and half are greater than 20. This explains why we use p = .5. The sign test is simply a test using the binomial distribution. The referenced example is equivalent to determining whether we have a fair coin when we see that when it is tossed 14 times it comes up heads at least 4 times.
      Charles

      • Marc says:

        Hello Charles,

        Thank you for your extensive work on making stats in Excel easier.

        I’m working on a baseline-intervention trial for which I have perfect information on the baseline population. I took a stratified sample out of this population to select people on which I apply treatment. I’m running group comparison tests on the intervention sample vs. baseline sample. Unfortunately, the baseline population data shows non-normality (negative skew and a small 2nd mode), to the point I cannot use Wilcoxons test. I am opting for Mood’s median test or sign test.

        A) Is is it possible to calculate sign test on the median of the baseline population and then allocate -1/+1/0 on the treatment sample? Or should I use Mood’s test? I have paired samples (same person in baseline and intervention).

        B) Can I use Sign test by calculating the difference between treatment and baseline, saying that 0-hypothesis is no change, then allocating +1/-1/0 in function of whether data is bigger or smaller or equal to 0? BINOM.DIST(number_obs_smaller, count_obs, 0.5, TRUE)?

        C) Is it right to dismiss Wilcoxon’s because of light bimodality and skew? Does the population data have to be strictly symmetrical or can I run it in confidence?

        Sorry, I have a lot of questions. Thank you in advance in case that you find the time to answer :-).

        • Charles says:

          Marc,

          Here are my answers to your questions, assuming that I have understood the problem correctly.

          1. Mood’s test requires that your two samples be independent. If I have understood your scenario correctly, the two samples are not independent, and so you shouldn’t use Mood’s test.

          2. To use Wilcoxon’s ranked-ranks test, you need to look at whether the differences between the paired samples are reasonably symmetric. It is possible that the baseline data is not symmetric, but these differences are symmetric.

          3. If the assumption for Wilcoxon’s test is not met (either as a paired samples test or a one sample test vs. the median), you can use the Sign test (as paired samples or one sample test).

          Charles

          • Marc says:

            Thank you for your reply Charles. I didn’t think about that. I was thinking to much about the population distribution. After all, I am testing if the observed difference can be generalized on the popuation…

            The distribution of the difference between the paired samples (treatment-baseline) is roughly symmetric. I got 8% of observation differences that are out of symmetry (outliers with huge differences). I guess the gain in reliability is not worth the loss of power when switching from Wilcoxon’s paired test to Sign test.

Leave a Reply

Your email address will not be published. Required fields are marked *