Friedman Test

The Friedman test is a non-parametric alternative to ANOVA with repeated measures. No normality assumption is required. The test is similar to the Kruskal-Wallis Test. We will use the terminology from Kruskal-Wallis Test and Two Factor ANOVA without Replication.

Property 1: Define the test statistic

Friedman test statistic

where k = the number of groups (treatments), m = the number of subjects, Rj is the sum of the ranks for the jth group. If the null hypothesis that the sum of the ranks of the groups are the same, then

image2514

when k ≥ 5 or m > 15. The null hypothesis is rejected when Q > \chi^2_{crit}.

Example 1: A winery wanted to find out whether people preferred red, white or rosé wines. They invited 12 people to taste one red, one white and one rose’ wine with the order of tasting chosen at random and a suitable interval between tastings. Each person was asked to evaluate each wine with the scores tabulated in the table on the left side of Figure 1.

Friedman test Excel

Figure 1 – Friedman’s test for Example 1

The ranks of the scores for each person were then calculated and the Friedman statistic Q was calculated to be 1.79 using the above formula. Since p-value = CHITEST(1.79, 2) = 0.408 > .05 = α, we conclude there is no significant difference between the three types of wines.

Observation: Just as for the Kruskal Wallis test, an alternative expression for Q is given by

image2520

where SS'_{Col} is the sum of squares between groups using the ranks instead of raw data.

For Example 1, we can obtain SS'_{Col} from the ranked scores (i.e. range F3:I15) using Excel’s Anova: Two-Factor Without Replication data analysis tool (see Figure 2), and then use this value to calculate Q as described above.

Friedman statistic Excel

Figure 2 – Alternative way of calculating Friedman’s statistic

Real Statistics Excel Functions: The Real Statistics Resource Pack contains the following functions:

FRIEDMAN(R1) = value of Q on the data (without headings) contained in range R1 (organized by columns).

FrTEST(R1) = p-value of the Friedman’s test on the data (without headings) contained in range R1 (organized by columns).

For Example 1, FRIEDMAN(B5:D14) = 1.79 and FrTEST(B5:D14) = .408.

20 Responses to Friedman Test

  1. SIM says:

    Hi Charles,

    Thank you for your tutorial, i am new to statistic. I would like to know how can i determined or select from the table based on my amount of subjects. ie N=5, N=6.
    From the table of critical values for three conditions, how should we determined which P values to choose from?

    Thanks!

    • Charles says:

      Sorry, but I would have to see the table that you are using since I am not providing such a table from my site.
      Charles

  2. Andy says:

    Hi Charles,

    This is one of best explanations I have came across while wanting to know a bit more about Friedman statistic… Thank you!

    I have a couple of questions on the method and conclusion:
    1. Can I use this method as a product tradeoff analysis like conjoint analysis and having no levels?
    As an example, lets say I ask people to share preferences of disruptive technologies. I give them $100 to distribute in the list. I then run Friedman test on collected samples to determine whether difference in choices were significant.

    2. You mention – as p value > 0.05 we conclude there is no significant difference between the three types of wines.
    >> What if p was < 0.05 and in this case does the Ranking order matter? How/When do I definitely say people like first Red then White and then Rose?

    Andy.

    • Charles says:

      Andy,

      I am pleased that you found my explanation of Friedman’s test useful. The following are some observations about your two questions.

      1. I am not sure why you saw “no levels”. Suppose that people have three disruptive technologies to choose from. They assign their $100 among the three technologies. My initial thought is that you perform Friedman’s test (or repeated measures ANOVA if the assumptions are satisfied) to determine whether there is a significant difference between the three technologies. Offhand I don’t see any problem with this approach, but I frankly haven’t enough time to think through this response in sufficient detail to make sure that this is so.

      2. You can order the average preferences as you have indicated, but just because White has a higher ranking than Red in the sample, doesn’t mean that this represents a significant difference (with respect to the population). You would need to perform a post-hoc test to determine this. I have provided tests for post-hoc tests for ANOVA. I am now adding some for Kruskal-Wallis and soon I will look at follow-up testing for Friedman’s test.

      Charles

  3. Katerina says:

    Dear Charles Zaiontz,

    I am using Friedman test to compare three different conditions. I have 33 participants who took part in each condition.

    My question is can I use Friedman test when the three groups are of unequal size? e.g. group 1 12 particip., groups 2 21 particip. etc.

    Thank you in advance.
    Katerina

    • Charles says:

      Katerina,
      Friedman’s test would be used when you have 33 participants who experienced all three conditions. In this case the sample sizes would all be the same, namely 33.
      If you are instead looking at three independent groups, the participants in each group experience one and only one condition, then you want to use fixed factor ANOVA, Kruskal-Wallis or some other similar test.
      Please describe your experiment more clearly.
      Charles

  4. Lucas D. Mazza says:

    To check the p-value use formula: CHISQ.DIST.RT (1.79,2)

  5. Noemi says:

    Dear Mr. Zaiontz,

    I have a very specific problem (surprisingly similar with the one posted in the comments above) and am a bit confused on which statistical test to use. I am sorry if I bother you but I do not understand what you mean with simple arithmetic?
    I have investigated the effect of 5 different treatments and used a scale between 1-5 to evaluate the effect. I repeated the investigation 6 times. My data is therefor ordinal (ranked), non-parametric and not normally distirbuted.
    However, I also calculated the means of each treatment over time of the 6 runs.
    I’m not sure if I have to compare the means of the 5 Treatments or do not use the means at all and make a Friedman test for each treatment of the six runs. I would be very thankful for any Information on how to proceed and what statistical test to use. (I was thinking about Man-Whitney U or Friedman’s test?)
    Thank you!

    • Charles says:

      Noemi,

      I need the some additional information before I can answer your question. First of all I need to better understand the experiment. In particular,

      1. Are all 5 treatments applied to each of the 6 people in the sample?
      2. Are the 5 treatments applied to different people (for a total of 30 people in the study)?
      3. Is the sample divided into 5 groups, one for each treatment, where each person in the sample gets one treatment over 6 different time intervals?

      Before answering the question about which statistical test you should use, I need to understand what hypothesis are you trying to test.

      Charles

  6. Umer says:

    Dear Mr. Zaiontz,
    Your article was really helpful. I just want to confirm the use of Friedman’s test for analysis of data collected through 5-scale likert-type questions. Basically, we have 5 factors and we want to find which one of these are more effective. We have gathered data using 5-scale Likert-type questions with ordinal ranking from 1 to 5 (ranging from Strongly Disagree to Strongly Agree). Can we get a sort of ranking of these five factors according to their effectiveness using this test?
    Thank you!

    • Charles says:

      You don’t need to use Friedman’s test to get a ranking of the five factors. Simple arithmetic is sufficient. What Friedman’s test will tell you is whether any differences between the mean rankings are statistically significant (or are just due to random effects).
      Charles

      • Umer says:

        Thank you very much Mr. Zaiontz!
        So that means that i can find the difference (significant or insignificant) between 4 or 5 groups of data? Basically, it is for research thesis and we cannot simply deduce something based on the ordinal 5-scale. We intent to use t-test for comparing pairs of data set but it will be a good thing if we could test all 5 of them together?

        • Charles says:

          Friedman’s Test tests all 4 (or 5) groups together to determine whether the 4 groups statistically have the same mean (the null hypothesis). If the answer is yes, then you are done. If the answer is no (i.e. at least 2 of the groups have different means), then you would typically perform a t test (or Mann-Whitney test) to determine which two groups are the ones with different means (or some other similar test).
          Charles

  7. Jiahui says:

    Dear Mr. Zaiontz,

    I am writing to ask which data analysis method is appropriate for non-parametric 2 within-subject factors ANOVA.

    My dependent variable is error rate (0-1), which does not apply normal distribution. I transformed the DV data, and conducted the repeated measures ANOVA. I am interested in the results based on non-parametric methods too. But I did not found a proper way yet.

    Thanks!

    Jiahui

    • Charles says:

      Jiahui,
      Friedman’s test is a way of conducting a non-parametric repeated measures ANOVA.
      Charles

      • Jiahui says:

        I know Fiedman’s test is the non-parametric alternative to the one-way ANOVA with repeated measures. Can I use Friedman’s test for two-within factors ANOVA? In case I am interested in the non-parametric two-way repeated ANOVA, which method I can use?

  8. Roger Bakeman says:

    This was very helpful. Even more helpful were your comments for the Wilcoxon signed ranks test because it gave me the information I needed to calculate the 95% confidence interval for T and to calculate the effect size r. Increasingly, journal editors are asking for these. Could you provide information on how to compute a stander error for H (as provided for T) and the effect size r for the Friedman test?

    • Charles says:

      Roger,
      I don’t know of any commonly accepted values for the standard error or effect size for Friedman’s test, although Kendall’s W is often cited as an effect size for Friedman’s H. Here W = H/(m(k-1)) where k = the number of groups (treatments) and m = the number of subjects. Also used as an effect size is the r coefficient for Kendall’s W, which is r = (mW-1)/(m-1). In fact it can be shown that r is the average Spearman correlation coefficient computed on the ranks of all pairs of raters.
      Charles

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>