ANOVA with Repeated Measures

In this part of the website we apply the ANOVA methodology of One-way ANOVA and Two-way ANOVA to the extension of the paired samples problem studied in Paired Sample t Test. In this analysis, known as ANOVA with Repeated Measures, the rows correspond to subjects or participants in the experiment and the columns represent various treatments (often based on time) for each subject.

Topics:

90 Responses to ANOVA with Repeated Measures

  1. Eliza says:

    Hey there!
    I’m am getting confused when finding the F value. My SSB is 143.44 my SSW is 715.5 and my SSS is 658.3
    Any help would be greatly appreciated!

  2. Andrea says:

    Hi Charles,

    Your website is definitely making people’s lives easier!

    I have a question and I hope you will be able to give me a hint.

    I am experimenting with electrical stimulations on subjects.
    Each stimulation has 2 parameters (frequency and pulse), in total I have 12 combinations of frequency/pulse (3 frequencies and 4 pulses).

    I have 16 subjects.
    Each of the subjects has been stimulated with 36 random stimulations (each combination frequency/pulse has been given to them 3 times).

    In order to characterize I thought to use a 3 way repeated measurement ANOVA, do you think is a good idea?

    • Charles says:

      Andrea,
      I understand that your fixed factors are Frequency and Pulse and the repeated factor is Time (time 1, time 2 and time 3). You have 16 replications (the subjects). This does seem to fit the profile of a 3-way repeated measures ANOVA. If there is a learning element, then this seems to be an appropriate model. If not, the times are completely interchangeable, and perhaps you can simplify the model by adding (or averaging) the three time measurements to obtain a two factor ANOVA.
      Charles

  3. Alessandro Zarpellon says:

    Hi Charles. Thanks again for such a great website!
    I have run some experiments comparing 2 groups, control and treatment, monitoring an output for 5 days, thus I believe ANOVA with repeated measures is the right model to analyze my data. I then run an F-test for variance between subjects which turns out statistically significant, so that I do not have common variance. You state that ANOVA is pretty robust to reasonable violations of this assumption, but what it is “reasonable” in this case? What do you suggest to do in this case? I can run more experiments of course, but there is any other way to analyze the data?
    You also use the same dataset for two factor multivariate Repeated Measures Analysis with Two Dependent Variables and conclude saying that in this model “All the tests show there is no significant difference between the mean hours of sleep for the three age groups. Interestingly enough this is the opposite conclusion from the univariate test” . My question is how do I choose which one is the most powerful analysis?
    Ciao!

    • Charles says:

      Alessandro,

      A common variance is an assumption of the ANOVA test, not a conclusion. I significant result from ANOVA is that the means (not the variances) are different. To test the assumption that the variances are equal, you can use Levene’s test. See the following webpage:
      Homogeneous Variances

      Regarding whether to use the univariate test or MANOVA, some guidelines are given on the following webpage:
      Sphericity
      This is one of the areas that makes statistics as much art as science. Different tests may yield different conclusions, usually based on different assumptions, which may not be 100% satisfied. Also keep in mind, if one test yields p-value = .049 and the other yields p-value = .051, the formal conclusion will be different, but clearly both tests show a borderline result.

      Charles

      • Alessandro Zarpellon says:

        Hi Charles, thanks for your prompt reply. Yes of course the test of homogeneous variances is an assumption that needs to be tested before to compare means with ANOVA. My question though is, if I run an F-test (with two groups) or a Levene`s test (with more than two), and I do not find equal variances, can I still “trust” the ANOVA results? Or if I cannot do it, what other tests can be run, in the absence of equal variances? Is it simply better to generate more data?
        Thanks!

        • Charles says:

          If the variances are not reasonably similar, then you can’t trust the ANOVA results. A rule of thumb is that if the largest variances is more than 3-4 times the smallest variance, then I will be concerned about the accuracy of the ANOVA results. This is only a rule of thumb, sometimes ANOVA can tolerate a larger spread, sometimes it can’t even tolerate this spread. When the assumption fails, you should consider Welch’s ANOVA. If your samples were very small, then generating more data could be helpful in order to better estimate the real (i.e. population) variances. But with reasonable sized samples, generating more data is probably not helpful.
          Charles

  4. Dan Green says:

    Hi Charles,

    I’m studying the effect on Fv:Fm (photosynthetic efficiency of photosystem II) of a chemical over a period of 5 days.

    I have 6 concentrations (including 1 control) and 6 repeats /treatment.
    All treatments are continuous over a 5 day period.

    Would a repeated measures ANOVA be the correct test?

    Thank you in advance for your time

    • Charles says:

      Dan,
      This sounds like a fit for repeated measures ANOVA, with the proviso that I am not sure what you mean by “treatments are continuous…”
      Charles

  5. chi says:

    hi sir. what test do i use in an experiment wherein there are 3 groups assigned with different doses of a drug with pre- and post-treatment blood glucose levels taken? sample sizes for each group are 6, 6, and 8. thank you in advance!

  6. Elliot says:

    Hi Charles,

    I am investigating the number of prey brought home by 26 domestic cats over 4 seasons.

    1) Is the repeated-measures ANOVA the correct test to use when comparing the no. of prey brought home between each season?

    2) Does Excel 2013 allow you to conduct a repeated-measures ANOVA or is it best to use SPSS?

    Thanks for your time,

    Elliot

    • Charles says:

      Elliot,
      1. If you have the data for the same 26 cats over 4 seasons, then the repeated measures ANOVA looks like a good test to use.
      2. As described on the referenced webpage, you can use Excel 2013’s Two Factor ANOVA without Replications to do the One Factor ANOVA with Repeated Measures test, but without accounting for sphericity. You can use the Real Statistics Excel tool to perform the full repeated measures test including sphericity. You can download this tool for free from the website.
      Charles

  7. cyntechtic says:

    Dear Charles the example you referenced is repeated with discrete values for the variables of interest, i.e. each variable’s value was taken exclusive of its previous value. But mine is also repeated but with continuous values that are cumulative i.e. the value of the plant heights for the sixth week is cumulative of the previous 5 weeks value, the value for the fifth week is cumulative of the previous 4 weeks and so on
    My question:
    Should the analysis be on the RATE OF CHANGE OF GROWTH (difference in growth from week to week) or the CUMMULATIVE VALUE OF GROWTH (continuous height values)?

  8. cyntechtic says:

    I have an experiement of a plant which was randomly assigned to 4 treatment with 9 replication each, the weekly height was taken over six weeks. how do I carry out my anova.

  9. sooraj.r says:

    Dear Charles,

    I have 3 doctors performing the same operation individually . i have the hemoglobin level data for pre and post. How do i compare the results?

    Now if i have 1 week after hemoglobin values to go with this like pre, post and 1 week after how do i compare the 3 doctors values?

    Please guide me on this.

  10. Marvin Toledo says:

    Dear Sir Charles,
    I have a problem with my data, I have 317 respondents, 3 groups/ treatment (Cow Milk, Blend Milk, Coco Milk), and time period (Baseline, Midline and Endline). I want to see the difference of the time periods to see if there is an effect in weight of children for each group/treatment. My problem is at baseline (before the experiment was conducted) I have 317 respondent measured there weight in kg, at midline some of the respondent had drop out in the experiment. Now I only 158 respondent at midline (middle stage of the experiment) and at endline (final stage) i now only have 111 respondent. Is it valid to use RM ANOVA with unequal sample? or shoul I use list deletion for my data?
    I hope you can help me Sir Charles,
    Thank you
    Marvin

    • Charles says:

      Marvin,
      You can us RM ANOVA with unequal samples, but I don’t support this yet.
      Right now, the only approach that I support is listwise deletion of samples with missing data.
      Charles

      • Marvin Toledo says:

        Thank you Sir Charles,
        One last question sir Charles, can I use mixed model ANOVA with my data?

        • Charles says:

          Marvin,
          It depends on what you mean by mixed model ANOVA. If I remember correctly, you have one repeated measures factor (Time) and one fixed factor. In this sense it is a mixed model.
          Charles

  11. Sue says:

    Dear Dr Charles,

    I’m a beginner in statistic and Im not quite sure what design should I used to analyse my data. I hope you can help me figure it out. I took my sample on the upstream and downstream of a river, and I measured the total cell concentration in both samples (without replicates). I repeated this steps monthly for one whole year (12 times). Can I use two-way anova without replication to analyse my data? And if there is significant difference, how can I determined in which month the upstream and downstream are significantly different?
    Im really sorry for asking basic questions. I would really appreciate your help.

    • Charles says:

      Sue,
      Each month are you taking one sample upstream and one sample downstream? Or more than one sample?
      Charles

      • Sue says:

        Each month I take one sample upstream and one sample downstream. I also have another sampling point which is at the outfall, which makes my total sampling point= 3. But for this test, I just want to analyse data from upstream and downstream point. Is it possible to do analysis with only these data?

        Fixed factor : Sampling point (2 level: upstream, downstream)
        Repeated factor: month (12 level: jan-dec)
        Depended variable: total cell concentration

  12. Pedro says:

    Hi, I’m a little confused as to what to use to analyse my data, and I wonder if you could assist me.

    My participants have watched a tv show from three different distances, and I want to see if there is a relationship between engagement and distance. I have 3 means from answers for each participant, but I’m unsure which ANOVA test to run?

    • Charles says:

      Pedro,
      If the same participants are watching the tv shows from the three distances then this is ANOVA with repeated measures.
      Charles

  13. Ane Iversen says:

    Dear Charles

    I really need some advice about which statistics to use.
    I have conducted a study on tumor growth over time in two groups of mice. One group was treated with placebo, one group with a drug. In each group there were 6 mice.
    Tumor size (volume) was measured at day 4, 8, 11, 15, 19, 22, 25 and 28.
    Do I use a repeated measures ANOVA to finde out, if there is any significant reduction in growth, when mice are treated with the drug.
    And can you recommend an online internet-site to d0 these calculations?
    Thank you! Ane

  14. Sowmya Ramesh says:

    Dear Charles,

    My data consists of a. Control (untreated), b. Treatment 1, c. Treatment 2, d. Treatment 3. I measure the bone length at day 0, 4, 7 and 14. I follow the same bones each time I measure the length. The experiment was repeated twice and each experiment had 4 bones per group. I have combined the data of the experiments and included in the sheet.
    I basically want to know if the treatment is having any effect as compared to the control and also within the treatment groups (different doses) does it have any effect.
    I tried doing One way ANOVA for this and Bonferroni’s Multiple Comparison Test and the result was not significant. When I performed RM ANOVA with Bonferroni’s Multiple Comparison Test, the result is significant.
    Please let me know which is the ideal test?

    Sowmya

    • Charles says:

      Sowmya,

      You have one fixed factor Treatment (with 4 levels). You have repeated measures factor Time (with 4 levels). The dependent variable is bone length. So far this looks like a Repeated Measures ANOVA of the type shown on the following webpage
      http://www.real-statistics.com/anova-repeated-measures/one-between-subjects-factor-and-one-within-subjects-factor/

      There are three things that are not clear to me:
      1. You mention that you want to understand the effect of different dosages. Do the treatment reflect the different dosages or is Dosage yet another fixed factor? If the latter, this makes the design more complicated.
      2. What do you mean that the experiment was repeated twice? Was it repeated on the same bones or different bones?
      3. When you say that there are 4 bones per group, are these the same type of bone selected from 4 different individuals or are these 4 bones from the same individual? In the section case another factor, namely Bones, may be necessary.

      Charles

      • Sowmya Ramesh says:

        Thanks a lot for the prompt response!

        To clarify:
        1. Yes, treatment reflects different doses. Each group has different dosages (like group 1 – 90mJ, group 2 – 120mJ, group 3 – 180mJ). So does the treatment (different doses) affect the bone length is what I would like to know?

        2. I had done one set of experiment on the bones, monitored their lengths individually at different time point. And the same experiment was repeated on another set of bones on another day (another animal). Therefore it’s is n=2. For the purpose of statistics, I averaged the values from set one and set two and then did the statistics to be able to get individual error bar at different time points.

        3. Yes, it is 4 bones per group from the same individual.

        Is RM ANOVA, a suitable for this experimental design?

        Thanks

        Sowmya

        • Charles says:

          It does seem like repeated ANOVA is the correct test. The sample is quite small and so I would expect that the power of the test would be quite low.
          Charles

  15. Katie Robinson says:

    Hello,

    I have three treatments with six individuals in each treatment. I have collected mass and length data for each individual for a five year period. I would like to see if the treatments have had an impact on growth. I was told that an RM ANOVA was required and I wanted to ask if this is correct.

    Thank you!

    Katie

    • Charles says:

      Katie,
      This is correct provided you can use the mass and length data to create a combined statistic for growth. Otherwise you need to use Repeated Measures MANOVA.
      Charles

  16. Panniker says:

    Dear Charles,
    I am doing a quasi experimental research with three therapies. A,B, C. In this there are three groups of 10 each who will be given three treatments such as: group 1-treatment A, group2- treatment A-B, group 3- treatment A-C. There is a control group as well with other routine remedies. What statistical analysis I can use to identify intervention impact.
    Note: treatment -A is common for the three groups.

  17. Sandra says:

    Dear, Experts!
    I have a little problem with my statistical analysis for Master’s work. In my experiment, I am observing changes in composite material radiation attenuation after material repeated use. So, I have 6 samples: three of them are 3mm thick structured with particle sizes 50nm, 500nm and 5um; and three of them are 7mm thick structured similar as first. Radiation attenuation changes are observed on 5 electron energies: 6, 9, 12, 16 and 20MeV. The measurements were performed four times: first time – material mixed with US method and other 3 times after hand mixing – the transmission factor was measured. I wanted to perform N-way ANOVA test in SPSS to understand are there any relations between these factors and is there any impact on transmission factor changes. I used General Linear Model->Repeated Measures. Defined 4 repeated levels – four transmission factor measurement and then, “Between Subject Factor(s) ” are energy, particle size and thickness. But when I get output, there are no calculated F values or significance levels and the following message is displayed: Box’s Test of Equality of Covariance Matrices is not computed because there are fewer than two nonsingular cell covariance matrices.
    What is wrong with my SPSS?
    Best wishes,
    Sandra

  18. Zoey says:

    If I am doing multiple trials, do the trials correspond to the rows (subjects) and the various treatments for the columns? Ex: Testing drift on a sensor with multiple pressures over time and repeating 3 times

    • Charles says:

      Zoey,
      If in your example, the treatments are multiple pressures over time, then the answer is yes.
      Charles

  19. kashif prince says:

    i want to apply statistical tool in which i have 8 treatment groups and five animals in each group and 3 time i have to collect data which statistical tool will fit on my data

    • Charles says:

      If I understand your scenario correctly, you can use mixed ANOVA with one fixed factor (treatment) and one repeated measures factor (time).
      Charles

  20. Sunita says:

    Hi Charles
    I have data for pre test post test 1(30minutes) post test 2 (60minutez).I have three scale.
    I want to find the mean and difference within the group.how can I finf

    • Charles says:

      You can use the Excel function AVERAGE to find the mean. When you say “difference within the group”, do you mean the difference from the mean? This is x – m where x is a data point and m is the mean.
      Charles

  21. Hung Bui says:

    Hi Dr. Charles,
    My name is Hung Bui and doing experiment with chemicals to inhibit the hatching of nematode eggs.
    My experiment was conducted with 5 chemicals with 4 reps. Each rep contains 100 eggs soaked in chemical. I collected number of hatched nematodes in each rep/day. I did a repeated measures ANOVA and I wonder this is a good statistical analysis?
    Please give me more advice.
    Thanks

    • Charles says:

      From your description ANOVA with repeated measures could be the appropriate test, but I would need more details about the scenario to be sure.
      Charles

      • Hung Bui says:

        This is my make-up data (#of hatched nematodes/rep/day)
        Day UTC ChemA ChemB
        1 2, 3, 4, 5 1,2,2,3 1,1,1,1
        2 10,12,13,14 5,6,6,7 1,2,1,2
        .
        .
        .
        15 100,100,100,100 30,30,35,35 5,5,6,7
        I would like to see whether Chem A or B inhibit the hatching of the nematode’s eggs in comparison with control.

  22. Luc says:

    Dear Charles,

    I have two groups of test subjects who both did a certain test 6 times. Group 1 had extra help in the 4 of the tests and the other group had no help at all. The first and last test for group 1 were without help. I cannot figure which ANOVA analysis to apply, none of the examples seem to have different people in 2 groups.

    Any help would be greatly appreciated, thanks for your great website 🙂

    • Charles says:

      I don’t completely understand your question. What hypothesis are you trying to test?

      You can use ordinary ANOVA, i.e. ANOVA with independent groups (not ANOVA with repeated measures) if you have different people in the two groups, but I am not sure this fits with the problem you are trying to solve.

      Charles

      • luc says:

        The idea is that the first test administered to both groups is a “zero” test without help. The last test is a control test without help. test 2-5 are different between the 2 groups as one has help and one does not.

        The hypothesis is: What is the effect of added help on training in these subjects.
        The training is caused by them repeating the test 6 times.

        • Charles says:

          As I understand the problem, you have two factors. Factor A contains two levels, representing the two groups of subjects. These groups are independent. Factor B represents the tests given to the two groups. This factor has 6 levels, and the factor contains repeated measures since the same subjects take all 6 tests. Under this interpretation, you would use a two factor mixed model ANOVA.

          This is the model described on the webpage
          http://www.real-statistics.com/anova-repeated-measures/one-between-subjects-factor-and-one-within-subjects-factor/

          Let me know whether this captures what you are trying to model.

          Charles

          • Luc says:

            Ah! That makes sense! Thanks a lot for your detailed site and answer to my question.

            While working I discovered a possible mistake in the repeated measures:mixed ANOVA plugin for excel.

            The F crit value calculated between subjects; rows has is currently calculated by taking FINV(alpha,df(rows),MS(rows)) I think the correct way would be FINV(alpha,df(rows),error(rows)) .

            Hope this helps you.

            Luc

          • Charles says:

            Luc,
            This mistake was corrected in the latest release of the software (Rel 4.3), and so you just need to upgrade to the newest release.
            In any case, thanks very much for informing me of the error. This helps make the software better for everyone.
            Charles

  23. Manoj Oak says:

    Dear Dr. Charles,

    I would appreciate if you help me to do stability analysis of multi year data, of plant breeding. I am using crop stat 7.2, i did the analysis but interpretation of the results is difficult, as you know.

    with best regards
    Manoj

    • Charles says:

      Manoj,
      Sorry, but I am not familiar with CropStat. I can help you with the Real Statistics software, but not a tool that I have never used.
      Charles

  24. Yi Zhang says:

    Is it possible to do multiple comparison following repeated measures ANOVA with your tool?

    Thanks,
    Yi

  25. Ed says:

    Regarding the comment above left by Brijesh, I am using ANOVA with matched samples – 1 within-subjects factor and 1 between-subjects factor to compare contaminants in fish from three locations over time. I catch varying numbers of fish from these three locations once per year. The fish are sacrificed to yield a mean contaminant concentration for each location for each year. My sample size varies. Am I using the wrong approach to analyze this data set?. Thanks for providing this excellent resource.

    • Charles says:

      Ed,
      If the fixed factor is Location and the repeated factor is Contaminant then you are using the correct test (assuming that each fish is measured for the various contaminants).
      Charles

  26. Jon says:

    I have run the repeated measures anova with contrast (which I believe is the post-hoc function), however this function results in Zeros in the ‘diff’ column and a subsequent mean of zero – therefore the contrast does not yield a result.
    Is there something wrong with my method?

    • Charles says:

      Jon,
      You need to fill in the contrast values on the output. E.g. in Figure 3 of the referenced webpage the contrast values in range B4:E4 are initially blank. You need to fill this range with values that add up to 0. Admittedly this is not so clear from the description. I will try to update the webpage to make this clearer.
      Charles

      • Jon says:

        Many thanks for your reply. You suggest any numbers that add up to 0 , is that correct? Altering these number seem to affect the resultant p-value. However…
        I am interesting in conducting a post-hoc analysis – the contrast option does not appear to complete a post-hoc analysis. Is it possible to complete a post-hoc for repeated measures anova?

        • Charles says:

          Jon,

          Generally you choose non-negative numbers that add up to 1 and negative numbers that add up to -1, for a total of 0. Altering these numbers will definitely affect the resulting p-value. This is to be expected since different combinations of contrast values correspond to different tests. E.g. the test in Figure 3 of Repeated Measures ANOVA the test is the “before treatment” compared with “the average of the after treatment measurements”. If instead you used +1 as the contrast for “before treatment” and -1 as the contrast for “3 weeks after treatment” (with zeros for the other two groups), you would perform a test for before treatment vs 3 weeks after treatment.

          You can find more information about performing post-hoc analyses for repeated mesures anova on the above referenced webpage, namely

          http://www.real-statistics.com/anova-repeated-measures/one-within-subjects-factor/

          Charles

  27. Brijesh says:

    Hello Charles,
    will “ANOVA with Repeated Measures” work correctly for unequal sample sizes of different groups? Is it possible to incorporated unequal group-size in SPSS also?

    Thanks

    Brijesh

    • Charles says:

      Brijesh,

      For repeated measure designs, generally the same subject is measured multiple times (these are the groups). Thus unequal group size can only mean that some of the data is missing, in which case you need to handle missing data as described elsewhere on the website; see especially .

      The situation for ordinary ANOVA is quite different. Here unequal group size normally doesn’t mean missing data but simply that the samples for each of groups may have different sizes.

      Charles

  28. Rory Rosszell says:

    So, just to clarify, am I correct in understanding that the most recent version of Real Statistics can carry out mixed-model ANOVAs? And if so, is it capable of does do in combination with Excel 2007?
    One more question (as a long-term SPSS user), is there a(n easy) way to get Excel to read my SPSS files?
    Thanks for your trouble, Rory

    • Charles says:

      Rory,

      You are correct. The latest release of the Real Statistics Resource Pack handles two factor ANOVA with two fixed factors, two random factors or one fixed and one random factor (mixed). These are available for use with Excel 2007, 2010 and 2013.

      I don’t know of any way for Excel to read SPSS files, but there is a way around this. If you have a copy of SPSS, it should be able to output data in Excel format. If you don’t have access to SPSS, you can use PSPP, which is a free, open-source (limited) version of SPSS. It can read SPSS files and then output the data in Excel format (using a procedure that works, although it is more complicated than it should be). The procedure for doing this is described at http://lists.gnu.org/archive/html/pspp-users/2011-11/msg00033.html

      Charles

      • Rory Rosszell says:

        Charles, Thank you for your clear and helpful response. After searching high and low for an alternative to SPSS, it seems that I have finally found one. I only occasionally need to run such tests, and with the IBM/SPSS prices now somewhere in the stratosphere, finding your website is a godsend. All the best, Rory

  29. William Agurto says:

    Charles:

    The new version of Real Statistics has disabled the option “Two factor Repeated Measures Anova”: when I click that option and press OK button, the macro doesn’t run. All the rest of options seems OK.

    William.

    • Charles says:

      William,
      Thanks for catching this. The “Two factor Repeated Measures Anova” option should have been removed (it was originally on the main menu only for testing purposes). This capability is assessible, however, from the “Analysis of Variance” data analysis tool using the “Repeated Measures Anova – mixed” tool.
      Charles

  30. David says:

    I have done a repeated measures Anova using this software but I cannot get the interaction between the subjects and groups in the Anova table. I am wondering is it possible to calculate the interaction between both using the software? I would also like to thank you for this excellent resource.

    David.

  31. Matt Green says:

    Is the p value provided by excel in the ANOVA single factor (and other ANOVA’s) a 1 or two tailed p value?? SPSS historically only provides the two tail p value with instructions to divide in two if you wanted the 1 tail value.
    thanks in advance
    matt G

    • Charles says:

      Matt,
      Excel’s single factor ANOVA test calculates the p-value based on a one-tailed F-test. The resulting p-value is the same as the output from SPSS, at least based on all the ANOVA examples I have tested in the past. Despite the use of a one-tailed F-test I have often seen this referred to as a two-tailed Anova test.
      Charles

Leave a Reply

Your email address will not be published. Required fields are marked *