# Kuder and Richardson Formula 20

The Kuder and Richardson Formula 20 test checks the internal consistency of measurements with dichotomous choices. It is equivalent to performing the split half methodology on all combinations of questions and is applicable when each question is either right or wrong. A correct question scores 1 and an incorrect question scores 0. The test statistic is

where

k = number of questions

pj = number of people in the sample who answered question j correctly

qj = number of people in the sample who didn’t answer question j correctly

σ2 = variance of the total scores of all the people taking the test = VARP(R1) where R1 = array containing the total scores of all the people taking the test.

Values range from 0 to 1. A high value indicates reliability, while too high a value (in excess of .90) indicates a homogeneous test.

Example 1: A questionnaire with 11 questions is administered to 12 students. The results are listed in the upper portion of Figure 1. Determine the reliability of the questionnaire using Kuder and Richardson Formula 20.

Figure 1 – Kuder and Richardson Formula 20 for Example 1

The values of p in row 18 are the percentage of students who answered that question correctly – e.g. the formula in cell B18 is =B16/COUNT(B4:B15). The values of q in row 19 are the percentage of students who answered that question incorrectly – e.g. the formula in cell B19 is =1–B18. The values of pq are simply the product of the p and q values, with the sum given in cell M20.

We can calculate ρKR20 as described in Figure 2.

Figure 2 – Key formulas for worksheet in Figure 1

The value ρKR20 = 0.738 shows that the test has high reliability.

Real Statistics Function: The Real Statistics Resource Pack contains the following supplemental function:

KUDER(R1) = KR20 coefficient for the data in range R1.

Observation: For Example 1, KUDER(B4:L15) = .738.

Observation: Where the questions in a test all have approximately the same difficulty (i.e. the mean score of each question is approximately equal to the mean score of all the questions), then a simplified version of Kuder and Richardson Formula 20 is Kuder and Richardson Formula 21, defined as follows:

where μ is the population mean score (obviously approximated by the observed mean score).

For Example 1,  μ = 69/12 = 5.75, and so

Note that ρKR21 typically underestimates the reliability of a test compared to ρKR20 .

### 243 Responses to Kuder and Richardson Formula 20

1. Mike says:

Thank you for this. It will be a great help if you provide the Excel file in the example as well.

2. Tycho says:

if pj = number of people in the sample who answered question j “correctly”
Would this be applicable to test with likert scale as scoring too? Since there will be no correct answers if this would be the case.
Thank you 😀

• Charles says:

Tycho,
As you correctly point out, with Likert scoring there is no correct answer. To calculate Cronbach’s alpha in this case, you use the actual Likert scores. Yesterday I added an example of how to do this in Example 4 on webpage http://www.real-statistics.com/reliability/cronbachs-alpha/. Take a look at the example and let me know if this addresses your question.
Charles

3. abbey says:

Hi!

k 7
Epq 0.888888889
var 2.888888889
P 0.807692308

Does this mean that we have high reliability? thank you!

• Charles says:

Abbey,
Yes, a p-value of .80769 would normally be considered to indicate high reliability.
Charles

4. jeeha says:

under what condtions kudar richardson formula is applied and what are its different forms?

• Charles says:

Dear Jeeha,
This test is used when there two choices for each answer: correct and incorrect. It can be applied for example for multiple choice tests even with say 4 choices. Here the wrong answers count as 0 and the right answer counts as 1. If you have non-dichotomous questions (e.g. with a Likert scale) you should use Cronbach’s alpha instead.
Charles

• kevin says:

Kindly explain to me why KR-20 is special than KR-21. Under what circumstances should one use either

• Charles says:

Kevin,
You should always use KR-20. KR-21 was created in the era prior to computers since it was easier to calculate. Since KR-20 is more accurate you should use KR-20.
Charles

5. Tatiana says:

Hello, Thank you very much for providing this example.
Above you note that:
A high value indicates reliability, while too high a value (in excess of .90) indicates a homogeneous test.

What do you mean by ‘homogeneous test’ is this a problem? If you are aware of a source explaining this issue, could you please provide a reference?
Thank you.

• Charles says:

Tatiana,
By homogeneous test I am referring to tests which are likely testing the same thing. E.g. if I design a questionnaire with two questions whose sample data has a correlation of .98, it may be presumed that I don’t need to retain both questions since both are pretty much testing the same thing. In this case I might be justified in dropping one of the two questions from the questionnaire. The Cronbach’s Alpha page of the website (http://www.real-statistics.com/reliability/cronbachs-alpha/) also provides some information about this issue.
Charles

hi, there could i ask about the different spesific usage between: Kr20 & Kr21

• Charles says:

I have now updated the Kuder and Richardson Formula 20 webpage to also show how to calculate KR21 and when it can be used. Please look at that page for help in answering your question. Frankly KR21 is simply an approximation of KR20, and so it is of limited use.
Charles

7. Helen says:

Can you direct me on the use of LDC? Apparently it is used when the KR20 is low (i.e. below .80). I would like a reference to read, some understanding of the assumptions upon which it is based, and why it is used.
Thanks

• Charles says:

Helen,
I just sent you an email response, but it bounced back to me with an error message.
Charles

8. Christine says:

Thank you so much for the information. That is a great help!

9. zuraidah nordin says:

can i use KR20 for ‘fill in the blank questionnaire’, which could be given either right or wrong answer.

• Charles says:

Zuraidah,
I believe so, as long as there as a clear criteria for right or wrong.
Charles

10. Robert says:

Good day, just wanna ask, at what value of KR 20 would we start to interpret as having low or no correlation? Is there a value that we can say this items have low, moderate or high correlation?

• Charles says:

Hello Robert,

I have usually seen that a value of at least.70 is desired for most exams. The following are the criteria used by Imran Zafar in http://com.ksau-hs.edu.sa/eng/images/DME_Fact_Sheets/fs_24.doc

Charles

Reliability Interpretation
.90 and above Excellent reliability; at the level of the best standardized tests
.80 – .90 Very good for a classroom test
.70 – .80 Good for a classroom test; in the range of most. There are probably a few items which could be improved.
.60 – .70 Somewhat low. This test needs to be supplemented by other measures (e.g., more tests) to determine grades. There are probably some items which could be improved.
.50 – .60 Suggests need for revision of test, unless it is quite short (ten or fewer items). The test definitely needs to be supplemented by other measures (e.g., more tests) for grading.
.50 or below Questionable reliability. This test should not contribute heavily to the course grade, and it needs revision.

• Charles says:
11. farida shullai says:

Hi, can I usd KR20 for calculate a test which is someehat like a likert scale.? If yes how do I go about it.

• Charles says:

Hi, if you have scores which are in Likert scale (or somewhat similar) I suggest that you explore using Cronbach’s alpha instead of KR20 (which is designed for correct vs incorrect only). You can get further info at http://www.real-statistics.com/reliability/cronbachs-alpha/.
Charles

12. azlina says:

evening dr charles, thank you for your great guidance on KR20. it helps.
But, I would like to ask, to test realibility, there is another test on Pearson correlation to test on exam scores reability. Which is the better test ? -thank you

• Charles says:

Azlina,
I don’t completely understand your question, but Cronbach’s alpha is often used instead of KR20 to test releiability.
Charles

13. Salima says:

hi, thanks, the information was worthy.
What if I want to test the reliability of a self-designed questionnaire to test participant’s knowledge and attitude on some theme/subject like hepatitis, questionnaire has multiple choices however no choice is right.

• Charles says:

Salima,
Charles

14. Raphaela says:

Hello Charles,
How could I know if the coeficient KR20 is high or low? could you send me a reference from any author?
thanks

• Charles says:

Hello Raphaela,

There doesn’t seem to be a common view on this, but I most frequently see a cutoff of .6 or .7. The following call for a cutoff of .60::

Charles

• Mary says:

Dear Charles,
I have calculated my KR 21 of my pretest and post test in excel. I found the results are o.3 and 0.5. How do I know if my tests are reliable or not?

• Charles says:

Mary,
These are not generally viewed to be very high figures, and so there is doubt about the reliability.
Although the values probably won’tbe that different, I suggest that you use KR20 (or Cronbach’s alpha) which are more accurate.
Charles

15. judy says:

good evening.. just wana ask ..how did you get the answer in the example shown above , where VAR is equal to 6.52083.. please. thankyou so much 🙂 🙂

• Charles says:

Judy,
The answer is shown in Figure 2 of the referenced webpage. I used the VARP function.
Charles

i just want to ask how to calculate the varices just give me the example

17. Jarvis says:

Thanks Charles for this information. I do have a question; is it possible to calculate the KR20 using the split-half method? When I attempt, I’m always getting a number higher than 0-1 ..i.e… 1.9, etc.

• Charles says:

Jarvis,
I don’t understand what it means to calculate KR20 using the split-half method or why you would want to do so. If you calculate KR20 as described on the referenced webpage you should get a value no higher than 1.
Charles

18. Umut says:

Dear Dr. Charles,
First of all I would like to express my gratitude for your precious efforts. This page is a gold mine for people like me.

I wish to use Kuder and Richardson Formula for a number of tests for language learning. Do you think I should use any other formulas or would KR-20 fit for my purposes.

Here are my tests:

Test A: Translation
1- DOG: …………….. (Write the meaning in your first language)

Test B: Multiple Choice
Bow Wow – What animal is this?
a) CAT b)DOG c)…… etc.

Test C: Fill in the blank
A …………. barks.

• Charles says:

KR20 only supports dichotomous questions (e.g. 1 for correct answer and 0 for incorrect answer). This seems to be the case with the three questions you are posing, and so KR20 could be used. You can also use Cronbach’s Alpha.

Whether KR20 (or Cronbach’s Alpha) is a fit for your purposes depends of course on what you are trying to do (which is not stated in your comment).

Charles

• Umut says:

Thank you Charles, I really appreciate your response. I aim to measure learners existing vocabulary knowledge on some technical engineering vocabulary the above examples have been given to show you my test formats.

19. Mark Campbell says:

We gave the exact same 100 question exam to students in consecutive classes (1000 students in both 2013 and 2014). The calculated KR21 values for each class were equal to within 3 digits (0.8763 vs 0.8765). I assume this tells me something about the student populations. Does does it mean?

• Charles says:

Mark,
The results seem to show high internal consistency of the exam. If the 1,000 students are independent then the similar KR21 scores is not surprising.
Charles

20. Mark Campbell says:

Question should read “What does it mean?”

21. Tony says:

Charles, I was wondering if you could look at a screen-shot I have provided and help explain the results to me. I have many different content areas to work on, but in the following example, I have a 10 question dichotomous exam which was given to about 11,300 students. I am using all the results from these exams to find p, which, is quite low. What I do not understand, is that the higher my variance is, the better my p value, or, the lower my Σpq the higher my p.

KR-20 Example

Obviously, I have a bias; and am sure that our questions are reliable, but, the numbers appear to say otherwise. The only other thing I can think of, is that our exam is random. So while we have 10 questions per exam, Q1 may be different than the next student’s Q1, although the question is designed to measure the same knoweldge of the given topic. Do I need to use this formula with a static exam? Where Q1 is the exact same question for all student’s?

Your assistance is much appreciated, Thanks

• Charles says:

Tony,
KR20 and Cronbach’s Alpha assume that Q1 is the same for all the students. If this is not the case, then I wouldn’t rely on the KR20 or Cronbach’s alpha measures.
Charles

• Tony says:

Thank you Charles. Due to the fact, that we provide randomization w/ our questions, would you be able to provide a plausible approach for measuring the reliability/consistency of our examinations? When we run a t-test; our results indicate a high probability that our results correlate. Aside from Pearson’s R/Spearman (which can change based on how we collect/sort data) is there another method we can use to prove internal consistency?

• Charles says:

Sorry Tony, but I don’t know of an approach in this case.
Charles

22. Lizzy says:

Thanks for your explanation on KR20. Please I calculated the reliability of my research instrument and I got KR20 = 0.65. Can I call this a high reliability? If no, what should be done to the instrument.

• Charles says:

Lizzy,

There isn’t complete agreement as to what constitutes “high reliability”, but here are my observations about Cronbach’s alpha. These apply to KR20 as well (since the index produced are identical):

A commonly-accepted rule of thumb is that an alpha of 0.7 (some say 0.6) indicates acceptable reliability, and 0.8 or higher indicates good reliability. Very high reliability (0.95 or higher) is not necessarily desirable, as this indicates that the items may be entirely redundant. These are only guidelines and the actual value of Cronbach’s alpha will depend on many things. E.g. as the number of items increases, Cronbach’s alpha tends to increase too even without any increase in internal consistency.

Charles

23. Godspower says:

Pls i stil don’t understand how the variance was calculated. i.e B24

• Charles says:

This is already shown in Figure 2.
Charles

24. Godspower says:

Pls Charles, can you break it down for me, pls, if possible use figure frm the table.
How did you work VARP(M4:M15)?

25. vangie says:

Hello sir. Thank you for the above info. Very helpful indeed. Please help me how you came up with variance or 02 of 6.52083. I just cant figure it out. Thank you sir!

• Charles says:

The formula to calculate 6.52083 is shown in Figure 2. I use the VARP function for all the variances. If I had used VAR I would get the same result.
Charles

26. Justin says:

What do I do when I only have a single question which is like this?

• Charles says:

Justin,
Cronbach’s alpha can be used even when the questions take a value 1 (say for correct) and 0 (say for incorrect). This would cover True/False and multiple choice questions. The problem is that you can’t use Cronbach’s alpha with only one question. After all, Cronbach’s alpha measures internal consistency, but withonly one question there is no internal consistency to measure. The same is true for Kuder and Richardson.
Charles

Hello Dear Charles,

Kindly give reference of this

• Charles says:

There are lots of references for the KR20 test. Most statistic textbooks as well as many websites on the Internet. The original paper is

Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160.

Charles

27. erie says:

is KR-20 just for multiple choice question?

• Charles says:

KR-20 is not just for multiple choice questions, but it is just for questions which take only two values (e.g. 1 = correct, 0 = incorrect). This includes True/False questions as well.
Charles

Hello Dear Charles,

I would like to ask you that can we calculate KR 20 by using SPSS?kind provide some reference.
Thanks

• Charles says:

Probably so, but I don’t use SPSS.
Charles

29. Yijun says:

Hi,
I’ve tried your formula, but it turns out the KR-20 can be negative if I randomly generate students’ correctness. Can you explain why? According to Wiki, KR-20 range from 0-1, my data is as follow:
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11
1 0 0 0 1 0 0 1 1 0 0 0 3
2 0 1 1 0 0 0 1 1 0 0 1 5
3 0 0 1 1 0 0 0 1 1 1 1 6
4 1 1 0 0 0 0 1 1 1 1 0 6
5 1 0 1 1 0 0 0 1 0 0 1 5
6 1 0 0 1 0 1 0 0 1 0 1 5
7 1 1 1 1 0 1 0 0 0 0 1 6
8 0 0 0 0 1 1 1 0 0 0 0 3
9 1 1 1 1 0 0 0 0 0 0 0 4
10 1 0 0 1 0 1 0 0 1 0 0 4
11 1 0 1 1 1 1 0 0 0 0 1 6
12 0 0 0 0 1 0 1 0 0 1 1 4
Total 7 4 6 8 3 5 5 5 4 3 7 57

• Charles says:

The simple answer is that KR-20 can indeed be negative. This is not so common in practice but it can happen.
Charles

30. Josef says:

100 Schüler
14 Fragen
1123 positive Antworten
sd² = 2,1587
KR-21 = -0,0316

Wie interpretiert man negatives Ergebnis von KR-21?
Danke

• Charles says:

Josef,
Negative value for KR-21, KR-20 and Cronbach’s Alpha can happen, although in practice they are not so common. These are generally signs that reliability is very very poor.
Charles

i got the result of my KR21 is 0.983
what does it mean?
do you have a list to show whether the score has high/medium/low reliability?
it would be very helpful..thank you, Charles.

• Charles says:

The result indicates extremely high reliability (almost the maximum value), but this is not necessarily good since a value this high indicates homogeneity of questions (i.e. there isn’t much difference between the questions).
Charles

32. Grace says:

Hi Sir Charles,

1. What reliability test can be used for essay type of exam?
2. If the set of exam is mixed of different types of test (e.g. I. Multiple Choice, II. True or False, III. Fill in the blanks, IV. Enumeration, V. Essay), will it be ok to still use the KR-20? How will it be done, will it be per type of exam or will it be as total of all question? If not, what reliability test is more appropriate to use?
3. For enumeration type of exam, how will the KR-20 be done? (e.g. Q1 enumerate 5 things, Q2 enumerate 3 things, Q3 enumerate 4 things) Will k = 3 or k = 12?

• Charles says:

Grace,
1. You need to be able to assign a score of 1 for correct and 0 for incorrect you can use KR20 or any of the other commonly used reliability tests.
2. As long as all the questions are testing the same thing and can be coded 0 or 1 as described above, you can use KR20. If you have 20 questions that test one thing and 20 questions that test something else, then you should create two separate KR20 measurements
3. It depends on how you code correct vs. incorrect. If for example, a person is given 5 words to recall and you define “correct” as they recall all five (or 4 out of 5), then this counts as one question. If they get one point for each word they correctly recall then this counts as 5 questions. In this latter case, you have the further problem that the order of the words will be relevant, plus you have the issue of how to score a response of a word that wasn’t even on the original list.
Charles

33. charl arl says:

hi sir charles.

if i am going to solve it on paper what formula should i use for KR20? because there is so many formula on net and i dont know what to use.. i just need a simple example and process of solving using KR20 for my report, tommorow. and kindly give me another example because your sample is easier to understand than other source on net.. hope that you can help me.. thanks a lot.. God bless.

• Charles says:

Hi, the example on the referenced webpage provides all the information you need to calculate KR20. You can also download the Excel spreadsheet for this example from Examples Workbooks so that you can see all the formulas better.
Charles

34. ayu says:

hello…
I have exercises that consist of multiple choice, true-false question and short answear to analyze the scores of this test which is better split-half question or K-20, I tried both I used of split-half Test odd-even and the result unreliable, later I tried to K-20 results are reliable, how come this happened?

test instruments is valid valid but unreliable , why this condition can occur?

pls help me with this..thankyou

• Charles says:

It is not too surprising in general that two tests would give different results. This is especially true with small samples.
If you send me an Excel file with your data I can try to understand why you are getting such different results in this case. You can find my email address at Contact Us.
Charles

• Ayu says:

Thankyou fr yr response,
I already sent you my excel file to your contact, pls be considerable to check it

• Charles says:

I have now looked at the spreadsheet that you sent me. The value I calculated for KR20 = .0677, which agrees with the value you calculated. I calculated a value of -.052 for split-half (odd-even). These values are both very low, indicating unreliability.
Charles

35. joseph says:

Hello Charles,
You are doing a good job. Keep it up. God bless you.

36. applefabela says:

Sir Charle
what exact meaning about kuder richrd formula 20 thanks

• Charles says:

It is a measure of internal consistency of measurements with dichotomous choices. It produces the exact same result as Cronbach’s alpha when all the choices are dichotomous. See Cronbach’s alpha for more information.
Charles

37. Marc says:

Dear Charles,
Thanks a million for these excellent explanations. I conducted an online poll for my MBA Thesis. Now I tried to compute the Cronbach’s Alpha. As it turned out the variance in my case is 0 (cell B21 in your example) because it was not possible to give a wrong answer, e.g. all participants answered all questions.
As it is not possible to divide through 0 I get an invalid result for the Alpha. My suggestion is to state that the questionnaire provides a perfect reliability. What would be the best way to interpret this fact or did I misunderstand something?
Marc

• Charles says:

Marc,
If there aren’t any wrong answers, it is not clear to me why you even want to use Cronbach’s alpha. I guess in this case you could consider the questionnaire reliable.
Charles

38. Luke says:

Hi Charles,
Just wondering! Which of the methods, Cronbach’s alpha or KR20, is better to investigate the reliability of dichotomous questions?

Luke

• Charles says:

Luke,
They should give identical results.
Charles

39. Luke says:

Many thanks Charles.

40. Penny Villars says:

Why does exam software (like Examsoft) report KR20 on multiple choice exams if the assumptions for the statistic are dichotomous answer choices? Most Examsoft items have at least 4 choices with only 1 correct answer. Is this a relevant statistic in this scenario?

• Charles says:

The coding is not based on the 4 choices but on 1 if the answer is correct or 0 if the answer is incorrect (this is dichotomous).
Charles

41. KIm says:

Hi everyone,
I’m doing master of Tesol & thesis now. I got a problem that need your advices.
The answers of questionnaire are wrong or right with 68 participants? Should I use KR20 or Cronbach’ Alpha to do reliability test for pilot study ?
Thanks so much

• Charles says:

Hi Kim,
Yes, you can use KR20 and Cronbach’s Alpha to measure reliability. If the coding you use is 0 for wrong and 1 for right, KR20 and Cronbach’s Alpha will yield the exact same answer.
Charles

42. catherine says:

I will like to know the conditions that calls for the use of Kuder Richardson 21 and 22 fomular and also Crombac bach Alpha.

• Charles says:

Sorry, but I don’t understand what you mean by “the conditions that call for the use of”.
Charles

• catherine says:

simply put, what is the difference between Kuder Richardson 21 and 22 formular.

• Charles says:

Catherine,
I assume that you mean “what is the difference between Kuder Richardson 21 and 20 formulas?” The KR21 formula is a simplified version of the KR20 formula, which was useful in the days before computers. There is no reason that I can think of for using the KR21 formula. You should use KR20 instead.
Charles

43. KIm says:

Inter-Item Correlation Matrix
V2 V4 V5 V6 V9 V10 V12
V2 1.000 1.000 -.049 .386 .702 -.021 .702
V4 1.000 1.000 -.049 .386 .702 -.021 .702
V5 -.049 -.049 1.000 .214 -.034 .434 -.034
V6 .386 .386 .214 1 .000 .569 -.026 .569
V9 .702 .702 -.034 .569 1.000 -.015 1.000
V10 -.021 -.021 .434 -.026 -.015 1.000 -.015
V12 .702 .702 -.034 .569 1.000 -.015 1.000

Cronbach’s Alpha is 0.723 now , there are 7 items chosen totally now as well ( after running KR20 with 16 items at the first time )
Because item V5 & V10 still have negative value. However, I do want to use it as the final result , can’t I? If not, may u explain why not .
( the reason is I have to make sure the No of these items in this test are equal to the no of items in another one, then I will do correlation for language transfer between 2 Vietnamese & English)

Thanks so much !
res

• Charles says:

Sorry Kim, but I don’t completely understand the data or your question.
Charles

44. Kim says:

Good morning Charles,
I did something wrong with my data. Sorry.
May I know how can I explain with KR 20 ?
For instance,
I will use Cronbach’s alpha based on standardized items to explain instead of Cronbach’s Alpha , right ?
The way I will explain when I use Cronbach’s alpha based on standardized items is similar to Cronbach’s Alpha or not? If not, how ?
Thank you

• Charles says:

Kim,
I don’t know whether did something wrong or not with your data. I simply did not have time to try to interpret the data that you sent me.
Sorry, but I also don’t understand your questions. I receive so many questions from so many people, and don’t really have the time to answer unless the questions are clearly worded.
Charles

45. Julie says:

Is the KR-20 appropriate for scales with only 2 or 3 dichotomous items?

• Charles says:

Julie,
KR20 is only viable for dichotomous items (i.e. 2 choices only). In this case it is equivalent to Cronbach’s alpha.
Charles

Hello Good Day!

Can you give me 1 example of KR Formula 20 and explanation because I don’t have any idea about this. This will be my report in class. Thanks!

• Charles says:

This is already provided on the referenced webpage.
Charles

47. Dung says:

Dear Charles, reliability testing for multiple choice questionnaire (a question of one or two or three correct answers) can be used Kuder and Richardson Formula 21? Thanks.

• Charles says:

Dear Dung,
Code each question with 1 if the person answers the question correctly and 0 if the person answers the question incorrectly.
Charles

• Dung says:

Yes, so the number of patient for reliability testing will increase? For Kuder and Richardson Formula, how many times does the minimum number of patients than the question? Thanks.

• Charles says:

Dung,
Sorry, but I don’t understand your first question. To get an idea about the minimum number of subjects required, see the following webpage:
Sample Size for Cronbach’s Alpha
Note that Kruder-Richardson is just a special case of Cronbach’s Alpha.
Charles

48. Aira says:

Dear Prof. Charles,

Regards,
Aira

• Charles says:

Aira,
If the coding for the open-ended and closed-ended questions are the same, then you could use KR20 on all 6 questions. Otherwise you need to calculate two values for KR20: one for open-ended questions and another one for closed-ended questions.
Charles

49. Tupennrose says:

How do we correlate two tests? Our test was about level of affection. And we surfed the net to search for questionnaires that are almost the same with our questionnaire which is the level of affection. So how do we correlate this test? How do we know if are questionnaire is reliable and does have a correlation with the other test? What formula should we use? Our statistics teacher told us to use Kuder and Richardson but I don’t know how to.

• Charles says:

I assume that when you say you want to correlate two tests you mean correlate the data. You can use the CORREL function for this.
You can measure reliability using Kuder and Richardson. This is explained on the referenced webpage, assuming that you have the appropriate data.
Charles

50. deborah says:

Pls on wat conditions can we use the kuder and Richardson formulae 20

51. susa says:

Hello, need help
Dry run test
Epq =3.62
var =6.46
KR20=.45 what does it mean?

Final test
Epq =7.04
var =16.19
KR20 = .58what does it mean?

it was a 30-item test with 80 respondent?

• Charles says:

Susa,
Clearly the final test had a higher level of reliability than the dry run. Most would say that the reliability the dry run test was not that high, while that if the final test was at best borderline.
Charles

• Charles says:

Susa,
Clearly the final test had a higher level of reliability than the dry run. Most would say that the reliability the dry run test was not that high, while that of the final test was at best borderline.
Charles

52. Mamta says:

Sir i am doing PhD and for this i am using self made tool and for this i have to do the survey about the perception of teachers and i am using Likert scale (5 point) to know their views and opinions… Can i use KR20 and Cronbach’s alpha . pls suggest how to calculate reliability if the data is based on perceptions not actually on true or false answers.

• Charles says:

Mamta,
KR20 is not appropriate since it only works with dichotomous data, not 5 point Likert scales. You can use Cronbach’s alpha, as described on the webpage Cronbach’s Alpha.
Charles

• mamta says:

Thanks Prof. Charles

53. Ei Ei Moe says:

reliability for research instrument that include 25 items is 0.81 (both kuder 20 and cronbach alpha). but 3 items were excluded from the scale for testing inter-item correlation by testing reliability by alpha. this instrument can be used to conduct research.

54. Gin says:

Good pm. I would like to know which scores should I use in getting the variance of which scores? since we have the pH and pL, right? or should I get the var of all the test scores?
Thanks a lot!

• Charles says:

Gin,
I don’t understand your question. There is no pH and pL on the referenced webpage. Please look at Example 1 on the referenced webpage; it should explain how to calculate the KR20 statistic.
Charles

55. Charles says:

Hello Prof,

I would like to know which methodology is appropriate for a meta analysis data reliability for more than 2 variables?

• Charles says:

It depends on the nature of the meta analysis data and the type of reliability. Could be Kuder and Richardson Formula 20.
Charles

• Charles says:

The meta analysis is measuring the strength wire based on different material properties and sizes.

56. Azin says:

Dear Charles
Generally, What is the acceptable value for the Kuder and Richardson?

• Charles says:

Azin,
There isn’t agreement on what acceptable values are, but you can find guidelines on the following webpage:
Cronbach’s Alpha.
Keep in mind that Kuder-Richardson is just a special case of Cronbach’s Alpha.
Charles

• Azin says:

Dear charles
Thank you for answer, so we can calculate Kuder and Richardson with SPSS Such as Cronbach’s Alpha?

• Charles says:

Azin,
You should be able to use SPSS’s Cronbach’s Alpha to calculate Kuder and Richardson.
Charles

57. Alias says:

Dear Charles,
How about the items not dichotomous types (open ended question)? For example, full marks for the item is 3 and some students get 1 0r 2 or 3 marks. How can we find the reliability??

• Charles says:

Alias,
You can use Cronbach’s Alpha. See example using Likert scale on the following webpage.
Cronbach’s Alpha
Charles

58. may thet says:

I can’t understand to calculate true and false question in research for validity and reliability. Please answer to me.

• Charles says:

Kuder and Richardson 20 can be used for reliability in such cases.
Charles

59. Sandesh says:

Hi,

How do we improve the reliability of a test.

For Ex. if a test has a reliability of 0.5.How do we improve this
1.Should we remove items which are answered by all candidates correctly
2.Should we remove items which are not answered by any candidate correctly

• Charles says:

Hi Sandesh,

In general neither of these approaches will do the trick.

More important is to make sure that the questions are all measuring the same concept. If the questions are measuring different concepts then calculate separate KR20 measures for each such groups of questions.

Also note that KR20 is equivalent to Cronbach’s Alpha (for dichotomous questions: correct/incorrect). If you look at the Cronbach’s Alpha webpage you will see that you can evaluate Cronbach’s Alpha if any one question is dropped. This might help you. You should also be careful about poorly worded questions and reverse coded questions (as described on that webpage).

Charles

• Sandesh says:

I completely understand that the items should be worded well. I was more interested to understand how do we manage the reliability at an item level.

For Ex: If an item is being answered correctly by all the candidates , this means the difficulty level of this item is easy. Is there a range of values which can be assigned to each item of different difficulty level(easy,medium,difficult), which will help in managing the overall reliability of the assessment

• Charles says:

Sandesh,
It is likely that this is true in the extreme cases (e.g. when almost everyone answer some question correctly or incorrectly), but I doubt that it is true in general. In any case, you can test this yourself by seeing whether KR20 increases when removing such questions.
Charles

• Sandesh says:

Thanks Charles

60. Rowland says:

sir, when is it necessary to apply kuder richardson 20 reliability instrument test and cronbach alpha in test reliability estimate

• Charles says:

Rowland,
These are used to measure the internal consistency of a questionnaire.
Charles

61. Kaypat says:

Hi Prof. Charles,
I have four questions
(1). Find the difference between KR20 and KR21
(2). At what point do we use KR20?
(3). At what point do we use KR21?
(4). At what point do we use both?

• Charles says:

Kaypat,
KR21 is defined at the bottom of the referenced webpage. It was invented as a measure similar to KR20 but easier to calculate by hand. Given that we now have computers you should never use KR21 and use KR20 instead.
Charles

• Kaypat says:

Thanks Charles for your quick response. Please I need you to demystify the question one by one. The explanation you gave is not detailed and not very clear to me. Thanks in anticipation.

• Charles says:

Kaypat,
I think that if you will review the information that I have provided you will be able to figure it out.
Charles

• joel says:

Solve it by your self dude…

62. Nazmi says:

Hi Prof Charles,

KR20 can indeed generate negative values, but is it possible to have a negative value of more than 1?
Thank you.

• Charles says:

Nazmi,
Once the value is negative, you know that you don’t have internal consistency, and so I have tried to figure out what the lowest negative value could be.
Charles

• Nazmi says:

Yes, negative value indicates that the instrument has a very poor reliability. So, are you saying that it is possible to get a value of >-1? Thank you.

• Charles says:

Nazmi,
What I am saying is that it is possible that value can be negative. I haven’t spend the time to check whether it can be less than -1.
Charles

• Nazmi says:

Noted Prof Charles, thank you

63. Ganesan says:

FIG1 the PQ values how they calculated because both adding coming wrong abswer

• Charles says:

Ganesan,
pq is just p multiplied by q. E.g. the formula in cell B20 is =B18*B19.
Charles

64. Ganesan says:

fig2 B25 final step how to calculate kindly explain

• Charles says:

The formula is given on the referenced webpage.
Charles

65. stella says:

can l use kuder and Richardson to find reliability of a three option multiple choice quetionnaire

• Charles says:

Stella,
Yes, provided you use the coding 1 for a correct answer and 0 for an incorrect answer.
Charles

66. cherryl says:

SIR, how can i get the VARP?

• Charles says:

Cherryl,
VARP is a standard Excel function. You can use VAR.P instead (it is equivalent).
Charles

67. Dila says:

Thank you very much sir, this helped me so much.

68. Dila says:

Sir, what do you mean by a ‘homogeneous test’?

• Charles says:

Dila,
It means that the questions are too much alike.
Charles

69. J says:

Hi, need help

Can I use KR 20 if my questions are seeking only for the demand of a product? in other words there is no correct or wrong answer.
For example:
what flavors of ice cream do you mostly want to eat?
given answers: vanilla, chocolate, cheese, mango, etc.

If KR 20 is not applicable with this, what other reliability formula can you suggest?
Thank you very much.

70. J says:

Hi, need help

Can I use KR 20 if my questions are seeking only for the demand of a product? in other words there is no correct or wrong answer.
For example:
what toppings would you like to put in your ice cream?
given answers: syrup, sprinkles, marshmallows, etc.

If KR 20 is not applicable with this, what other reliability formula can you suggest?
Thank you very much.

• Charles says:

J,
KR20 only accepts a 0 or 1 answer. There doesn’t need to be a correct/incorrect answer.
If you have more choices, you might be able to use Cronbach’s alpha.
Charles

71. J says:

what if if it is negative?
what shall I do?
thanks

• Charles says:

J,
If there are only a finite number of outcomes, you can always make all of them non-negative (e.g. by adding the absolute value of the smallest negative number). If you really need negative values, then probably KR20 and Cronbach’s alpha are not the right measures to use.
Charles

• J says:

What I mean is the result was negative.
What shall I do?

• Charles says:

J,
KR20 is only useful if you aren’t testing multiple concepts. Also you need to make sure you don’t have any reverse coding issues (see Cronbach’s Alpha webpage for more details about these sorts of issues).
If you don’t have any of these issues, then a negative result shows that the questionnaire is not internally consistent. You need to work on the wording of your questionnaire.
Charles

72. don juan says:

Dear Prof;

I’m a master student. i develop an instrument with dichotomous questions. I check the reliability by using the KR 20. For the validity side, i’m using the expert review. Is it enough for the examiner to accept my instrument during my viva ? or do you have any suggestions so that the examiner will accept my instrument without any doubt. Need your advise.

• Charles says:

It seems like you are covering the bases, but I can’t comment on what any individual might be looking for.
Charles

73. Mang Jose says:

Sir Charles,

Is it OK if I encode 1 as favorable answer and 0 as unfavorable answer?
Because our questionnaire would measure only our respondent’s choice.

Thank You.

• Charles says:

Seems like a reasonable encoding.
Charles

74. Timothy Hyer-Devine says:

I am a Professional counseling student struggling my way through basic statistics. This has been a great help.

75. David says:

Hi Charles,

Is KR-20 suitable for determining the intra-rater reliability of a screening device that just gives a normal/abnormal outcome?

The device is a tuning fork used in screening for loss of vibration perception and is just being applied to 1 site on the body
I am repeating the test 3 times at the same site in the same session
I am encoding the results to normal = 1, abnormal = 0

David

• Charles says:

KR20 is not used for inter-rater reliability.
Charles

76. Yassine says:

Hi Charles,

Please I want to ask, can KR21 BE MORE than 1 ? when calculating using the KR21 I found a score of 1.033 ? is there any mistake of calculation? Thank you for answering me 🙂

Yassine

• Charles says:

Yassine,
No it shouldn-t be more than 1.
Charles

77. Amy says:

Good day prof.
Pls if am using KR21 for a yes and no question . It it possible I use a slipt half reliability or test retest reliability

• Charles says:

Amy,
If you are using KR21, why do you want to use split-half or test-retest? Also, KR20 is more accurate than KR21 and should generally be used instead.
Charles

hi Charles sir. i do have queries. for a 5 point likert scale, is KR21 to be computed better coz KR20 is to be used with 2 point scale only. also i got .42 reliability index when computed by product moment (upper/lower half & odd even) and o.63 reliabilty index when computed by brown prohecy formula. is this value to be considered good for the scale as i learnt from my friends that r value (range) should be between .7 to .9 and below .7 it is considered as low reliability. remember v r dealing with behavioiural science.

• Charles says:

You should always use KS20. KS21 is an estimate of the KS20 value. It was used in the era prior to computers since it was easier to compute manually.
Charles

ks20???i m constructing a scale on school climate(perceived by secondary school students).

• Charles says:

Charles

79. Teresa says:

Hi Sir,

I followed the steps above and got a result of 0.77378 on my pilot test… just wanna know if the result means “okay” as in I can conduct it already or do I still need to improve my test questions i,e, I need to get a result of 0.9? or something….

• Charles says:

Teresa,
There is not universal agreement on what is a sufficiently good result. To me, .77 seems like a pretty acceptable result.
Charles

80. joyce says:

Good day, sir.. if the result is -4.9540, what does it indicate? thank you sir

• Charles says:

Joyce,
This indicates that there was an error in the calculation.
Charles

81. Bismark says:

Prof Charles can u give me the uses of kuder Richardson 20

• Charles says:

Bismark,
See the following webpage about Cronbach’s alpha, which is equivalent to Kuder-Richardson for dichotomous data.
Cronbach’s Alpha
Charles

82. Maribel says:

Hi sr, what if the computed reliability was 1.099

• Charles says:

Maribel,
Reliability should not be larger than 1.
Charles

83. Kat says:

Hi Sir can I ask if the Cronbach’s Alpha is also suited in determining the reliability of the pretest I created. The test is a multiple type of test. Thank you.

• Charles says:

Kat,
Charles

84. dike John says:

hello sir Charles can u help me with a link on how to get a more simpler calculation on reliability of research instrument or simple formula with less complex variable

• Charles says:

The KR21 formula is simpler and it is described on the referenced webpage.
Charles

85. Luis says:

Hello sir, I applied the kr20 formula for 3 groups of student, ten questions test, 30 students each group. The RO was very different on each group. What could be the reason?

• Charles says:

Luis,
I can’t comment without seeing your data. If you send me an Excel file with your data and calculations, I will try to figure out what is going on. You can find my email address at Contact us.
Charles

86. Emmanuel says:

how do you get 6.52802? in your example

• Charles says:

Emmanuel,
The answer is shown in cell B24 of Figure 2.
Charles

87. Enas says:

Dear Charles,
thank you so much for this helpful website.
I have designed a classification system for medical errors. then I designed fictitious cases and I identified the errors and I asked the raters to classify theses errors based on the newly developed classification system. Can I use Kuder and Richardson Formula 20 and consider the correct classifcation=1 and wrong class=0.
and I am wondering about the number of raters or students required to be able using the Kuder and Richardson Formula 20. I meant, are there any assumptions?
On the other hand, can I use Fliess Kappa as well to measure the inter- rater reliability of the same test?

• Enas says:

4-5 raters are sufficient to utilize the KR20?
thanks

• Charles says:

Enas,
It is not clear to me why you would use Kuder Richardson 20.
Fleiss’s Kappa could be appropriate (depending on the details).
Charle

• Enas says:

Dear Charle,
Thanks for your response. Fleiss Kappa might be appropriate for my data to measure the inter-rater reliability since I will have dichotomous nominal variables. But my concern is about measuring the internal consistency of my classification system which has categories and subcategories. I wanted to convert each category into question and if the rater classified it right (1), wrong (0). I meant each category will be matched with one question. After that, I will have dichotomous variables (nominal) which may suite for Fleiss’s kappa and KR-20. I do not know if I can use the same data to measure both types of reliability (inter-rater reliability and internal consistency)
and my second concern about the sample size required to measure the KR-20.
Thanks Charle,

• Charles says:

Enas,
If you are able to view your situation as a questionnaire with many True/False questions where you have the data corresponding to the answers of multiple subjects filling in the questionnaire, then you can use KS20 to measure the internal consistency.
Regarding sample size, first note that KS20 is a special case of Cronbach’s alpha. You can calculate the sample size as described on the following webpage:
http://www.real-statistics.com/reliability/cronbachs-alpha/cronbachs-alpha-continued/
Charles

• Enas says:

Thank you so much, Charles.
Enas

88. Mark says:

Can you discuss why you used the population versus the sample variance

• Charles says:

Mark,
No reason. You can use either and get the same answer.
Charles

• Mark says:

Charles,
When I use VAR(M4:M15) the result is 7.1136 versus VARP(M4:M15) of 6.5028. The KR20 is 0.7682 versus 0.7380 using the sample versus the population variance, respectively. Thanks for letting me know what I’m missing.

• Charles says:

Mark,
You need to use VARP (or VAR.P). You can use VAR, but then you need to make other changes to the formula. In any case, the result should be identical to that obtained for Cronbach’s alpha.
Charles

89. Liza says:

hi

i have read that kr20 and Cronbach’s alpha are mathematically the same. But when I manually computed for kr20, it is way different from the spss result of Cronbach’s Alpha. Please help!

• Charles says:

Liza,
This is true provided the data only takes values 0 and 1. If so, I don’t know why the results from SPSS would be different.
Charles

90. Krizzia Baconguis says:

Hi I just want to ask how did ylu get the variance?

• Charles says:

Krizzia,
I used Excel’s VARP function.
Charles

91. Abdulfatai says:

Sir, I found this very useful for my pilot study.

please is there any excel function to calculate P (B25) as you did to B24.

so I can cross check my calculation.

• Charles says:

It is given in Figure 2 of the referenced webpage.
Charles

92. Rowena Genuino says:

Dear Charles,
Can KR-20 coefficient be also used in testing internal consistency of an atopic dermatitis questionnaire, wherein a patient is diagnosed with the disease if he fulfills the major criterion (itch) and at least 3/5 minor criteria (onset<2 years old, history of flexural rash, dry skin, history of asthma or hay fever, visible flexural rash)?

• Charles says:

Rowena,
I am not familiar with this type of questionnaire, but KR-20 can only be used where each question can only receive a 0 or 1 score (e.g. 0 = wrong answer, 1 = right answer). If this is not the coding that results, you should consider Cronbach’s alpha.
Charles

93. Brian Dates says:

Charles,

I cannot find the Real Statistics module for the KR-20. It is not an option in the internal consistency directory. Where should I be looking, or does it no longer exist, and computations should be done according to your Example 1? Thanks.

• Charles says:

Brian,
You have two choices. (1) Use the KUDER function, as described on the referenced webpage. (2) Use the Cronbach’s Alpha option from the Internal Consistency Reliability data analysis tool. KR-20 is equivalent to Cronbach’s alpha if the data consists only of 0’s and 1’s.
Charles

• Brian Dates says:

Thanks!

94. roberto basas says:

• Charles says:

Roberto,
The referenced webpage explains how to use KR20. You can also use Cronbach’s alpha, which produces KR20 as well.
Do you have a specific question?
Charles

95. vanni says:

helo sir, can i use K20 if my data is not normally distributed ?

Thank you

• Charles says:

Vanni,
KR20 is a measurement. You are not performing any test, and so there is no normality assumption.
Charles

• niah says:

charles,
can i ask for step by step on how to get the standard deviation? pleaseee

• Charles says:

Niah,
The formula for the variance is shown in Figure 2. The standard deviation is the square root of the variance.
Charles

96. Paul says:

Is it possible to have a negative coefficient as a result of computing k-r20?

• Charles says:

Paul,
I believe that it is possible, although rare. It is clearly not a good result.
Charles

97. Robert says:

This is quite interesting, Charles. Thanks a lot
I used that formula and got a result of 0.15 which is too low. However, for my questionnaire (seeking respondents’ perceptions) no answer is right or wrong although I awarded 1 for “yes” and 0 for “no”.
Does it mean that the scale is still unreliable?

• Charles says:

Robert,
If there are no right or wrong answers, then it doesn’t seem that KR20 would be telling you anything meaningful.
Charles

98. Annie Clavon says:

Please help. I used a scrantron machine to scan two versions of the same test. Items equal 50 the KR:20 is .80 on version A, and .13 on version B. Why?

• Charles says:

Annie,
I don’t know what a scantron machine is nor what you mean by scanning two versions of the same test.
Why are you using KR20?
Charles

99. Daniel H says:

Hi I’m working with this KR20 and switching from 15 to 25 items of “similar” type.
Would I just change the 15/14 to 25/24?
as well as multiply the summation I originally had by 25/15?
Doing that is taking me from a KR20 of .6313 down to .3328….

• Charles says:

Daniel,
It doesn’t sound like any of these approaches are correct. Perhaps the approach described on the following webpage would be of assistance to you:
http://www.real-statistics.com/reliability/split-half-methodology/spearman-browns-predicted-reliability/
Charles

• Daniel H says:

I’ve been given this same formula from my professor but they say the n is the constant of proportionality… frustrating.
So in his explanation in going from 5 to 10 items n would be 2.
but in your example you go from 12 items to 24… but in the new formula you use 24 rather than 2 (which would be doubling it).
So i’m getting conflicting things here.

• Daniel H says:

Also under “example 1” on that page suddenly you’re using a formula that is solving for p not p*. Where did that come from? Just another formula?

• Charles says:

Daniel,
Sorry, I don’t know what your professor has in mind.
Charles

100. Lourence says:

• Charles says:

Lourence,
Of what statistic do you want standard deviation in Item Analysis?
Please enter you comments on the following webpage or one of the pages that follows from it:
Item Analysis
Charles

101. Arms says:

Hi! What’s the reliability of this

k 118
pq 9.938271605
var 17.83950617
0.651

• Charles says:

Arms,
What does 0.651 represent?
Charles

102. Kayode says:

What is the reliability of:
k = 10
£pq = 8.6
Var = 17.2
pKR20 = 0.556

• Charles says:

The reliability is 0.556, the pKR20 value.
Charles