Basic Concepts
We can use the ICC(1,1) model to compare two measurements of the same variable where one is considered to be the “gold standard”. This is especially important if you are trying to introduce a new measurement capability that has some advantages (e.g. it is less expensive or safer to use) over an existing measurement technique).
We explore this issue in more detail in Bland-Altman Analysis and Lin’s Concordance Correlation Coefficient.
Example
Example 1: A nuclear power plant has been using a fairly expensive method (Old) for measuring the strength of the rods in the nuclear reactor. They would like to implement a more cost-effective method (New), but first want to make sure that there is agreement between the measurements done by these two methods. They do this by taking the measurements of 20 rods, as shown in columns A and B of Figure 1.
Figure 1 – ICC(1,1) calculation
We calculate the ICC(1,1) statistic for this data by pressing Ctrl-m and selecting the Interrater Reliability data analysis tool (from the Corr tab if using the multipage user interface). Fill in the dialog box that appears as shown in Figure 2.
Figure 2 – Interrater Reliability dialog box
Upon clicking on the OK button, output similar to that shown on the right side of Figure 1 will appear. You will need to manually change the value in cell E6 from 2 to 1, at which point the output will be as shown in Figure 1. The ICC(1,1) value is .93 (cell E10), which is a reasonable level of reliability.