The ROC Curve is a plot of values of the False Positive Rate (FPR) versus the True Positive Rate (TPR) for all possible cutoff values from 0 t o 1.
Example 1: Create the ROC curve for Example 1 of Comparing Logistic Regression Models.
The first portion of the analysis from Comparing Logistic Regression Models is shown here in Figure 1.
We begin by taking the observed values of success and failures in Logistic Regression summary format together with the calculated p-Pred values (i.e. columns H, I and L from Figure 1) and sorting these by the p-Pred values. This can be done using the SelectCols supplemental function as follows (referring to Figure 1):
The result is shown in columns AI, AJ and AK of Figure 2.
Figure 2 – ROC Table and Curve
Next create the cumulative values for Failure and Success (columns AL and AM) and then the values of FPR and TPR for each row (columns AN and AO). E.g. these entries for row 8 are calculated via the following formulas:
Figure 3 – Selected formulas from Figure 2
The ROC curve can then be created by highlighting the range AN6:AO18 and selecting Insert > Charts|Scatter and adding the chart and axes titles. The result is shown on the right side of Figure 2. The actual ROC curve is a step function with the points shown in the figure.
Observation: The higher the ROC curve the better the fit. In fact the area under the curve (AUC) can be used for this purpose. The closer AUC is to 1 (the maximum value) the better the fit. Values close to .5 show that the model’s ability to discriminate between success and failure is due to chance.
For Example 1, the AUC is simply the sum of the areas of each of the rectangles in the step function. The formula for calculating the area for the rectangle corresponding to the p-Pred value in row 8 (i.e. the formula in cell AP8) is shown in Figure 3. The formula for calculating the AUC for Example 1 (cell AP19) is =SUM(AP6:AP18). The calculated value of .845587 shows a pretty good fit.
Observation: The Real Statistics Logistic Regression data analysis tool automatically creates the ROC curve as described above.