ROC curve and Area Under ROC Curve (AUC)

ROC curve stands for Receiver Operating Characteristic curve. It is often used to characterize the trade-off between sensitivity and specificity of quantitative models (typically a binary classifier). An example ROC curve (the blue line) is shown below (from https://www.mathworks.com/):

The x-axis is 1 – specificity, or false positive rate (FPR), calculated as FP/(FP+TN). The y-axis is the sensitivity or true positive rate (TPR), calculated as TP/(TP+FN). Here FP, TN, TP, and FN stand for the number of false positives, true negatives, true positives, and false negatives in a classification (e.g., disease and normal). Each point on the curve (blue line) corresponds to the performance of a specific classifier. For example, in a logistic regression model to predict disease status, given the probability cutoff to decide whether a case is disease or not, the performance of the classifier can be computed by testing it on an empirical data set. The different points on the ROC curve correspond to different classifiers specified by different probability cutoffs.

AUC is the area under a ROC curve (turquoise area in the above figure), which gives an overall evaluation of the models. At a first glance, it seems weird to discuss AUC, because most time we may want to choose the best classifier based on TPR and TPR and shouldn’t care about other classifiers. I agree with this idea. AUC however indicates the robustness of the model — it equals to the probability that the predicted score (the predicted probability in a logistic model) from a positive case is greater than that from a negative case. In another word, it measures how apart the distributions of the predicted scores between positive and negative cases. If the distributions are far from each other, then it will be easy to set a score cutoff to distinguish positive and negative cases. If the distributions largely overlap, it will be hard to distinguish positive and negative cases, no matter which score cutoff is chosen. This is the meaning of AUC.

Above is my thought on ROC curve and AUC after doing some research. I hope that my explanation is accessible to people without much quantitative experience.

Reference: http://mlwiki.org/index.php/ROC_Analysis