Model Evaluation

Confusion Counts

Thresholded Predictions

Classification evaluation compares predicted labels with actual labels. A probability threshold changes those counts.

Program

Play the script to change the threshold and see true-positive and false-positive counts.

confusion_counts.R 
actual <- c("yes", "no", "yes", "yes", "no", "no")
prob <- c(0.90, 0.40, 0.65, 0.55, 0.30, 0.80)
threshold <- 
predicted <- ifelse(prob >= threshold, "yes", "no")
tp <- sum(actual == "yes" & predicted == "yes")
fp <- sum(actual == "no" & predicted == "yes")
label <- paste(tp, fp, sep = "/")
cat(label, "\n", sep = "")

actual <- c("yes", "no", "yes", "yes", "no", "no")
prob <- c(0.90, 0.40, 0.65, 0.55, 0.30, 0.80)
threshold <- 
predicted <- ifelse(prob >= threshold, "yes", "no")
tp <- sum(actual == "yes" & predicted == "yes")
fp <- sum(actual == "no" & predicted == "yes")
label <- paste(tp, fp, sep = "/")
cat(label, "\n", sep = "")

actual <- c("yes", "no", "yes", "yes", "no", "no")
prob <- c(0.90, 0.40, 0.65, 0.55, 0.30, 0.80)
threshold <- 
predicted <- ifelse(prob >= threshold, "yes", "no")
tp <- sum(actual == "yes" & predicted == "yes")
fp <- sum(actual == "no" & predicted == "yes")
label <- paste(tp, fp, sep = "/")
cat(label, "\n", sep = "")
threshold `prob >= threshold` turns probabilities into predicted labels.
true positive `tp` counts rows where actual and predicted labels are both `yes`.
false positive `fp` counts `no` rows predicted as `yes`.