TypeScript: precision, recall, and F1 from a confusion matrix

The concept

A binary classifier makes four kinds of outcome. True positives (TP) are positives correctly found, false positives (FP) are negatives wrongly flagged, false negatives (FN) are positives that were missed, and true negatives (TN) are negatives correctly ignored. Together they form the confusion matrix.

Accuracy is (TP + TN) divided by all cases, but it misleads when one class is rare. Precision, TP / (TP + FP), answers how many flagged items were really positive. Recall, TP / (TP + FN), answers how many positives were found. F1 is the harmonic mean of the two and is high only when both are. The interface keeps the four counts together so they cannot be mixed up.

Code example

interface Confusion {
  tp: number;
  fp: number;
  fn: number;
  tn: number;
}

function confusion(actual: number[], predicted: number[]): Confusion {
  const c: Confusion = { tp: 0, fp: 0, fn: 0, tn: 0 };
  for (let i = 0; i < actual.length; i++) {
    if (actual[i] === 1 && predicted[i] === 1) c.tp++;
    else if (actual[i] === 0 && predicted[i] === 1) c.fp++;
    else if (actual[i] === 1 && predicted[i] === 0) c.fn++;
    else c.tn++;
  }
  return c;
}

const actual = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0];
const predicted = [1, 1, 1, 0, 1, 1, 0, 0, 0, 0];
const c = confusion(actual, predicted);

const precision = c.tp / (c.tp + c.fp);
const recall = c.tp / (c.tp + c.fn);
const f1 = (2 * precision * recall) / (precision + recall);
const accuracy = (c.tp + c.tn) / actual.length;

console.log("tp=" + c.tp + " fp=" + c.fp + " fn=" + c.fn + " tn=" + c.tn);
console.log("precision " + precision.toFixed(3) + ", recall " + recall.toFixed(3));
console.log("F1 " + f1.toFixed(3) + ", accuracy " + accuracy.toFixed(3));

tp=3 fp=2 fn=1 tn=4
precision 0.600, recall 0.750
F1 0.667, accuracy 0.700

Follow the code

Open the walkthrough to inspect the example one step at a time.

—

—

—

tp=3 fp=2 fn=1 tn=4

tp=3 fp=2 fn=1 tn=4
precision 0.600, recall 0.750

tp=3 fp=2 fn=1 tn=4
precision 0.600, recall 0.750
F1 0.667, accuracy 0.700

Behind the scenes

Cost and common mistakes

O(n) for n predictions.

• If TP + FP is 0 (nothing predicted positive), precision divides by zero and gives NaN. Handle that case.
• On data with 1% positives, a model that always predicts negative has 99% accuracy and zero recall.
• Which metric matters depends on the cost of each error. A cancer screen favours recall, a spam filter favours precision.

Try it yourself

What are the recall and precision of a model that flags everything as positive?

Further reading

• TypeScript Handbook (external)
• Node.js API documentation (external)
• Mathematics for Machine Learning (external)