Logistic regression for classification
Despite its name, logistic regression is a classification method. It computes a linear score z = w·x + b and passes it through the sigmoid function σ(z) = 1 / (1 + e⁻ᶻ), which maps any number to a value between 0 and 1 that is read as the probability of the positive class. A prediction is positive when that probability is at least 0.5, which happens exactly when z >= 0.
Python
import math
xs = [1, 2, 3, 4, 5, 6]
labels = [0, 0, 0, 1, 1, 1]
def sigmoid(z):
return 1 / (1 + math.exp(-z))
w = b = 0.0
for _ in range(3000):
grad_w = sum((sigmoid(w * x + b) - y) * x for x, y in zip(xs, labels)) / len(xs)
grad_b = sum((sigmoid(w * x + b) - y) for x, y in zip(xs, labels)) / len(xs)
w -= 0.5 * grad_w
b -= 0.5 * grad_b
predictions = [1 if sigmoid(w * x + b) >= 0.5 else 0 for x in xs]
print(predictions == labels)
print(round(sigmoid(w * 2 + b), 3), round(sigmoid(w * 3.5 + b), 3), round(sigmoid(w * 5 + b), 3))
print(round(-b / w, 2))