📐 Quick Formula Reference
Sigmoid / Logistic Function
g(z) = 1 / (1 + e−z), z(i) = b₀ + b₁x₁(i) + ... + bmxm(i)
Binary Cross-Entropy (Log Loss)
Loss = −∑ [ y(i)·ln(g(z(i))) + (1−y(i))·ln(1−g(z(i))) ]
Gradient (Binary & Multiclass)
∂Loss/∂bm = ∑ (g(z(i)) − y(i)) · xm(i) ← error × feature
Confusion Matrix Metrics
Accuracy = (TP+TN) / (TP+TN+FP+FN)
Precision = TP / (TP+FP), Recall = TP / (TP+FN)
F1 = 2·P·R / (P+R), Specificity = TN / (TN+FP)
ROC Curve Axes
TPR = Recall = TP/(TP+FN), FPR = 1−Specificity = FP/(FP+TN)
Optimum T: maximize (TPR − FPR) [Youden's J]
Softmax (Multiclass)
softmax(z)k = ezk / ∑j ezj, ŷ = argmaxk softmax(z)k
Decision Boundary (2D, T=0.5)
b₀ + b₁·x₁ + b₂·x₂ = 0