Loss Function

A function that measures how wrong a model's predictions are compared to the true values.

The loss function quantifies the difference between predicted and actual outcomes, providing a signal for the learning algorithm to improve. Different tasks require different loss functions, and the choice significantly impacts model behavior.

Types of Loss Functions

• Mean Squared Error (MSE) - Used for regression tasks, it squares the differences between predictions and actual values. Heavily penalizes large errors.

• Cross-entropy loss - Common for classification tasks, it measures the difference between predicted probability distributions and actual class labels.

• Categorical cross-entropy - Used when classifying into multiple distinct categories.

• Binary cross-entropy - Used for binary classification (yes/no, spam/not spam).