Overfitting

When a model learns the training data too well, including noise, and performs poorly on new data.

Overfitting occurs when a model becomes too complex relative to the training data, memorizing specific examples rather than learning generalizable patterns. The model performs excellently on training data but fails to generalize to new, unseen data.

Examples: A decision tree that perfectly classifies training data but fails on test data, or a neural network with too many parameters for the dataset size.

Challenges

Detecting overfitting early, balancing model complexity with generalization, choosing appropriate regularization techniques, and ensuring sufficient training data.