Underfitting
When a model is too simple to capture the underlying patterns in the data, resulting in poor performance.
Underfitting occurs when a model lacks sufficient complexity to learn the relationships in the training data. The model performs poorly on both training and test data because it cannot represent the true underlying function.
Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the data, resulting in poor performance on both training and test datasets. It's the opposite of overfitting, where the model is too complex and memorizes training data.
What Causes Underfitting
-
Insufficient model complexity - The model lacks enough parameters or layers to represent the data's true patterns. For example, trying to fit a curved relationship with a straight line.
-
Over-regularization - Too much regularization (techniques meant to prevent overfitting) can constrain the model so much that it can't learn even basic patterns.
-
Inadequate training - Not training long enough or with too few iterations for the model to learn the data patterns.
-
Poor feature selection - Using features that don't contain enough information to make accurate predictions.
-
Inappropriate algorithm choice - Using a linear model for highly non-linear data, or a simple algorithm for complex problems.
Signs of Underfitting
- High training error and high validation error
- Training and validation errors are similar (both poor)
- Model predictions seem overly simplistic
- Performance plateaus quickly during training
- The model fails to capture obvious patterns visible in the data
Example
Real-World Example Imagine trying to predict house prices using only the number of bedrooms. This single feature is too simple to capture the complexity of real estate pricing - you'd underfit because you're missing crucial information like location, size, condition, and market trends.
Solutions to Underfitting
-
Increase model complexity - Add more layers, neurons, or parameters to give the model more capacity to learn patterns.
-
Reduce regularization - Decrease regularization strength if it's too aggressive.
-
Feature engineering - Add more relevant features or create new ones that better represent the underlying patterns.
-
Train longer - Allow more epochs or iterations for the model to learn.
-
Choose better algorithms - Switch to more sophisticated models that can capture the data's complexity.
-
Balance - Finding the right balance between underfitting and overfitting is a core challenge in machine learning, often requiring experimentation and validation to achieve optimal model performance.