Concepts
Overfitting
When a model learns training data too well, including noise
What is Overfitting?
Overfitting occurs when a machine learning model learns the training data too well, including its noise and outliers, resulting in poor generalization to new, unseen data. The model becomes too complex relative to the amount and noisiness of the training data.
Key Points
1
High training accuracy, low test accuracy
2
Model too complex for the data
3
Learns noise instead of patterns
4
Prevented by regularization
Practical Examples
Decision tree with too many branches
Neural network with excessive parameters
Polynomial regression with high degree
Memorizing training examples