Overfitting

Overfitting occurs when a machine learning model captures not only the true underlying patterns in the training data but also the random noise and idiosyncrasies specific to that dataset. An overfitted model will perform exceptionally well on training data but fail to generalize to new, unseen examples, which defeats the purpose of building a predictive model.

Signs of overfitting include a large gap between training accuracy and validation accuracy, or a validation loss that begins increasing while training loss continues to decrease. Overfitting is more likely when the model is too complex relative to the amount of training data, when training runs for too many epochs, or when the data contains significant noise.

Numerous techniques exist to combat overfitting. Regularization methods (L1, L2, dropout) constrain the model's capacity. Data augmentation increases the effective size of the training set. Early stopping halts training when validation performance begins to degrade. Batch normalization can also have a regularizing effect. Cross-validation helps detect overfitting during model selection, and ensemble methods can improve generalization by combining multiple models.