Bitzu

Machine Learning Associate ยท Model evaluation and generalization

Underfitting vs overfitting and the bias-variance tradeoff

Underfitting versus overfitting, training versus validation error patterns, and responses such as regularization, simpler or more complex models, and more data.

Key points

  • Underfitting (high bias): the model is too simple, so training and validation error are both high.
  • Overfitting (high variance): training error is low but validation error is much higher.
  • Remedies for overfitting: regularization, a simpler model, more training data.
  • Remedies for underfitting: a more complex model, more or better features, less regularization.
  • More data helps variance, not bias.

Source: https://scikit-learn.org/stable/modules/learning_curve.html

Practice questions

Single choice

A model shows very low training error but much higher validation error. What is this pattern called?

  1. Underfitting
  2. Data leakage into the test set only
  3. Overfitting
  4. A well-fitted model
Show answer

Answer: C. Overfitting

A large gap with low training error is overfitting (high variance). Underfitting would show high error on both sets.

Select all that apply

Which actions can reduce overfitting?

  1. Increase the regularization strength
  2. Add more training data
  3. Reduce the model's complexity, for example a shallower tree
  4. Add more layers to the network
  5. Train for many more epochs without monitoring validation error
Show answer

Answer: A. Increase the regularization strength, B. Add more training data, C. Reduce the model's complexity, for example a shallower tree

Stronger regularization, more data and a simpler model all lower variance. Adding layers and training longer increases capacity and tends to worsen overfitting.

True or false

True or false: collecting more training data is a reliable fix for an underfitting model.

Show answer

Answer: False

False. A model that is too simple stays too simple with more rows; both errors remain high. More data mainly helps when variance is the problem.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More Machine Learning Associate topics in this module