Bitzu

Machine Learning Associate ยท Model evaluation and generalization

Regression metrics: MSE, RMSE, MAE and R-squared

RMSE, MSE, MAE and R-squared: units, sensitivity to outliers and interpretation.

Key points

  • MSE is the mean of squared errors (squared units of the target).
  • RMSE is its square root, back in the target's units, and still penalizes large errors heavily.
  • MAE is the mean absolute error: target units, less sensitive to outliers.
  • R-squared is unitless: 1 is perfect, 0 matches always predicting the mean, and it can be negative when a model is worse than that.

Source: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.r2_score.html

Practice questions

Single choice

Which metric is expressed in the squared units of the target variable?

  1. RMSE
  2. MAE
  3. MSE
  4. R-squared
Show answer

Answer: C. MSE

MSE averages squared errors, so a dollar target gives dollars squared. RMSE and MAE are in dollars; R-squared is unitless.

Select all that apply

Which statements about R-squared are correct?

  1. A value of 1.0 means the predictions match the targets exactly
  2. It can be negative when the model does worse than always predicting the mean
  3. A value of 0 matches a model that always predicts the mean of the target
  4. It always lies between 0 and 1
  5. It is expressed in the same units as the target
Show answer

Answer: A. A value of 1.0 means the predictions match the targets exactly, B. It can be negative when the model does worse than always predicting the mean, C. A value of 0 matches a model that always predicts the mean of the target

R-squared is 1 for a perfect fit, 0 for a constant mean predictor, and negative when a model is worse than that. It is not bounded below by 0 and it is unitless.

True or false

True or false: RMSE can never be larger than MAE on the same predictions.

Show answer

Answer: False

False. RMSE is always greater than or equal to MAE; they are equal only when all absolute errors are identical. The more uneven the errors, the bigger the gap.

Practice 8 questions on this topic in Bitzu

Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.

Download on the App StoreGet it on Google Play

More Machine Learning Associate topics in this module