Machine Learning Associate ยท Model evaluation and generalization
Regression metrics: MSE, RMSE, MAE and R-squared
RMSE, MSE, MAE and R-squared: units, sensitivity to outliers and interpretation.
Key points
- MSE is the mean of squared errors (squared units of the target).
- RMSE is its square root, back in the target's units, and still penalizes large errors heavily.
- MAE is the mean absolute error: target units, less sensitive to outliers.
- R-squared is unitless: 1 is perfect, 0 matches always predicting the mean, and it can be negative when a model is worse than that.
Source: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.r2_score.html
Practice questions
Which metric is expressed in the squared units of the target variable?
- RMSE
- MAE
- MSE
- R-squared
Show answer
Answer: C. MSE
MSE averages squared errors, so a dollar target gives dollars squared. RMSE and MAE are in dollars; R-squared is unitless.
Which statements about R-squared are correct?
- A value of 1.0 means the predictions match the targets exactly
- It can be negative when the model does worse than always predicting the mean
- A value of 0 matches a model that always predicts the mean of the target
- It always lies between 0 and 1
- It is expressed in the same units as the target
Show answer
Answer: A. A value of 1.0 means the predictions match the targets exactly, B. It can be negative when the model does worse than always predicting the mean, C. A value of 0 matches a model that always predicts the mean of the target
R-squared is 1 for a perfect fit, 0 for a constant mean predictor, and negative when a model is worse than that. It is not bounded below by 0 and it is unitless.
True or false: RMSE can never be larger than MAE on the same predictions.
Show answer
Answer: False
False. RMSE is always greater than or equal to MAE; they are equal only when all absolute errors are identical. The more uneven the errors, the bigger the gap.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
