Understanding Regression Evaluation Metrics
Introduction:
Linear regression is a widely used statistical technique for predicting continuous outcomes based on the relationship between independent and dependent variables. To assess the performance of a linear regression model, various valuation metrics are employed. In this blog post, we will explore and explain the formulas, advantages, disadvantages, and for some common linear regression valuation metrics.
Dataset Description:
The Salary Dataset consists of two columns: YearsExperience and Salary. The YearsExperience column represents the number of years of experience, and the Salary column represents the corresponding salary of individuals.
After applying linear regression, we can observe a best-fit line, marked as “x” in the screenshot. The data points marked with green circles represent the actual values. The disparity between these actual values and the corresponding predicted values from our model is known as the error. In other words, the error represents the deviation or the variance between the true outcomes and the predicted values generated by our linear regression model. The goal of evaluating these errors is to assess the accuracy and performance of the model in making predictions.
For example, lets see the actual versus predicted values for 3.3 years of experience.
Actual value is 54,446.
Predicted value is 57190.20107683.
So the error is (Actual — Predicted) in predicting the value which is at index1.
57190.20107683–54446.0
=2,744.20107683
The above error is only for one point, and the total error is equal to sum of squares of the errors from all the points.
Now, let’s explore different evaluation metrics that can be used to quantify and analyze the errors in our linear regression model’s predictions. These metrics provide a quantitative measure of the disparities between the actual values and the predicted values. By examining these evaluation metrics, we can gain insights into the accuracy and performance of the model in making predictions.
Mean Squared Error (MSE): Mean Squared Error measures the average squared difference between the predicted and actual values. It quantifies the overall accuracy of the model’s predictions.
Advantages:
1. MSE is widely used and easy to interpret.
2. It penalizes larger errors more than smaller errors due to the squaring of residuals.
3. It ensures the residuals sum to zero, making it an unbiased estimator.
Disadvantages:
1. The metric is sensitive to outliers, as their squared residuals can significantly impact the value.
2. Since MSE is calculated using squared units, it may not provide a clear understanding of the magnitude of the error.
Mean Squared Error (MSE) calculated for our dataset.
Root Mean Squared Error (RMSE):
Root Mean Squared Error is the square root of MSE. It provides the average error between the predicted and actual values in the original scale of the dependent variable.
Advantages:
1. RMSE is highly interpretable as it is expressed in the same units as the dependent variable.
2. It provides a straightforward measure of the average magnitude of the prediction error.
Disadvantages:
1. Similar to MSE, RMSE is sensitive to outliers.
2. It may not be suitable when the focus is on the percentage or proportional error.
Root Mean Squared Error (MSE) calculated for our dataset.
Mean Absolute Error (MAE):
Mean Absolute Error measures the average absolute difference between the predicted and actual values. It provides a linear measure of the average prediction error.
Advantages:
1. MAE is not sensitive to outliers because it uses absolute differences instead of squared residuals.
2. It is easier to interpret as it represents the average magnitude of errors.
Disadvantages:
1. MAE treats all errors equally, regardless of their magnitude, which may not be desirable in certain scenarios.
Mean Absolute Error calculated for our dataset.
R-squared (R²) :
R-squared is another evaluation metric used in linear regression to assess the goodness of fit of the model. Unlike mean squared error (MSE), which focuses on the magnitude of the errors, R² measures the proportion of the variance in the dependent variable that can be explained by the independent variables.
R² ranges from 0 to 1, where:
· An R² value of 0 indicates that the model explains none of the variance in the dependent variable, suggesting a poor fit.
· An R² value of 1 indicates a perfect fit, where all of the variance in the dependent variable is explained by the independent variables.
Advantages:
· R² is easy to interpret and understand, as it represents the percentage of variance explained by the model.
· It allows for direct comparison between different models, with a higher R² indicating a better fit.
Disadvantages:
· R² tends to increase with the addition of more independent variables, even if they have no real predictive power. This can lead to overfitting.
A higher R² score suggests that a larger proportion of the variance in the dependent variable can be explained by the independent variables, indicating a better fit. Conversely, a lower R² score implies that the model explains less of the variance and may not be as effective in predicting the dependent variable accurately.
It’s important to note that R² should be used in conjunction with other evaluation metrics, such as MSE or RMSE, to gain a comprehensive understanding of the model’s performance and to avoid relying solely on R² for model assessment.
The formula for calculating R-squared (R²) is as follows:
where:
SSR (Sum of Squared Residuals) is the sum of the squared differences between the predicted values and the actual values.
SST (Total Sum of Squares) is the sum of the squared differences between the actual values and the mean of the dependent variable.
The numerator (SSR) represents the unexplained variation in the dependent variable, while the denominator (SST) represents the total variation in the dependent variable.
Conclusion:
In evaluating linear regression models, different valuation metrics offer distinct advantages and disadvantages. Choosing the appropriate metric depends on the specific requirements of the analysis and the nature of the data. Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) are suitable when overall error minimization is the goal, while Mean Absolute Error (MAE) provides a linear measure of the average error. R-squared helps assess the goodness of fit and the proportion of variance explained. By understanding these metrics, analysts can effectively evaluate the performance of linear regression models and make informed decisions.