The sum of the squared residuals is one of the most important quantities in statistics and machine learning because it tells you how far a model’s predictions are from the actual data. Now, when you compute the sum of the squared residuals, you are measuring the total squared distance between observed values and predicted values. Day to day, this single number helps you compare models, evaluate regression fit, and understand whether a line or curve is doing a good job of explaining the data. Whether you are a student learning least squares regression or an analyst checking a predictive model, knowing how to compute the sum of the squared residuals accurately is a foundational skill.
What Is the Sum of the Squared Residuals?
A residual is the difference between an observed value and the value predicted by a model. In regression, the residual for the i-th observation is written as:
[ e_i = y_i - \hat{y}_i ]
where (y_i) is the actual observed value and (\hat{y}_i) is the predicted value. The residual can be positive, negative, or zero. A positive residual means the model predicted too low; a negative residual means the model predicted too high.
To compute the sum of the squared residuals, you square each residual and then add all the squared values together:
[ SSR = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 ]
The result is always non-negative. Here's the thing — squaring does two things: it removes the sign so positive and negative errors do not cancel each other out, and it gives larger errors more weight. That is why the sum of the squared residuals is especially sensitive to large misses Most people skip this — try not to..
You may also see this quantity called the residual sum of squares (RSS) or the sum of squared errors (SSE). The names vary across textbooks and software, but the calculation is the same: take each residual, square it, and sum the results And it works..
Why Computing the Sum of Squared Residuals Matters
The sum of the squared residuals is central to ordinary least squares (OLS) regression. In OLS, the goal is to choose model coefficients that minimize the sum of the squared residuals. In plain terms, the best-fitting line is the one that makes this total squared error as small as possible Took long enough..
This is the bit that actually matters in practice Simple, but easy to overlook..
Computing the sum of the squared residuals is useful for several reasons:
- Model evaluation: A smaller SSR generally indicates that the model’s predictions are closer to the observed data.
- Model comparison: When comparing models on the same dataset and same response variable, lower SSR suggests a better fit.
- Optimization: Many regression algorithms use SSR as the loss function they minimize during training.
- Diagnostics: Large residuals can reveal outliers, missing variables, or a model that is too simple for the data.
- R-squared calculation: SSR is used to compute the coefficient of determination, (R^2), which measures the proportion of variance explained by the model.
Even so, SSR alone is not a perfect measure. It depends on the scale of the response variable and the number of observations. So a large SSR may simply reflect large values in the data rather than a poor model. That is why analysts often pair SSR with other metrics such as root mean squared error (RMSE) and R-squared.
Worth pausing on this one.
Step-by-Step Guide to Compute the Sum of the Squared Residuals
Follow these steps to compute the sum of the squared residuals for any set of observed and predicted values.
Step 1: Collect the Observed and Predicted Values
List each observed value (y_i) and its corresponding predicted value (\hat{y}_i). Think about it: make sure the values are paired correctly. If you are working with a regression equation, plug each predictor value into the equation to get the predicted value.
Step 2: Calculate Each Residual
Subtract the predicted value from the observed value:
[ e_i = y_i - \hat{y}_i ]
Keep the sign for now. You will square the residual in the next step.
Step 3: Square Each Residual
Square every residual:
[ e_i^2 = (y_i - \hat{y}_i)^2 ]
Squaring removes negative signs and ensures that all errors contribute positively to the total.
Step 4: Sum the Squared Residuals
Add all the squared residuals together:
[ SSR = e_1^2 + e_2^2 + \cdots + e_n^2 ]
The final result is the sum of the squared residuals.
Worked Example: Computing SSR by Hand
Suppose you have four observations and a model that produces the following predicted values:
| Observation | Observed (y) | Predicted (\hat{y}) | Residual (y - \hat{y}) | Squared Residual |
|---|---|---|---|---|
| 1 | 4 | 3.Consider this: 5 | 0. 5 | 0.Think about it: 25 |
| 2 | 6 | 6. 2 | -0. |