ISS 2016 Statistics Paper-2 Solution: Question 8 (Properties of Regression Residuals)
Continuing our question-by-question walk through the ISS Statistics Paper-2 archive, this post picks up right where the last one left off. Each entry in this series takes a single previous-year item and solves it from first principles, so that by the time you reach the answer you also understand exactly why it is the answer.
Quick Summary
Topic:Properties of residuals in a linear regression model (Ordinary Least Squares)
Question Reference:ISS 2016, Statistics Paper-2, Question 8
Correct Answer:Option (b) — Statements 1, 3 and 4
The Question, As Asked
Which of the following statements are associated with residuals of regression model?
The sum of the residuals in any regression model that contains an intercept is always zero.
The sum of the observed values yi is not equal to sum of the fitted values ŷi.
The sum of the residuals weighted by the corresponding regressor variable is zero.
The sum of the residuals weighted by the corresponding fitted values is zero.
Select the correct answer using the code given below:
(a) 1, 2 and 3
(b) 1, 3 and 4
(c) 1, 2 and 4
(d) 2, 3 and 4
Step 1: Set Up the Model and Define the Residual
Start with the simple linear regression model, which is the cleanest way to see what is actually going on:
yi = β0 + β1xi + εi, for i = 1, 2, ..., n
Here yi is the observed (actual) value, xi is the value of the regressor (the explanatory variable), and εi is the unobservable random error. Once we estimate β0 and β1 from the data using the method of least squares, we get the fitted (predicted) value:
ŷi = β̂0 + β̂1xi
The residual for the i-th observation is simply the leftover gap between what was observed and what the model predicted:
ei = yi − ŷi
Everything in this question boils down to studying what happens when you add up these ei values in different weighted combinations. To do that properly, we need to go back to how β̂0 and β̂1 were obtained in the first place.
Step 2: Derive the Normal Equations From Least Squares
The method of least squares chooses β0 and β1 to minimise the sum of squared errors:
S(β0, β1) = Σ (yi − β0 − β1xi)2
To minimise S, we differentiate it with respect to β0 and β1 separately, and set each derivative equal to zero. This is just ordinary calculus — no shortcuts, every line written out.
Differentiating with respect to β0:
∂S/∂β0 = −2 Σ (yi − β0 − β1xi) = 0
Dividing both sides by −2 and substituting the least-squares estimates β̂0, β̂1 in place of β0, β1:
Σ (yi − β̂0 − β̂1xi) = 0 ⟹ Σ ei = 0 ...(Equation A)
Differentiating with respect to β1:
∂S/∂β1 = −2 Σ xi(yi − β0 − β1xi) = 0
Again dividing by −2 and substituting the estimates:
Σ xi(yi − β̂0 − β̂1xi) = 0 ⟹ Σ xiei = 0 ...(Equation B)
These two results, Equation A and Equation B, are called the normal equations of least squares, and every single statement in this question can be checked directly against them.
Step 3: Check Statement 1 — Sum of Residuals is Zero
This is exactly Equation A above:
Σ ei = 0
So whenever the model contains an intercept term β0, the residuals must add up to exactly zero.Statement 1 is TRUE.Notice that this only holds because we included β0 in the model — if you were to force the line through the origin (no intercept), this equation would not fall out of the calculus in the same way, and the residuals would not necessarily sum to zero.
Step 4: Check Statement 3 — Residuals Weighted by the Regressor Sum to Zero
This is exactly Equation B:
Σ xiei = 0
In words: if you multiply each residual by its corresponding x-value and add everything up, the total is zero. This is often described as saying "the residuals are uncorrelated with the regressor."Statement 3 is TRUE.
Step 5: Check Statement 2 — Comparing Sum of yi and Sum of ŷi
We already know from Equation A that Σ ei = 0. Now expand ei back out:
Σ (yi − ŷi) = 0
Splitting the summation into two separate sums:
Σ yi − Σ ŷi = 0 ⟹ Σ yi = Σ ŷi
So the sum of the observed values is exactly equal to the sum of the fitted values — not unequal. Statement 2 claims the opposite of what we just derived, so Statement 2 is FALSE.This is the one place in the question where a careless reader can get tricked, because the statement is phrased to sound plausible while actually contradicting the very first normal equation.
Step 6: Check Statement 4 — Residuals Weighted by Fitted Values Sum to Zero
This is the step most candidates fumble, because it looks like it needs a brand-new idea when really it is just Equations A and B combined. Start by writing out Σ eiŷi and substitute the formula for ŷi:
Σ eiŷi = Σ ei(β̂0 + β̂1xi)
Now distribute ei inside the bracket and split the sum into two parts:
= β̂0 Σ ei + β̂1 Σ xiei
But we already showed Σ ei = 0 (Equation A) and Σ xiei = 0 (Equation B). Substituting these in:
= β̂0(0) + β̂1(0) = 0
So Σ eiŷi = 0, confirming that residuals weighted by the fitted values also sum to zero.Statement 4 is TRUE.
Pro Tip: Whenever a question asks you to check something involving ŷi, don't treat it as a new object to reason about from scratch — always rewrite it as β̂0 + β̂1xi first. Once you do that, every "fitted value" statement collapses back into the two normal equations you already know, Σei = 0 and Σxiei = 0. The way this trick is usually taught is to treat the normal equations as your only two tools for this entire topic — residual-orthogonality questions in every past paper reduce to combinations of just these two facts, and once that clicks, this whole category of question stops requiring any memorisation at all.
Final Answer
Putting it all together:
Statement 1 — TRUE
Statement 2 — FALSE
Statement 3 — TRUE
Statement 4 — TRUE
The statements that hold are 1, 3 and 4, which corresponds to option (b). This matches the official answer key for ISS 2016 Statistics Paper-2, Question 8, which also lists the correct response as (b). Our derivation and the official key are in full agreement.
Why This Question Matters
This question is really testing whether you understand the normal equations as the foundation of ordinary least squares, rather than testing regression as a memorised formula. Almost every "true/false about residuals" question that shows up in ISS Paper-2 — orthogonality with regressors, orthogonality with fitted values, the sum of residuals, R² decompositions — can be solved by returning to these same two equations and substituting carefully. Students who drill this kind of derivation until it becomes automatic tend to stop losing marks on the entire residuals-and-ANOVA block of the paper, since the same two lines of algebra reappear in disguise across a dozen different question formats.
Frequently Asked Questions
Why is the sum of residuals always zero when the model has an intercept?
It falls directly out of the least-squares minimisation. When you differentiate the sum of squared errors with respect to the intercept β0 and set the derivative to zero, you get Σei = 0 as a direct consequence. It isn't an assumption — it's a guaranteed outcome of how β̂0 is calculated.
What happens to this property if the regression model has no intercept?
If you force the line through the origin (no β0 term), the derivative with respect to β0 is never taken, so Equation A never arises. In that case the residuals do not generally sum to zero, which is exactly why the question specifically says "any regression model that contains an intercept."
Is Σxiei = 0 the same as saying x and e are uncorrelated?
Yes, in the sample sense. Since Σei = 0, the sample covariance between xi and ei is proportional to Σxiei, which we showed equals zero. This is the mathematical basis for saying that OLS residuals carry no leftover linear relationship with the regressor used to fit the model.
Does the orthogonality with fitted values (statement 4) hold in multiple regression too, with several x variables?
Yes. In matrix notation, the normal equations for any linear model with design matrix X are X′e = 0. Since the fitted values are ŷ = Xβ̂, we get e′ŷ = e′Xβ̂ = (X′e)′β̂ = 0, exactly the same argument as in Step 6, just written with matrices instead of separate β0 and β1 terms.
Why is statement 2 designed to look correct at first glance?
It borrows the true fact — that residuals sum to zero — and quietly negates the conclusion, claiming the sums of yi and ŷi are unequal when they are in fact always equal. This kind of "true premise, false conclusion" trap is common in ISS objective papers, which is exactly why re-deriving the identity rather than recalling it from memory keeps you safe.
Can I use this property to sanity-check my own regression calculations in the exam?
Absolutely. After computing β̂0 and β̂1 and the fitted values for a numerical regression question, quickly summing your residuals is a fast way to catch arithmetic slips — if they don't add up to (approximately) zero, you have made an error somewhere upstream.
If any step above felt rushed or you'd like it explained a different way, drop your doubt in the comments — and if you found this useful, consider sharing it with another ISS aspirant who is working through the same paper.
Comments