ISS 2016 Statistics Paper-2 Solution: Question 8 (Sum of Regression Residuals)
We're back with the next entry in our ongoing question-by-question walkthrough of the ISS Statistics Paper-2 previous year papers. Today we pick up right where Question 7 left off and work through Question 8 of the 2016 paper, which tests a property of regression residuals that trips up a lot of first-time learners because it "feels obvious" until you're asked to actually prove it.
Quick Summary
Topic: Properties of OLS residuals in a linear regression model
Question Reference: ISS 2016, Statistics Paper-2, Question 8
Correct Answer: Option (b) — Statements 1, 3 and 4
The Question, As Asked
Which of the following statements are associated with residuals of regression model?
The sum of the residuals in any regression model that contains an intercept is always zero.
The sum of the observed values yi is not equal to sum of the fitted values ŷi.
The sum of the residuals weighted by the corresponding regressor variable is zero.
The sum of the residuals weighted by the corresponding fitted values is zero.
Select the correct answer using the code given below:
(a) 1, 2 and 3
(b) 1, 3 and 4
(c) 1, 2 and 4
(d) 2, 3 and 4
Step 1: Set Up the Model and the Least Squares Problem
Start with the usual multiple linear regression model written in matrix form:
y = Xβ + ε
Here y is the n × 1 vector of observed responses, X is the n × p design matrix (its first column is a column of 1's if the model has an intercept β0), β is the p × 1 vector of unknown coefficients, and ε is the error vector.
Ordinary Least Squares (OLS) chooses the estimate β̂ that minimizes the sum of squared errors:
S(β) = (y − Xβ)′(y − Xβ)
Once we have β̂, we define two things every regression course leans on constantly:
Fitted values: ŷ = Xβ̂
Residuals: e = y − ŷ
Step 2: Derive the Normal Equations (This Is the Key That Unlocks the Whole Question)
To minimize S(β), differentiate it with respect to β and set the result to zero:
∂S/∂β = −2X′(y − Xβ̂) = 0
Dividing both sides by −2 gives the famous normal equations of least squares:
X′(y − Xβ̂) = 0, which is the same as X′e = 0
This single matrix equation is doing all the work in this question. In plain words, it says: the residual vector e is orthogonal (perpendicular, in the dot-product sense) to every single column of X. Every one of the four statements in this question is really just asking "is this particular consequence of X′e = 0 true or false?" — so once you have this equation, the rest is just picking off columns of X one at a time.
Pro Tip: Whenever an ISS or IIT JAM style MCQ throws four different "sum of residuals weighted by ___" statements at you, don't memorize each one separately. Just write down X′e = 0 once, identify what each column of X actually is in that statement (a column of 1's? the regressor x? the fitted values ŷ, which are themselves a linear combination of the columns of X?), and read the answer straight off. This is the habit that's usually drilled into students early — reduce every residual-property question to a single line, X′e = 0, rather than carrying four separate facts in your head — and once it's automatic, this entire category of question stops being a source of silly mistakes.
Step 3: Check Statement 1 — Sum of Residuals With an Intercept
If the model has an intercept, the first column of X is a column of all 1's. Call this column vector 1 (size n × 1). Since X′e = 0 must hold for every column of X, it must in particular hold for this column:
1′e = 0
Writing this out in scalar form, 1′e is just e1 + e2 + ... + en, i.e. the plain sum of all the residuals. So:
Σ ei = 0
This is exactly Statement 1, and it is true. It only requires that the model contain an intercept — no other assumption about X or the distribution of ε is needed.
Step 4: Check Statement 2 — Sum of yi vs Sum of ŷi
Since ei = yi − ŷi for every observation, summing over all i gives:
Σ ei = Σ yi − Σ ŷi
We already showed in Step 3 that Σ ei = 0 (as long as there is an intercept, which this question's setup assumes throughout). So:
Σ yi − Σ ŷi = 0, which rearranges to Σ yi = Σ ŷi
So the sum of the observed values is equal to the sum of the fitted values. Statement 2 claims the opposite ("is not equal to"), so Statement 2 is false. This is a nice one for the examiner to slip in, because it's testing whether you actually understood the consequence of Statement 1 or just memorized it as an isolated fact.
Step 5: Check Statement 3 — Residuals Weighted by the Regressor
Now apply X′e = 0 to a regressor column instead of the intercept column. Suppose x is any one of the regressor (independent variable) columns of X, containing values x1, x2, ..., xn. The corresponding normal equation is:
x′e = 0, i.e. Σ xi ei = 0
That is precisely "the sum of the residuals weighted by the corresponding regressor variable is zero." So Statement 3 is true — and it holds for every regressor column in the model, not just one.
Step 6: Check Statement 4 — Residuals Weighted by the Fitted Values
This is the one most students hesitate on, because ŷ isn't one of the "raw" columns of X — it's a derived quantity, ŷ = Xβ̂. But that's exactly what makes the proof quick. Consider the weighted sum:
Σ ŷi ei = ŷ′e = (Xβ̂)′e = β̂′X′e
We already know X′e = 0 from Step 2. Substitute it in:
β̂′X′e = β̂′(0) = 0
So Σ ŷi ei = 0, which is exactly Statement 4. It is true. (You may recognize this as the same "fitted values and residuals are uncorrelated" idea that shows up as its own separate question elsewhere in this very paper — it's one of the most frequently recycled facts in the whole regression syllabus.)
Step 7: Combine the Results and Match the Options
Collecting what we found:
Statement 1 — True
Statement 2 — False
Statement 3 — True
Statement 4 — True
So the correct statements are 1, 3 and 4, which is option (b).
The official answer key for ISS 2016 Statistics Paper-2 (Series A) also lists B for Question 8, so our derivation matches the official key exactly.
Why This Question Matters
This question is really testing one idea — the normal equations X′e = 0 — dressed up as four separate statements. That single equation is the backbone of everything else you'll be asked about OLS: unbiasedness of β̂, the Gauss-Markov theorem, the decomposition of total sum of squares into regression and residual sum of squares, and even why R² is bounded between 0 and 1. If this derivation is solid, a whole cluster of ISS and other statistics-exam MCQs on regression become one-line answers instead of things to memorize individually. It's a small investment of algebra that pays off across dozens of future questions.
Frequently Asked Questions
Does the sum of residuals always equal zero in every regression model?
No — only when the model includes an intercept term. If you force the regression line through the origin (no intercept), the column of 1's is no longer part of X, so the normal equation 1′e = 0 no longer applies, and the residuals will generally not sum to zero.
What exactly are "normal equations" in least squares regression?
They are the system of equations you get by setting the derivative of the sum of squared errors with respect to each coefficient equal to zero. In matrix form this is X′(y − Xβ̂) = 0, or equivalently X′e = 0. Solving this system for β̂ gives the familiar formula β̂ = (X′X)−1X′y.
Is Statement 4 the same as saying residuals and fitted values are uncorrelated?
Essentially yes. Σ ŷi ei = 0 combined with Σ ei = 0 means the sample covariance between ŷ and e is zero, which is what "uncorrelated" means in this context. This is one of the standard facts examiners like to test from multiple angles.
Why is Statement 2 written as "not equal to" instead of just being obviously wrong?
This is a common MCQ technique — take a true relationship (Σyi = Σŷi) and flip it into a false one by inserting "not." It rewards candidates who derived the result themselves in Step 3 and Step 4, rather than those who only remember the raw fact "residuals sum to zero" without tracing through its consequences.
Do these residual properties hold for weighted least squares or only for OLS?
As stated here, they are specific to ordinary least squares with the standard normal equations X′e = 0. Under weighted least squares the normal equations change to X′We = 0 for the weight matrix W, so the plain (unweighted) sum of residuals is generally not zero — only the weighted sum is.
How should I revise this topic for the ISS exam?
Rather than memorizing separate "residual facts," practice re-deriving X′e = 0 from scratch every time until it takes under a minute, then apply it to whatever column the question asks about — intercept, a regressor, or fitted values. Students who drill this kind of derivation until it becomes second nature, the way it's taught at coaching programs like Sunrise Classes, tend to handle every variant of this question — including ones with slightly different wording — without needing to recall each version separately.
That's Question 8 done and dusted. If any step felt rushed or you'd solve it differently, drop a comment below with your doubt — and if a fellow ISS aspirant is stuck on the same topic, feel free to share this post with them.
Comments