ISS 2016 Statistics Paper-2 Solution: Question 15 (Least Squares and Error Assumptions)
Continuing our question-by-question walkthrough of the ISS Statistics Paper-2 previous year papers, today we pick up right where the last post left off and tackle Question 15 from the 2016 paper. This one is a favourite of examiners because it looks like a pure regression-formula question but is really testing whether you understand exactly which assumptions the method of least squares actually needs — and which ones it doesn't.
Quick Summary
Topic:Simple Linear Regression — the Method of Least Squares and the role of assumptions on the disturbance term
Question Reference:ISS 2016, Statistics Paper-2, Question 15
Correct Answer:(a) 1 and 2 only
The Question, Exactly As Asked
If in a linear model yi = β0 + β1xi + εi for i = 1, 2, 3, ..., n, E(εi) = 0, E(εiεj) = 0 for all i ≠ j, then which of the following statements are correct for the method of estimation by least squares?
The least square estimates of the parameters β0 and β1 are determined by minimising the sum of squares of residuals.
The observations yi are pairwise uncorrelated.
The observations yi are jointly independent and each yi is a normal random variable.
Select the correct answer using the code given below:
(a) 1 and 2 only
(b) 2 and 3 only
(c) 1 and 3 only
(d) 1, 2 and 3
Step 1: Write Down Exactly What Is Given, and Nothing More
Before touching any of the three statements, it helps to list precisely what the question hands us:
The model is yi = β0 + β1xi + εi, where xi are fixed (non-random) values of the regressor.
E(εi) = 0 for every i — the errors have zero mean.
E(εiεj) = 0 for all i ≠ j — the errors are pairwise uncorrelated (since their means are already zero, E(εiεj) is exactly the covariance Cov(εi, εj)).
Notice what is not given: there is no statement that the εi are normally distributed, and no statement that they are mutually independent (only that they are uncorrelated, which is a weaker condition). This distinction is the entire crux of the question, so keep it firmly in mind as we go through each statement.
Step 2: Checking Statement 1 — Is Least Squares "Minimising the Sum of Squares of Residuals"?
The method of least squares, by its very definition, chooses the estimates of β0 and β1 — call them β̂0 and β̂1 — so as to minimise the residual sum of squares:
S(β0, β1) = Σ(yi − β0 − β1xi)2
This is not something we derive from the assumptions on εi — it is literally the definition of the least squares criterion. You get the normal equations by taking ∂S/∂β0 = 0 and ∂S/∂β1 = 0 and solving them simultaneously. Since this defining property holds no matter what assumptions we place on the error term, statement 1 is true.
Step 3: Checking Statement 2 — Are the yi Pairwise Uncorrelated?
This is where the given information does real work. Since xi is treated as a fixed constant (not a random variable) in the classical linear regression setup, the only source of randomness in yi = β0 + β1xi + εi is the error term εi itself. Constants don't affect covariance, so:
Cov(yi, yj) = Cov(β0 + β1xi + εi, β0 + β1xj + εj) = Cov(εi, εj)
Now expand the covariance using its definition:
Cov(εi, εj) = E(εiεj) − E(εi)E(εj)
We are given E(εiεj) = 0 for i ≠ j, and E(εi) = E(εj) = 0, so E(εi)E(εj) = 0 as well. Substituting:
Cov(εi, εj) = 0 − 0 = 0
Hence Cov(yi, yj) = 0 for every i ≠ j, which is exactly what "pairwise uncorrelated" means. Statement 2 is true.
Step 4: Checking Statement 3 — Independence and Normality
Statement 3 makes two separate claims bundled into one: (i) the yi are jointly independent, and (ii) each yi is normally distributed. Neither of these follows from what we were given.
Uncorrelatedness is a statement about second moments only — it says nothing about the full joint distribution of the εi's. Two random variables can be uncorrelated yet still statistically dependent (for instance, through their variances or higher moments); uncorrelated implies independent only under an additional assumption, most commonly that the variables are jointly normally distributed. Since the question never assumes normality for εi, we cannot conclude independence, and we certainly cannot assert that each yi is normal.
Pro Tip: This is the exact spot where most candidates lose the mark on this question — they see "uncorrelated" in the stem and instinctively read it as "independent," because in a lot of numerical problems the two get treated interchangeably. The way this distinction is usually drilled into students is with a one-line rule:
So statement 3 is false, since it smuggles in assumptions (independence and normality) that the question never granted us.
Step 5: Final Answer and Cross-Check With the Official Key
Putting the three findings together: statement 1 is true (it's the definition of least squares), statement 2 is true (uncorrelated errors plus fixed xi give uncorrelated yi), and statement 3 is false (uncorrelated does not imply independent or normal without extra assumptions). That means the correct combination is 1 and 2 only, which corresponds to option (a).
This matches the official ISS 2016 Statistics Paper-2 answer key (Series A), which also lists the answer to Question 15 as (a). So our independently derived answer and the official key are in full agreement — no discrepancy to flag here.
Why This Question Matters
This question is a great example of how ISS Paper-2 tests conceptual precision rather than pure computation. There's no arithmetic here at all — the entire question hinges on knowing exactly which properties of least squares estimation follow purely from its definition (minimising residual sum of squares), which follow from a second-moment assumption (uncorrelated errors), and which require a much stronger distributional assumption (normality) that was never stated. This same reasoning pattern — separating "what minimisation gives you for free" from "what you need normality for" — resurfaces constantly in questions on the Gauss-Markov theorem, hypothesis testing in regression, and ANOVA, so getting comfortable with it here pays off across the rest of the paper.
Frequently Asked Questions
Does the method of least squares require the errors to be normally distributed?
No. Least squares estimation only requires you to minimise the sum of squared residuals, which is purely an algebraic/optimisation exercise. Normality is needed later, when you want to construct confidence intervals or carry out t-tests and F-tests on the estimated coefficients — not for obtaining the estimates themselves.
What is the actual difference between "uncorrelated" and "independent" random variables?
Independence is a much stronger condition: it means the joint distribution factorises completely, so knowing one variable tells you nothing whatsoever about the other, at any moment. Uncorrelated only means the covariance (a second-moment measure) is zero. Independent variables are always uncorrelated, but uncorrelated variables need not be independent unless you add an assumption such as joint normality.
Why are xi treated as fixed constants rather than random variables in this question?
In the classical linear regression framework used throughout ISS Paper-2, the regressor values xi are assumed to be fixed in repeated sampling — they represent design points chosen in advance, not outcomes of a random experiment. This assumption is what lets us say the only randomness in yi comes from εi, which is essential for the covariance calculation in Step 3.
If E(εiεj) = 0 is given, why do we still need E(εi) = 0 to conclude Cov(εi, εj) = 0?
By definition, Cov(εi, εj) = E(εiεj) − E(εi)E(εj). If E(εi) were not zero, the product term E(εi)E(εj) could be non-zero and would have to be subtracted from E(εiεj) to get the true covariance. It's only because the question separately tells us E(εi) = 0 that E(εiεj) = 0 directly gives us Cov(εi, εj) = 0.
Is this type of "select the correct statements" question common in ISS Paper-2?
Yes, very. A large share of Paper-2 questions on regression and ANOVA are framed exactly this way — a set of numbered statements followed by a "select the code" instruction. The efficient strategy is always the same: evaluate each statement completely independently against only the assumptions explicitly given, then match against the option codes at the end.
Does this result change if the errors have unequal variances (heteroscedasticity)?
Not for this particular question. Statements 1 and 2 only used E(εi) = 0 and E(εiεj) = 0 for i ≠ j — neither depends on whether Var(εi) is the same for every i. Heteroscedasticity would matter if the question asked about the efficiency of the least squares estimator (that's where the Gauss-Markov / BLUE property and constant variance come in), but it doesn't affect whether the yi are pairwise uncorrelated.
Hope this step-by-step breakdown made the uncorrelated-versus-independent distinction click. If anything here still feels shaky, or if you spot a different way to look at it, drop a comment below — and if a fellow ISS aspirant is grinding through the same paper, send this one their way.
Comments