ISS 2016 Statistics Paper-2 Solution: Question 9 (Properties of OLS Regression Estimators)

Continuing our question-by-question walk through the ISS Statistics Paper-2 previous-year papers, we now arrive at Question 9 of the 2016 paper. This one looks like a simple "which statements are correct" question on regression, but it is really testing something much sharper: whether you read the notation carefully enough to notice a missing bar over a letter.

Quick Summary

• Topic:Properties of Ordinary Least Squares (OLS) estimators in simple linear regression

• Question Reference:ISS 2016, Statistics Paper-2, Question 9

• Correct Answer:Option (a) — Statements 1, 2 and 3 are correct

The Question, Exactly As Asked

For a simple linear regression model y = β0 + β1x + ε, consider the following statements where β̂0 and β̂1 are estimates of β0 and β1.

1. Cov (y, β̂1) ≠ 0

2. Cov (β̂0, β̂1) = −x̄ V(β̂1)

3. E(y) = β0 + β1x

4. β̂0 = ȳ − β̂1x

Which of the above are correct?

1. 1, 2 and 3

2. 1, 3 and 4

3. 2, 3 and 4

4. 1, 2 and 4

Step 1: Set Up the Model Carefully

Before touching any of the four statements, let us write down exactly what we are working with, because every statement will be checked against this setup.

We have n pairs of observations (x1, y1), (x2, y2), ..., (xn, yn), and the model is

yi = β0 + β1xi + εi, for i = 1, 2, ..., n

with the usual assumptions: the xi are treated as fixed (non-random) numbers, and the errors εi are random with E(εi) = 0, Var(εi) = σ² for every i, and Cov(εi, εj) = 0 whenever i ≠ j (the errors are uncorrelated with each other).

The least squares estimators are the familiar

β̂1 = Sxy / Sxx, where Sxy = Σ(xi − x̄)(yi − ȳ) and Sxx = Σ(xi − x̄)²

and

β̂0 = ȳ − β̂1x̄

Notice the bar over the x in that last formula — ȳ minus β̂1 times x̄ (the mean of x), not any old x. Keep that in mind; it is exactly the detail that decides this question.

Step 2: Rewrite β̂1 as a Weighted Sum of the y's

Almost every property of β̂1 and β̂0 becomes easy once you write β̂1 as a linear combination of the yi's, because then you can use the simple covariance rules for sums.

Define the weight ci = (xi − x̄) / Sxx. Then

β̂1 = Σ ci(yi − ȳ) = Σ ciyi − ȳΣci

Since Σci = Σ(xi − x̄)/Sxx = 0/Sxx = 0 (the deviations of x from their own mean always sum to zero), the second term vanishes and we get the clean result

β̂1 = Σ ciyi

We will also need two small facts about the ci's, both easy to verify from the definition: Σci = 0 (shown above) and Σcixi = 1 (substitute and simplify using Sxx = Σ(xi − x̄)xi). These two facts are what make β̂1 an unbiased estimator of β1, but here we only need them for covariance calculations.

Step 3: Check Statement 3 — E(y) = β0 + β1x

This is the most basic statement of all. Taking expectations on both sides of the model equation yi = β0 + β1xi + εi, and using E(εi) = 0, we get directly

E(yi) = β0 + β1xi

That is just the definition of the regression line as "the expected value of y at a given x." So statement 3 is correct.

Step 4: Check Statement 1 — Cov(y, β̂1) ≠ 0

Here "y" refers to an individual observation yi (there is no bar over it, so it is not the sample mean ȳ). Using β̂1 = Σj cjyj from Step 2,

Cov(yi, β̂1) = Cov(yi, Σj cjyj) = Σj cjCov(yi, yj)

Because the yj's inherit their randomness only from the independent εj's, Cov(yi, yj) is zero for every j except j = i, where it equals σ². So the sum collapses to a single term:

Cov(yi, β̂1) = ciσ² = [(xi − x̄)/Sxx]σ²

This is nonzero for any observation whose xi is different from x̄ — which, in a real data set, is true for almost every observation. So, read as a general statement about "a" y (not the mean), statement 1 is correct.

Step 5: Check Statement 2 — Cov(β̂0, β̂1) = −x̄ V(β̂1)

Start from β̂0 = ȳ − β̂1x̄ and take its covariance with β̂1:

Cov(β̂0, β̂1) = Cov(ȳ − β̂1x̄, β̂1) = Cov(ȳ, β̂1) − x̄ Var(β̂1)

(x̄ is just a fixed number here, so it comes straight out of the covariance, and Cov(β̂1, β̂1) is simply Var(β̂1).)

Now we need Cov(ȳ, β̂1). Using ȳ = (1/n)Σyi and β̂1 = Σcjyj:

Cov(ȳ, β̂1) = (1/n) Σi Σj cj Cov(yi, yj) = (1/n) Σi ciσ² = (σ²/n) Σci

and since Σci = 0 (established in Step 2), this whole term is exactly zero:

Cov(ȳ, β̂1) = 0

Substituting back, Cov(β̂0, β̂1) = 0 − x̄ Var(β̂1) = −x̄ V(β̂1), which is exactly statement 2. So statement 2 is correct.

Pro Tip: The single most useful fact buried in this question is Cov(ȳ, β̂1) = 0 — the sample mean of y and the slope estimator are always uncorrelated in a fixed-x design. Once you have this fact sitting ready in your head, statement 2 becomes a two-line derivation instead of something you need to memorise as a formula. This is exactly the kind of "derive it, don't memorise it" habit that is drilled into students until it becomes automatic — the sign and the x̄ in Cov(β̂0, β̂1) = −x̄V(β̂1) then follow with almost no extra work.

Step 6: Check Statement 4 — β̂0 = ȳ − β̂1x

This is where you have to slow down and read every character. Compare it letter by letter with the actual OLS formula we wrote in Step 1:

Correct formula: β̂0 = ȳ − β̂1 x̄(bar over the x)

Statement 4: β̂0 = ȳ − β̂1 x (no bar over the x)

These are not the same equation. The correct identity involves x̄, the mean of all the x-values used to fit the line — it comes from the fact that the least squares line always passes through the point (x̄, ȳ). Statement 4 instead uses a plain x, which would mean the fitted line passes through (x, ȳ) for any x you like, which is false — a straight line does not pass through (x, ȳ) for every x unless the slope is zero. The only x for which the stated equation holds is x = x̄ itself.

Because the statement replaces x̄ with x, it is not a valid general identity, and statement 4 is incorrect.

This one-character difference between x and x̄ is the crux of the whole question — everything else in the four statements is a fairly standard textbook result, but statement 4 is deliberately written to look right at a glance while being wrong on close reading.

Step 7: Put It Together and Confirm the Final Answer

Collecting the results:

• Statement 1 (Cov(y, β̂1) ≠ 0):Correct

• Statement 2 (Cov(β̂0, β̂1) = −x̄V(β̂1)):Correct

• Statement 3 (E(y) = β0 + β1x):Correct

• Statement 4 (β̂0 = ȳ − β̂1x):Incorrect

So the correct combination is 1, 2 and 3, which is option (a). This matches the official answer key for this question exactly, so no discrepancy to flag here — our derivation and the key agree.

Why This Question Matters

This question is a good example of how ISS Paper-2 tests regression theory: it rarely asks you to just recall a formula, it asks you to verify one, and it hides a small notational trap to see if you actually understand where the formula comes from rather than pattern-matching the shape of the equation. The underlying results here — that β̂1 is a linear combination of the y's, that ȳ and β̂1 are uncorrelated, and that Cov(β̂0, β̂1) depends on x̄ — reappear constantly in later topics like the variance of predicted values, confidence intervals for the regression line, and the derivation of the ANOVA F-test for regression. Getting comfortable deriving them from first principles, rather than memorising them, pays off across many other questions in the paper.

Frequently Asked Questions

Why is Cov(ȳ, β̂1) always zero in simple linear regression?

It comes from the fact that ȳ and β̂1 = Σciyi are both linear combinations of independent errors, and the "weights" ci = (xi − x̄)/Sxx sum to exactly zero by construction. When you compute the covariance, it reduces to σ² times Σci, and that sum is zero regardless of what the actual x-values are, so the result holds for any data set with a fixed-x design.

Is Cov(y, β̂1) always exactly zero for a specific observation?

No — it equals (xi − x̄)σ²/Sxx, so it is zero only for the one observation (if any) whose xi happens to equal x̄ exactly. For every other observation it is nonzero, which is why the question's statement 1, read as a general claim about an individual y, is correct.

Why does the least squares line always pass through (x̄, ȳ)?

This falls directly out of the normal equations used to derive the OLS estimators. One of the two normal equations is Σ(yi − β̂0 − β̂1xi) = 0, and dividing through by n gives ȳ − β̂0 − β̂1x̄ = 0, which rearranges to β̂0 = ȳ − β̂1x̄. Plugging x = x̄ into the fitted line ŷ = β̂0 + β̂1x then gives ŷ = ȳ, confirming the point (x̄, ȳ) always lies on the line.

What is the variance of β̂1, and how is it used in statement 2?

Var(β̂1) = σ²/Sxx, obtained the same way we found Cov(yi, β̂1) in Step 4, except now you take the covariance of β̂1 with itself instead of with yi. This exact quantity, written as V(β̂1), is what statement 2 multiplies by −x̄ to get Cov(β̂0, β̂1).

Are questions like this common in ISS Paper-2?

Yes. Linear model theory — properties of OLS estimators, ANOVA decompositions, and distributional results for regression coefficients — is one of the heaviest-weighted areas across the papers we have solved in this series so far, and it tends to reward exactly this kind of careful, derive-it-yourself approach rather than rote formula recall.

What is the quickest way to avoid falling for the statement 4 trap in future questions?

Get into the habit of writing out the correct formula from memory first, on your rough sheet, before you even look closely at the option or statement given in the paper — then compare character by character. If you compare from memory against the printed statement rather than reading the printed statement in isolation, a missing bar, a flipped sign, or a swapped subscript jumps out immediately instead of blending in.

If any step above felt rushed, drop your doubt in the comments and we will walk through it further — and if you found this useful, do pass it along to a fellow ISS aspirant working through the same paper.