ISS 2017 Statistics Paper-2 Solution: Question 10 (Hat Matrix Symmetry and Idempotency)
This post continues our question-by-question walkthrough of the ISS 2017 Statistics Paper-2 objective paper, picking up right after the last solution in the series. Today's question pulls us into the algebra of the general linear model — specifically the famous hat matrix that sits underneath almost every regression computation you will ever do. If phrases like 'generalized inverse' or 'idempotent matrix' still feel like things to memorize rather than understand, this question is a good place to make them concrete.
Quick Summary
Topic:Generalized inverses, the hat (projection) matrix, and core properties used throughout linear model theory
Question Reference:ISS 2017, Statistics Paper-2, Question 10
Correct Answer:Option (c) — Statements 1, 2 and 3 are correct
Question 10.Consider a model Y =X β + ε, where Y is n×1,X is n×p and β is p×1. Let H =X (X T X)−X T. Which of the following are correct?
(In − H) is symmetric
(In − H) is idempotent
(In − H)X = 0
(In − H)X ≠ 0
Select the correct answer using the code given below.
(a) 1, 2 and 4
(b) 2 and 3 only
(c) 1, 2 and 3
(d) 1 and 3 only
Step 1: Understanding What H Actually Is
In the model Y = Xβ + ε, X need not have full column rank — this is common in ANOVA-type linear models where columns of X can be linearly dependent (think of indicator/dummy-coded group variables). When that happens, XTX is a square but singular matrix, meaning it has no ordinary inverse. So instead of (XTX)−1, we use a generalized inverse (also called a g-inverse), written (XTX)−. By definition, if we call A = XTX and G = A− its generalized inverse, the one defining property we are guaranteed is:
A G A = A, i.e. (XTX) G (XTX) = XTX
That single equation is the only thing we are allowed to assume about G — everything else about H = XGXT has to be derived from it. This is exactly why the question is testing careful derivation rather than a memorized formula.
Step 2: The Key Lemma — XGXTX = X
Everything in this question hinges on one algebraic fact: multiplying H by X on the right just gives back X. In symbols, HX = X, i.e. XGXTX = X. Let's actually prove this instead of asserting it.
Define the matrix M = XGA − X (recall A = XTX). We'll show M = 0 by computing MTM and showing it equals the zero matrix. This is a classic trick: for any real matrix M, if MTM = 0, then M itself must be 0. Why? Because the (i, i) diagonal entry of MTM is exactly the sum of squares of the entries in column i of M. If that sum of squares is zero, every entry in that column must individually be zero (squares of real numbers can't be negative, so they can only cancel by all being zero).
Now expand, using AT = A since XTX is always symmetric:
MTM = (AGTXT − XT)(XGA − X)= AGTXTXGA − AGTXTX − XTXGA + XTX= AGTAGA − AGTA − AGA + A (substituting XTX = A)
Now use the one fact we're given, AGA = A, to simplify the first and third terms:
AGTAGA = AGT(AGA) = AGTA (the bracketed AGA collapses to A)AGA = A (given directly)
Substituting back:
MTM = AGTA − AGTA − A + A = 0
So MTM = 0, which forces M = 0, which means XGA = X, i.e.
HX = X, equivalently (In − H)X = 0
Pro Tip: This 'show M, prove MTM = 0' trick is one of the most reusable moves in linear model algebra — it turns an otherwise messy generalized-inverse identity into three or four lines of substitution. The way this result is usually taught is to drill this exact lemma (HX = X) until it's automatic, because once you have it, statements 1, 2 and 3 in this question fall out almost for free, which is precisely the shortcut the exam is rewarding.
Step 3: What H Does to Any Vector
The lemma above tells us something very concrete about H as a transformation. Take any vector x in the column space of X, meaning x = Xb for some vector b. Then:
Hx = XGXT(Xb) = (XGXTX)b = Xb = x
So H leaves every vector in the column space of X completely unchanged.
Now take any vector x that is orthogonal to every column of X, meaning XTx = 0. Then:
Hx = XGXTx = XG(XTx) = XG(0) = 0
So H crushes every vector orthogonal to the column space of X down to zero. Since any vector y in n-dimensional space can always be split uniquely into a part inside the column space of X and a part orthogonal to it (y = x1 + x2), these two facts completely describe what H does to every possible input.
Step 4: Proving (In − H) is Idempotent (Statement 2)
A matrix P is called idempotent if PP = P — applying it twice has the same effect as applying it once (this is exactly what you'd expect from a 'projection': projecting an already-projected point again shouldn't move it further).
Take any vector y and split it as y = x1 + x2, with x1 in the column space of X and x2 orthogonal to it. From Step 3:
Hy = Hx1 + Hx2 = x1 + 0 = x1
Apply H again:
H(Hy) = Hx1 = x1 = Hy
Since this holds for every y, we get H2 = H, i.e. H is idempotent. Now for (In − H):
(In − H)(In − H) = In − 2H + H2 = In − 2H + H = In − H
So (In − H) is idempotent too —Statement 2 is correct.
Step 5: Proving (In − H) is Symmetric (Statement 1)
A matrix is symmetric if it equals its own transpose. We'll show HT = H directly, using the same split y = x1 + x2 idea but applied to an inner product (a·b meaning aTb, the ordinary dot product).
Take any two vectors u and v, and split each one: u = u1 + u2 and v = v1 + v2, where the subscript-1 parts are in the column space of X and the subscript-2 parts are orthogonal to it. From Step 3, Hu = u1 and Hv = v1.
Compute uT(Hv) = uTv1 = (u1 + u2)Tv1 = u1Tv1 + u2Tv1. Since u2 is orthogonal to the column space and v1 lies inside it, u2Tv1 = 0. So uT(Hv) = u1Tv1.
Now compute (Hu)Tv = u1Tv = u1T(v1 + v2) = u1Tv1 + u1Tv2. Since u1 is inside the column space and v2 is orthogonal to it, u1Tv2 = 0. So (Hu)Tv = u1Tv1 as well.
So uT(Hv) = (Hu)Tv for every u and v. But (Hu)Tv = uTHTv, so uTHv = uTHTv for all u, v — and the only way two matrices can agree against every possible u and v like this is if the matrices themselves are equal, entry by entry. Hence HT = H, so H is symmetric, and therefore:
(In − H)T = InT − HT = In − H
Statement 1 is correct.
Step 6: Checking Statements 3 and 4
We already proved HX = X directly in Step 2. So:
(In − H)X = X − HX = X − X = 0
Statement 3 is correct, and since Statement 4 claims the exact opposite ((In − H)X ≠ 0),Statement 4 is false.
Final Answer
Collecting the results: Statement 1 (true), Statement 2 (true), Statement 3 (true), Statement 4 (false, it directly contradicts Statement 3). That matches exactly Statements 1, 2 and 3 — which is Option (c). This agrees fully with the official ISS 2017 answer key, which lists C as the key for Question 10 of Series A.
Why This Question Matters
The matrix H = X(XTX)−XT is not just an abstract exam object — it is literally the matrix that produces fitted values in regression: if Ŷ is the vector of fitted values, then Ŷ = HY, and the residuals are (In − H)Y. Every property you proved above — H is symmetric, H is idempotent, (In − H)X = 0 — is exactly what makes least-squares regression work the way it does: residuals are uncorrelated with the fitted model, fitting twice changes nothing, and the decomposition of total variation into regression and residual sums of squares is mathematically guaranteed rather than a coincidence. Questions like this one are really checking whether you understand regression at the matrix level rather than just the formula level, which is precisely the skill ISS Paper-2 rewards across its linear models questions.
Frequently Asked Questions
What exactly is a generalized inverse, and why can't we just use the normal inverse?
A generalized inverse G of a matrix A is any matrix satisfying AGA = A, which is a weaker requirement than being a true inverse. We need it here because XTX can be singular (non-invertible) whenever the columns of X are linearly dependent, which happens routinely in ANOVA-style models with dummy variables. The generalized inverse still lets us build a meaningful hat matrix even though no ordinary inverse exists.
What does 'idempotent' mean in plain terms?
A matrix P is idempotent if multiplying it by itself gives back the same matrix, PP = P. Geometrically this means applying the transformation twice has no extra effect beyond applying it once — exactly the behavior you'd want from something that 'projects' a point onto a subspace, since projecting an already-projected point shouldn't move it anywhere further.
Why is H called the 'hat matrix'?
It gets its name because it puts a 'hat' on Y: Ŷ = HY produces the fitted values from the actual observations. It is also called the projection matrix because it geometrically projects the observation vector Y onto the column space of X, which is the space of all vectors the model is capable of producing.
Does the specific choice of generalized inverse change the final H matrix?
No. Even though a singular matrix can have more than one valid generalized inverse, the resulting H = XGXT comes out identical no matter which valid G you pick. This is because H is completely determined by what it does to the column space of X and its orthogonal complement, as shown in Step 3, and that behavior never depends on which particular G was used.
How does this result actually get used in real statistical analysis?
This exact machinery underlies regression diagnostics. The diagonal entries of H are called 'leverage values' and tell you how much influence each observation has on its own fitted value, while (In − H) is used to compute residuals and residual sums of squares, which drive F-tests and R² calculations in ANOVA tables.
Is there a faster way to answer this in the exam without writing out every proof?
Yes — once you recognize H as the hat/projection matrix, you can recall as standard facts that it is always symmetric, always idempotent, and always satisfies HX = X. Students who drill these three facts until they're automatic can eliminate options (a), (b) and (d) almost instantly just by spotting that option (d)'s inclusion of Statement 4 directly contradicts Statement 3, without redoing any algebra under time pressure.
If anything in the derivation above felt unclear, especially the symmetry proof in Step 5, drop a comment below with the exact step you're stuck on — and if this helped, consider sharing it with a fellow ISS aspirant who's working through the same linear models unit.

Comments