ISS 2016 Statistics Paper-2 Solution: Question 2 (Distribution of a Quadratic Form)

Continuing our question-by-question walk through the ISS Statistics Paper-2 archive, this post picks up right where the last one left off. Today we tackle the second question from the 2016 paper, which pulls together three ideas that show up again and again in this exam: multivariate normal vectors, idempotent matrices, and quadratic forms. If you found the algebra intimidating on first read, don't worry — every step is written out in full below.

Quick Summary

• Topic: Distribution of a quadratic form in independent standard normal variates

• Question Reference: ISS 2016, Statistics Paper-2, Question 2

• Correct Answer: (b) Chi-square distribution

The Question, As Asked in the Paper

Let yi ~ N(0,1) where i = 1, 2, 3, ... n be n independent standard normal variates, and y'Ay be a quadratic form where A is an idempotent matrix, then what will be the distribution of y'Ay?

• (a) Normal distribution

• (b) Chi-square distribution

• (c) Bivariate Normal distribution

• (d) Rectangular distribution

Step 1: Write Down What We Actually Know

We are told y1, y2, ..., yn are independent, each following a standard normal distribution, i.e., each has mean 0 and variance 1. Stack them into a column vector y = (y1, y2, ..., yn)'. Because the yi are independent standard normals, the vector y follows an n-variate normal distribution with mean vector 0 and covariance matrix equal to the identity matrix In. In shorthand: y ~ Nn(0, In).

We are also told A is an idempotent matrix, meaning A2 = A. We are asked about the distribution of the quadratic form y'Ay.

Step 2: Why We Can Treat A as Symmetric

A subtle but important point: whenever a matrix A appears only inside a quadratic form y'Ay, we are free to replace A by its symmetric part, (A + A')/2, without changing the value of the quadratic form at all. This is because y'Ay is a scalar, so y'Ay = (y'Ay)' = y'A'y, and averaging the two gives y'Ay = y'[(A + A')/2]y. So for the purpose of finding the distribution of y'Ay, we may as well assume A is symmetric. This is the standard convention followed throughout regression and ANOVA theory, and it's exactly what lets the rest of the argument go through cleanly.

Step 3: What Idempotency Tells Us About the Eigenvalues

Since A is symmetric, it can be diagonalized by an orthogonal matrix. That is, we can write A = PDP', where P is an orthogonal matrix (P'P = PP' = I) whose columns are the eigenvectors of A, and D is a diagonal matrix holding the eigenvalues of A.

Now use A2 = A. Substituting A = PDP' gives PDP'PDP' = PDP', and since P'P = I this simplifies to PD2P' = PDP', which forces D2 = D. For a diagonal matrix, this means every diagonal entry λ satisfies λ2 = λ, i.e., λ(λ − 1) = 0. So every eigenvalue of A is either exactly 0 or exactly 1 — there is no other possibility for a symmetric idempotent matrix.

Let r denote the number of eigenvalues equal to 1 (equivalently, r = rank(A) = trace(A), a standard fact for idempotent matrices — trace equals the sum of eigenvalues, and here that sum only counts the 1's).

Step 4: Transform the Normal Vector Using P

Define a new vector z = P'y. Since y ~ Nn(0, In) and z is just a linear transformation of y, z is also multivariate normal, with mean P'·0 = 0 and covariance matrix P'(In)P = P'P = In (using orthogonality of P). So z ~ Nn(0, In) as well — in other words, z1, z2, ..., zn are themselves independent standard normal variates. This is the rotational invariance property of the multivariate standard normal distribution: rotating (or reflecting) a standard normal vector with an orthogonal matrix produces another standard normal vector with exactly the same independent structure.

Step 5: Rewrite the Quadratic Form in Terms of z

Now substitute back:

y'Ay = y'(PDP')y = (P'y)'D(P'y) = z'Dz

Since D is diagonal with r ones and (n − r) zeros, z'Dz simply picks out the squares of the r coordinates of z that correspond to the eigenvalue-1 slots. Relabelling those coordinates z1, ..., zr for convenience:

y'Ay = z12 + z22 + ... + zr2

Step 6: Recognize the Chi-Square Distribution

By definition, the sum of squares of r independent standard normal variates is exactly what the chi-square distribution with r degrees of freedom describes. So:

y'Ay ~ χ²r, where r = rank(A)

This confirms option (b), Chi-square distribution, as the correct answer — and it matches the official answer key for this question, which also records the answer as B.

Pro Tip: The step students most often stumble on isn't the algebra — it's trusting that the orthogonal transformation z = P'y genuinely preserves the "independent standard normal" property. It's tempting to think you need to know the actual eigenvectors of A to proceed, but you never do. The whole argument works for

Final Answer

The correct option is (b) Chi-square distribution, with r = rank(A) degrees of freedom. This agrees with the official ISS 2016 Statistics Paper-2 answer key, which lists B for Question 2.

Why This Question Matters

This isn't just an abstract exercise — it's the theoretical backbone of ANOVA and regression theory. Every F-test you'll ever run in linear models relies on writing sums of squares as quadratic forms in symmetric idempotent matrices and then invoking exactly this result to know their distribution is chi-square. Students who drill this derivation until the eigenvalue argument becomes automatic tend to move through the analysis-of-variance and linear-models sections of the syllabus much faster, because half of those proofs are really just this same five-line argument dressed up in different notation.

Frequently Asked Questions

Why must A be symmetric for this chi-square result to hold?

The eigen-decomposition A = PDP' with an orthogonal P and real diagonal D is only guaranteed by the spectral theorem when A is symmetric. Without symmetry, A could have complex eigenvalues or a non-orthogonal eigenbasis, and the clean "sum of squared independent normals" argument would break down. Since y'Ay only ever depends on the symmetric part of A anyway, this restriction costs us nothing.

What exactly does "idempotent" mean, in plain terms?

A matrix A is idempotent if multiplying it by itself gives A back again, i.e., A² = A. Geometrically, idempotent matrices represent projections — applying the projection twice has the same effect as applying it once, because after the first application you're already "there."

How do we know the degrees of freedom equal rank(A) and not something else?

The degrees of freedom count how many independent squared normal terms survive in z'Dz, and that count is exactly the number of eigenvalues of A equal to 1. For a symmetric idempotent matrix, that count equals both the rank and the trace of A, so any of these three quantities can be used interchangeably to find r.

Would the result change if the yi had a nonzero common mean instead of mean 0?

Yes — if y ~ Nn(μ, In) with μ ≠ 0, then y'Ay follows a noncentral chi-square distribution with r degrees of freedom and noncentrality parameter μ'Aμ. The central chi-square case in this question is the special case where μ = 0, which makes the noncentrality parameter vanish.

Where will I actually use this result later in the syllabus?

This exact argument underlies Cochran's theorem, which is what lets you decompose the total sum of squares in ANOVA into independent chi-square pieces (between-group and within-group), and it's also what justifies the chi-square distribution of the residual sum of squares in linear regression.

Is there a quicker way to identify the answer without doing the full derivation in the exam?

Yes, once you've internalized the result: whenever you see "quadratic form in independent standard normal variates" plus "idempotent matrix" in the same question, the answer is almost always chi-square, with degrees of freedom equal to the rank (or trace) of that matrix. It's still worth being able to reconstruct the derivation quickly, though, since some variants of this question ask for the degrees of freedom itself rather than just the distribution family.

If any step above still feels unclear, drop a comment below — working through where exactly the confusion starts usually clears it up faster than re-reading the derivation alone. And if this helped, consider sharing it with a fellow ISS aspirant who might be stuck on the same question.