ISS 2016 Statistics Paper-2 Solution: Question 20 (R² vs Adjusted R² Behavior)
Continuing our question-by-question walk through the ISS Statistics Paper-2 archive, we now arrive at Question 20 of the 2016 paper. If you have been following along, you already know the drill: read the question carefully, work out the underlying theory from scratch, and only then check it against the official key.
Quick Summary
Topic: Behaviour of R2 and Adjusted R2 (R̄2) when a new explanatory variable is added to a regression model
Question Reference: ISS 2016, Statistics Paper-2, Question 20
Correct Answer: Option (c) — Statements 2 and 3 only
Here is the question exactly as it appears in the test booklet:
Question 20.Consider the following statements associated with R2 and R̄2 criteria:
R2 increases with the inclusion of relevant explanatory variable only.
R2 increases whether the included explanatory variable is relevant or irrelevant.
R̄2 decreases if the included explanatory variable is irrelevant.
Which of the above statements are correct?
(a) 1 and 2 only
(b) 1 and 3 only
(c) 2 and 3 only
(d) 1, 2 and 3
Step 1: Recall what R2 actually measures
Start from the basic decomposition of variation in a linear regression. If yi is the observed value, ŷi the fitted value and ȳ the sample mean, then:
Total Sum of Squares: TSS = Σ(yi − ȳ)2
Explained Sum of Squares: ESS = Σ(ŷi − ȳ)2
Residual Sum of Squares: RSS = Σ(yi − ŷi)2
These satisfy TSS = ESS + RSS (this identity holds for any OLS regression that includes an intercept), and R2 is defined as:
R2 = ESS / TSS = 1 − RSS/TSS
Since TSS depends only on the y-values and never changes as you add or remove regressors, R2 is completely governed by what happens to RSS.
Step 2: Show that R2 can never decrease when a variable is added
This is the crux of the question, so let's reason it out carefully rather than just quoting the result.
Suppose you have a model with k regressors, and OLS chooses the coefficients that minimise RSS over all possible values of those k coefficients. Call this minimised value RSSk.
Now add one more explanatory variable, xk+1, to the model. OLS now minimises RSS over k+1 coefficients instead of k. Every combination of coefficients that was available in the smaller model is still available in the bigger model — you simply set the coefficient of the new variable xk+1 to zero, and you are back to exactly the old model with RSSk.
So the new minimisation problem searches over a strictly larger set of options that already contains the old optimum as one candidate. The new minimum, RSSk+1, therefore can be no worse than the old one:
RSSk+1 ≤ RSSk
This inequality has nothing to do with whether xk+1 is a "real" driver of y or complete statistical noise — it is a pure consequence of how OLS minimises over a larger parameter space. Since R2 = 1 − RSS/TSS and TSS is fixed, RSSk+1 ≤ RSSk immediately gives:
R2k+1 ≥ R2k
In words: R2 is mechanically non-decreasing every time you throw in another regressor, whether that regressor genuinely explains y or is pure noise like a random number column.
This directly settles Statements 1 and 2:
Statement 1 says R2 increases only for relevant variables — false, because the argument above never used relevance anywhere.
Statement 2 says R2 increases regardless of relevance — true, exactly as shown above.
Pro Tip: The trap in this question is assuming R2 and R̄2 behave the same way. R2 is a purely mechanical, non-decreasing quantity — it literally cannot tell the difference between a meaningful predictor and pure noise. The way this distinction is usually drilled into students preparing for ISS and similar exams is to always ask "does this quantity involve a penalty for the number of parameters, or not?" before deciding whether it can fall when a regressor is added. R2 has no such penalty; R̄2, as you'll see next, does.
Step 3: Bring in the Adjusted R2 (R̄2) formula
Adjusted R2 exists precisely to correct the flaw we just found in R2 — it penalises a model for using up degrees of freedom on extra regressors. With n observations and k regressors (plus an intercept, so k+1 estimated parameters), the formula is:
R̄2 = 1 − [RSS/(n−k−1)] / [TSS/(n−1)]
which can also be written in terms of R2 as:
R̄2 = 1 − (1 − R2) × (n−1)/(n−k−1)
Compare this with the plain R2 = 1 − RSS/TSS. The only difference is that RSS is divided by its degrees of freedom (n−k−1) instead of just RSS, and TSS is divided by (n−1) instead of just TSS. As k increases, the denominator (n−k−1) shrinks. That shrinking denominator works against R̄2 — it is the "penalty" for adding another parameter.
Step 4: Work out exactly when R̄2 rises and when it falls
This is the part most students skip, and it is exactly what Statement 3 is testing. There is a well-known and provable result linking the change in R̄2 to the t-statistic of the newly added variable:
R̄2 increases when a new regressor is added if and only if the absolute value of that regressor's t-statistic exceeds 1 (equivalently, its squared t-statistic exceeds 1).
You don't need to memorise a derivation to see why this is plausible: the t-statistic measures how much genuine, noise-adjusted explanatory power the new variable carries. A large |t| means the variable is pulling its weight enough to justify the lost degree of freedom; a small |t| means it isn't.
Now apply this to a genuinely irrelevant variable — one whose true population coefficient is 0, so it has no real relationship with y beyond chance correlation in the sample. Under the standard regression assumptions, its estimated t-statistic behaves like a draw from something close to a standard normal distribution centred at 0. For a standard normal variable Z, the probability that |Z| exceeds 1 is only about 32%. That means roughly two-thirds of the time, adding a truly irrelevant variable produces |t| < 1, and by the rule above, R̄2 falls.
This is the textbook justification behind Statement 3: including an irrelevant explanatory variable is expected to pull R̄2 down, even though R2 itself will have ticked up (possibly by a tiny amount) exactly as Step 2 showed. So Statement 3 is correct.
Step 5: Put it all together
Statement 1 — false (R2 rises regardless of relevance, not only for relevant variables).
Statement 2 — true (proved directly from the nested-minimisation argument in Step 2).
Statement 3 — true (justified by the t-statistic rule and the degrees-of-freedom penalty in R̄2).
So the correct combination is Statements 2 and 3 only, which is option (c).
Checking against the official ISS 2016 Statistics Paper-2 answer key (Series A): the key also lists (c)as the answer for Question 20. Our independently worked derivation matches the official key exactly.
Why This Question Matters
This question is a favourite in statistics and econometrics exams precisely because it exposes a very common misconception: that R2 is a trustworthy measure of "how good" a model is when comparing models with different numbers of predictors. Once you have internalised that R2 can be inflated simply by throwing in more variables — useful or not — you understand why practitioners lean on R̄2, and more advanced criteria like AIC or BIC, whenever model size varies. Students who drill this particular distinction until it becomes second nature rarely fall for the "just add more variables to boost R2" trap in applied problems, whether in an exam setting or in real data analysis.
Frequently Asked Questions
What is the basic difference between R2 and Adjusted R2?
R2 measures the proportion of total variation in y explained by the regressors, with no adjustment for how many regressors were used. Adjusted R2 (R̄2) applies a correction based on degrees of freedom, so it can fall when an added variable does not contribute enough explanatory power to justify the parameter it costs.
Why can R2 never decrease when you add a new explanatory variable?
Because OLS always minimises the residual sum of squares over the available coefficients, and adding a variable only enlarges the set of coefficient combinations available — the old solution (with the new variable's coefficient set to zero) is still reachable. So the new minimum RSS can only be equal to or smaller than the old one, which means R2 can only stay the same or rise.
Can R2 ever stay exactly the same after adding a variable?
Yes, in the special case where the new variable is perfectly uncorrelated with both y and the existing regressors' residual variation, RSS does not change at all and R2 stays exactly the same. In real data with continuous variables this exact case is extremely rare, so R2 almost always increases at least marginally.
How exactly does the t-statistic decide whether R̄2 rises or falls?
There is a direct algebraic link between the change in R̄2 from adding one regressor and the square of that regressor's t-statistic in the larger model. Whenever the squared t-statistic exceeds 1, R̄2 rises; whenever it is less than 1, R̄2 falls. This is why some textbooks describe R̄2 as effectively applying a very lenient "keep it only if |t| > 1" filter.
Does this mean we should never trust R2 at all?
Not quite — R2 is still useful for describing how much variation a single, fixed model explains. The caution is specifically about using R2 to compare models that have a different number of explanatory variables, which is exactly the scenario this question is testing.
Is Adjusted R2 always the right tool for choosing between models?
It is a reasonable default and far better than raw R2 for this purpose, but it isn't the only option. Criteria such as AIC and BIC impose different-sized penalties for extra parameters and are often preferred in practice, especially when comparing non-nested models.
If any step above felt unclear, drop a comment below — questions like this one are much easier to master once you have talked through the logic out loud rather than just memorised the rule. And if you know a fellow ISS aspirant who is also working through these papers, feel free to share this post with them.
Comments