ISS 2016 Statistics Paper-2 Solution: Question 22 (Randomised Tests for Discrete Distributions)
Continuing our question-by-question walk through the ISS Statistics Paper-2 archive, we now arrive at a pure testing-of-hypotheses theory question from the 2016 paper. Unlike the numerical questions in this series, this one asks you to reason carefully about why a certain mathematical fact is true rather than to crunch numbers — so we will build up the reasoning slowly, with a concrete example to anchor the abstract argument.
Quick Summary
Topic: Testing of Hypotheses — why non-randomised tests can fail to attain an exact significance level when the parent distribution is discrete
Question Reference: ISS 2016, Statistics Paper-2, Question 22
Our worked-out answer: Option (a) — both statements are true and Statement-II correctly explains Statement-I
Official answer key for this booklet series: Option (c) — we explain the discrepancy below
The Question, Exactly As Set
Consider the following statements:
Statement-I:Where the parent distribution is discrete, the non-randomised tests may not exactly attain a given level of significance α.
Statement-II:α usually has a low value like 0.01 or 0.05. Even inclusion of any single discrete point with positive probability in the critical region often makes the level greater than the prescribed value.
Which one of the following is correct in respect of the above two statements?
(a) Both Statement-I and Statement-II are true and Statement-II is the correct explanation for Statement-I
(b) Both Statement-I and Statement-II are true but Statement-II is not the correct explanation for Statement-I
(c) Statement-I is true but Statement-II is false
(d) Statement-I is false but Statement-II is true
Step 1: What Does "Level of a Test" Actually Mean?
Before touching the two statements, fix the basic setup. Suppose we are testing a null hypothesis H0 using a test statistic T, and we reject H0 when T falls into some critical (rejection) region C. The size of the test is defined as
P(T ∈ C | H0 is true).
When we say we want "a test of level α", we mean we want to choose C so that this probability equals α — exactly, not approximately. That word "exactly" is the whole crux of this question.
When T has a continuous distribution under H0, this is easy: the cumulative distribution function (CDF) of T takes every value between 0 and 1 continuously, so for any α between 0 and 1 you can always find a cut-off point c such that P(T > c) = α exactly. There are no "gaps."
Step 2: What Changes When the Distribution Is Discrete?
Now suppose T can only take a discrete set of values, say t1 < t2 < t3 < …, each with its own probability under H0. Any critical region we build by a non-randomised rule (that is, a rule that says "reject if T falls in this fixed set of values, don't reject otherwise" with no coin-flipping involved) has a size that is always a sum of some of these point probabilities.
Since there are only countably many such sums, the set of sizes that a non-randomised test can actually achieve is itself a discrete, "gappy" set of numbers between 0 and 1. If our target α (say 0.05) does not happen to be exactly one of these attainable sums, then no non-randomised test can have size exactly equal to α— the best we can do is get close, either just below or just above it. This is precisely what Statement-I is asserting, and it is a completely standard and correct fact in testing theory.
Step 3: Working Through a Concrete Example
Abstract arguments are easy to nod along to and easy to misremember, so let's build an actual example by hand.
Let X ~ Binomial(n = 10, p = 0.5) under H0, and suppose we want to test H0: p = 0.5 against H1: p > 0.5 using a critical region of the form "reject if X is large", at level α = 0.05.
First compute the individual point probabilities P(X = k) = C(10, k) × (0.5)10, using (0.5)10 = 1/1024 ≈ 0.0009766:
P(X = 10) = 1 × 0.0009766 = 0.0009766
P(X = 9) = 10 × 0.0009766 = 0.0097656
P(X = 8) = 45 × 0.0009766 = 0.0439453
P(X = 7) = 120 × 0.0009766 = 0.1171875
Now build up the critical region point by point, starting from the extreme tail, and track the cumulative size:
C = {X = 10}: size = 0.0009766
C = {X ≥ 9}: size = 0.0009766 + 0.0097656 = 0.0107422
C = {X ≥ 8}: size = 0.0107422 + 0.0439453 = 0.0546875
C = {X ≥ 7}: size = 0.0546875 + 0.1171875 = 0.1718750
Look closely at what just happened between the second and third rows. The region {X ≥ 9} has size 0.0107, comfortably below our target of 0.05. To get closer to 0.05 we add exactly one more discrete point, X = 8, to the critical region. But that single point carries a probability of 0.0439 — larger than the entire gap we were trying to close — and the size jumps straight past 0.05 to 0.0547. There is no non-randomised critical region built from these point masses whose size is exactly 0.05: 0.0107 undershoots it, 0.0547 overshoots it, and nothing in between is achievable without splitting a point probabilistically.
Pro Tip: The trickiest part of this question is not the theory statement itself but resisting the urge to treat "the level of a test" as something you can always dial to any value you like, the way you can with a continuous statistic. The moment the test statistic is discrete, you are constrained to a fixed menu of achievable sizes — and the standard way this is taught is to actually list out a small binomial or Poisson example like the one above and watch the cumulative probabilities jump past the target, rather than just memorising the sentence "discrete distributions can't attain exact levels." Once you have done this by hand once, the statement stops being an abstract fact and becomes something you can reconstruct in thirty seconds during the exam.
Step 4: Does Statement-II Correctly Explain Statement-I?
Statement-II makes two claims: (i) α is usually a small number like 0.01 or 0.05, and (ii) including even one more discrete point in the critical region often pushes the size past that small target.
Our worked example demonstrates exactly this mechanism: moving from {X ≥ 9} to {X ≥ 8} added a single point of probability 0.0439 and carried the size from 0.0107 (below 0.05) to 0.0547 (above 0.05). Because α is small and individual point probabilities near the tail are often not-so-small in comparison, this "overshoot on the very next point" behaviour is common — which is the reason, not merely a coincidental fact alongside it, that a non-randomised test cannot land exactly on α. In other words, Statement-II is not just true; it is the mechanistic cause of Statement-I.
Step 5: Final Answer and a Note on the Official Key
Working through the theory and the concrete example both point to the same conclusion: Statement-I is true, Statement-II is true, and Statement-II is the correct explanation of Statement-I. That corresponds to option (a).
The official answer key circulated for this booklet series marks the answer as option (c) (Statement-I true, Statement-II false). We have rechecked the derivation carefully, including constructing the binomial example above from scratch, and we are unable to find a mathematical flaw in Statement-II as worded — it accurately describes the standard textbook justification for introducing randomised tests when the parent distribution is discrete. We are flagging this openly as a discrepancy with the official key rather than silently adopting either answer, so that readers can evaluate the reasoning themselves and reach their own conclusion.
Why This Question Matters
This question is really testing whether you understand why randomised tests exist at all — a topic that shows up repeatedly across the ISS and related statistics syllabi whenever discrete distributions meet hypothesis testing (Binomial, Poisson, or any count-based test statistic). Students who drill this kind of small-sample cumulative-probability exercise until it becomes automatic tend to spot "Statement-I/Statement-II" questions on randomised tests very quickly in the exam hall, because the underlying arithmetic is always the same three or four lines shown above, just with different distributions plugged in.
Frequently Asked Questions
What exactly is a randomised test, and how does it solve this problem?
A randomised test allows you to reject H0 with some probability strictly between 0 and 1 at one boundary point of the sample space, instead of an all-or-nothing decision. By choosing that probability carefully, you can "top up" a size that undershoots α (like our 0.0107 example) until it lands exactly on α, which a purely non-randomised rule can never do when there is a gap between achievable sizes.
Is this issue specific to the Binomial distribution?
No. The same phenomenon occurs for any discrete distribution used as a test statistic, including Poisson, Negative Binomial, or Hypergeometric-based tests. The Binomial example here is just the easiest one to compute by hand within the exam's time constraints.
Why do we even insist on hitting α exactly instead of just being close to it?
In the Neyman-Pearson framework, comparing the power of different tests (for example, to find the Uniformly Most Powerful test) is only meaningful when all the competing tests share the same size. If sizes differ slightly because of discreteness, direct power comparisons stop being fair, which is why the theory cares so much about exact levels.
In practice, do statisticians actually use randomised tests, or do they just accept an approximate level?
In most applied work, practitioners simply pick the largest non-randomised critical region whose size does not exceed α and accept a slightly conservative (smaller) actual size, since flipping a metaphorical weighted coin to decide a real-world hypothesis test is rarely acceptable to a client or regulator. Randomised tests remain important mainly as a theoretical device for proving results like the existence of Uniformly Most Powerful tests.
How should I quickly recognise this type of question in the exam?
Look for the phrase "parent distribution is discrete" together with any mention of "exact level" or "size α" — that combination is almost always pointing at this exact randomised-test justification, regardless of which specific distribution or numbers are used in the statement.
What is the single most important idea to remember from this question?
That a non-randomised test's achievable sizes form a discrete, gappy set of numbers when the underlying distribution is discrete, so hitting a small target like α = 0.05 exactly is generally impossible without randomisation — everything else in this question is really just an illustration of that one idea.
If any step above felt rushed or you worked through the binomial example and got a different number, drop a comment below with where you got stuck — and if this explanation helped, consider sharing it with a fellow ISS aspirant who is also working through the Paper-2 archive question by question.
Comments