ST5224: Advanced Statistical Theory II - The Department of Statistics ...

ST5224: Advanced Statistical Theory II 

Chen Zehua 

Department of Statistics & Applied Probability 

Thursday, February 16, 2012 

Chen Zehua 

ST5224: Advanced Statistical Theory II

Lecture 9: Unbiased tests and UMPU 

Unbiased tests 

When a UMP test does not exist, we may use the same approach 

used in estimation problems, i.e., imposing a reasonable restriction 

on the tests. One such restriction is unbiasedness. 

A UMP test T of size α has the property that 

β T (P) ≤ α, P ∈ P 0 and β T (P) ≥ α, P ∈ P 1 , (1) 

since T is at least as good as the silly test T ≡ α. This leads to 

the following definition. 

Definition 6.3 

Let α be a given level of significance. A test T for H 0 : P ∈ P 0 

versus H 1 : P ∈ P 1 is said to be unbiased of level α if and only if 

(1) holds. A test of size α is called a uniformly most powerful 

unbiased (UMPU) test if and only if it is UMP within the class of 

unbiased tests of level α. 

The consideration of UMPU tests can be confined to the tests 

based on a sufficient statistics. Why? 

Chen Zehua 


UMPU tests in exponential families 

Consider the following hypotheses: 

H 0 : θ ∈ Θ 0 versus H 1 : θ ∈ Θ 1 , 

where θ = θ(P) is a functional from P onto Θ and Θ 0 and Θ 1 are 

disjoint and Θ 0 ∪ Θ 1 = Θ. (P j = {P : θ ∈ Θ j }, j = 0, 1.) 

Definition 6.4 (Similarity) 

Consider the hypotheses H 0 : θ ∈ Θ 0 vs H 1 : θ ∈ Θ 1 . 

Let α be a given level of significance and let ¯Θ 01 be the common 

boundary of Θ 0 and Θ 1 , i.e., the set of points θ that are points or 

limit points of both Θ 0 and Θ 1 . 

A test T is similar on ¯Θ 01 if and only if β T (P) = α for all θ ∈ ¯Θ 01 . 

Remark 

It is more convenient to work with similarity than to work with 

unbiasedness for testing H 0 : θ ∈ Θ 0 vs H 1 : θ ∈ Θ 1 . 

Chen Zehua 


Continuity of the power function 

For a given test T , the power function β T (P) is said to be 

continuous in θ if and only if for any {θ j : j = 0, 1, 2, ...} ⊂ Θ, 

θ j → θ 0 implies β T (P j ) → β T (P 0 ), where P j ∈ P satisfying 

θ(P j ) = θ j , j = 0, 1,.... If β T is a function of θ, then this 

continuity property is simply the continuity of β T (θ). 

Lemma 6.5 

Consider hypotheses H 0 : θ ∈ Θ 0 vs H 1 : θ ∈ Θ 1 . Suppose that, for 

every T , β T (P) is continuous in θ. If T ∗ is uniformly most 

powerful among all similar tests and has size α, then T ∗ is a 

UMPU test. 

Proof 

Under the continuity assumption on β T , the class of similar tests 

contains the class of unbiased tests. 

Since T ∗ is uniformly at least as powerful as the test T ≡ α, T ∗ is 

unbiased. 

Hence, T ∗ is a UMPU test. 

Chen Zehua 


Neyman structure 

Let U(X ) be a sufficient statistic for P ∈ ¯P = {P : θ ∈ ¯Θ 01 } and 

let ¯P U be the family of distributions of U as P ranges over ¯P. 

A test is said to have Neyman structure w.r.t. U if 

E[T (X )|U] = α a.s. ¯P U , 

Clearly, if T has Neyman structure, then 

E[T (X )] = E{E[T (X )|U]} = α 

P ∈ ¯P, 

i.e., T is similar on ¯Θ 01 . 

If all tests similar on ¯Θ 01 have Neyman structure w.r.t. U, then 

working with tests having Neyman structure is the same as working 

with tests similar on ¯Θ 01 . 

Lemma 6.6 

Let U(X ) be a sufficient statistic for P ∈ ¯P. 

A necessary and sufficient condition for all tests similar on ¯Θ 01 to 

have Neyman structure w.r.t. U is that U is boundedly complete 

for P ∈ ¯P. 

Chen Zehua 



(i) Suppose first that U is boundedly complete for P ∈ ¯P. 

Let T (X ) be a test similar on ¯Θ 01 . 

Then E[T (X ) − α] = 0 for all P ∈ ¯P. 

From the boundedness of T (X ), E[T (X )|U] is bounded. 

Since E{E[T (X )|U] − α} = E[T (X ) − α] = 0 for all P ∈ ¯P and U 

is boundedly complete, E[T (X )|U] = α a.s. ¯P U , i.e., T has 

Neyman structure. 

(ii) Suppose now that all tests similar on ¯Θ 01 have Neyman 

structure w.r.t. U. 

Suppose also that U is not boundedly complete for P ∈ ¯P. 

Then there is a function h such that |h(u)| ≤ C, E[h(U)] = 0 for 

all P ∈ ¯P, and h(U) ≠ 0 with positive probability for some P ∈ ¯P. 

Let T (X ) = α + ch(U), where c = min{α, 1 − α}/C. 

Then T is a test similar on ¯Θ 01 but T does not have Neyman 

structure w.r.t. U (because h(U) ≠ 0). 

Thus, U must be boundedly complete for P ∈ ¯P. 

This proves the result. 

Chen Zehua 


Theorem 6.4 (UMPU tests in exponential families) 

Suppose that X has the following p.d.f. w.r.t. a σ-finite measure: 

f θ,ϕ (x) = exp {θY (x) + ϕ τ U(x) − ζ(θ, ϕ)} , 

where θ is a real-valued parameter, ϕ is a vector-valued parameter, 

and Y (real-valued) and U (vector-valued) are statistics. 

(i) For testing H 0 : θ ≤ θ 0 versus H 1 : θ > θ 0 , a UMPU test of size 

⎧ 

α is 

⎨ 1 Y > c(U) 

T ∗ (Y , U) = γ(U) Y = c(U) 

⎩ 

0 Y < c(U), 

where c(u) and γ(u) are Borel functions determined by 

E θ0 [T ∗ (Y , U)|U = u] = α for every u 

and E θ0 is the expectation w.r.t. f θ0 ,ϕ. 

(ii) For testing H 0 : θ ≤ θ 1 or θ ≥ θ 2 versus H 1 : θ 1 < θ < θ 2 , a 

UMPU test of size α⎧ 

is 

⎨ 1 c 1 (U) < Y < c 2 (U) 

T ∗ (Y , U) = γ 

⎩ i (U) Y = c i (U), i = 1, 2, 

0 Y < c 1 (U) or Y > c 2 (U), 

Chen Zehua 


where c i (u)’s and γ i (u)’s are Borel functions determined by 

E θ1 [T ∗ (Y , U)|U = u] = E θ2 [T ∗ (Y , U)|U = u] = α for every u. 

(iii) For testing H 0 : θ 1 ≤ θ ≤ θ 2 versus H 1 : θ < θ 1 or θ > θ 2 , a 

UMPU test of size α is 

⎧ 

⎨ 1 Y < c 1 (U) or Y > c 2 (U) 

T ∗ (Y , U) = γ 

⎩ i (U) Y = c i (U), i = 1, 2, 

0 c 1 (U) < Y < c 2 (U), 

where c i (u)’s and γ i (u)’s are Borel functions determined by 

E θ1 [T ∗ (Y , U)|U = u] = E θ2 [T ∗ (Y , U)|U = u] = α for every u. 

(iv) For testing H 0 : θ = θ 0 versus H 1 : θ ≠ θ 0 , a UMPU test of 

size α is given by T ∗ (Y , U) in (iii), where c i (u)’s and γ i (u)’s are 

Borel functions determined by 

and 

E θ0 [T ∗ (Y , U)|U = u] = α for every u 

E θ0 [T ∗ (Y , U)Y |U = u] = αE θ0 (Y |U = u) for every u. 

Chen Zehua 



By sufficiency, we only need to consider tests that are functions of 

(Y , U). 

It follows from Theorem 2.1(i) that the p.d.f. of (Y , U) (w.r.t. a 

σ-finite measure) is in a natural exponential family of the form 

exp {θy + ϕ τ u − ζ(θ, ϕ)} and, given U = u, the p.d.f. of the 

conditional distribution of Y (w.r.t. a σ-finite measure ν u ) is in a 

natural exponential family of the form exp {θy − ζ u (θ)}. 

Hypotheses in (i)-(iv) are of the form H 0 : θ ∈ Θ 0 vs H 1 : θ ∈ Θ 1 

with ¯Θ 01 = {(θ, ϕ) : θ = θ 0 } or = {(θ, ϕ) : θ = θ i , i = 1, 2}. 

In case (i) or (iv), U is sufficient and complete for P ∈ ¯P and, 

hence, Lemma 6.6 applies. 

In case (ii) or (iii), applying Lemma 6.6 to each {(θ, ϕ) : θ = θ i } 

also shows that working with tests having Neyman structure is the 

same as working with tests similar on ¯Θ 01 . 

By Theorem 2.1, the power functions of all tests are continuous 

and, hence, Lemma 6.5 applies. 

Chen Zehua 


Thus, for (i), it suffices to show T ∗ is UMP among all tests T 

satisfying 

E θ0 [T (Y , U)|U = u] = α for every u (2) 

and for part (ii) or (iii)), it suffices show T ∗ is UMP among all 

tests T satisfying 

E θ1 [T (Y , U)|U = u] = E θ2 [T (Y , U)|U = u] = α for every u. 

For (iv), any unbiased T should satisfy (2) and 

∂ 

∂θ E θ,ϕ[T (Y , U)] = 0, θ ∈ ¯Θ 01 . (3) 

One can show (exercise) that (3) is equivalent to 

E θ,ϕ [T (Y , U)Y − αY ] = 0, θ ∈ ¯Θ 01 . (4) 

Using the argument in the proof of Lemma 6.6, one can show 

(exercise) that (4) is equivalent to 

E θ0 [T (Y , U)Y |U = u] = αE θ0 (Y |U = u) for every u. (5) 

Hence, for (iv), it suffices to show T ∗ is UMP among all tests T 

satisfying (2) and (5). 

Chen Zehua 


Note that the power function of any test T (Y , U) is 

∫ [∫ 

] 

β T (θ, ϕ) = T (y, u)dP Y |U=u (y) dP U (u). 

Thus, it suffices to show that for every fixed u and θ ∈ Θ 1 , T ∗ 

maximizes 

∫ 

T (y, u)dP Y |U=u (y) 

over all T subject to the given side conditions. 

Since P Y |U=u is in a one-parameter exponential family, the results 

in (i) and (ii) follow from Corollary 6.1 and Theorem 6.3, 

respectively. The result in (iii) follows from Theorem 6.3(ii) by 

considering 1 − T ∗ . 

To prove the result in (iv), it suffices to show that if Y has the 

p.d.f. given by exp {θy − ζ u (θ)} and if u is treated as a constant in 

(2) and (5), T ∗ in (iii) with a fixed u is UMP subject to conditions 

(2) and (5). We now omit u in the following proof for (iv), which 

is very similar to the proof of Theorem 6.3. 

Chen Zehua 


First, (α, αE θ0 (Y )) is an interior point of the set of points 

(E θ0 [T (Y )], E θ0 [T (Y )Y ]) as T ranges over all tests of the form 

T (Y ). By Lemma 6.2 and Proposition 6.1, for testing θ = θ 0 

versus θ = θ 1 , the UMP test is equal to 1 when 

(k 1 + k 2 y)e θ 0y < C(θ 0 , θ 1 )e θ 1y , 

where k i ’s and C(θ 0 , θ 1 ) are constants. This inequality is 

equivalent to 

a 1 + a 2 y < e by 

for some constants a 1 , a 2 , and b. This region is either one-sided or 

the outside of an interval. By Theorem 6.2(ii), a one-sided test has 

a strictly monotone power function and therefore cannot satisfy 

(5). Thus, this test must have the form of T ∗ in (iii). Since T ∗ in 

(iii) does not depend on θ 1 , by Lemma 6.1, it is UMP over all tests 

satisfying (2) and (5); in particular, the test ≡ α. Thus, T ∗ is 

UMPU. 

Finally, it can be shown that all the c- and γ-functions in (i)-(iv) 

are Borel functions of u (see Lehmann (1986, p. 149)). 

Chen Zehua 


Example 6.11 

A problem arising in many different contexts is the comparison of 

two treatments. 

If the observations are integer-valued, the problem often reduces to 

testing the equality of two Poisson distributions (e.g., a comparison 

of the radioactivity of two substances or the car accident rate in 

two cities) or two binomial distributions (when the observation is 

the number of successes in a sequence of trials for each treatment). 

Consider first the Poisson problem in which X 1 and X 2 are 

independently distributed as the Poisson distributions P(λ 1 ) and 

P(λ 2 ), respectively. 

The p.d.f. of X = (X 1 , X 2 ) is 

[e −(λ 1+λ 2 ) /x 1 !x 2 !] exp {x 2 log(λ 2 /λ 1 ) + (x 1 + x 2 ) log λ 1 } 

w.r.t. the counting measure on 

{(i, j) : i = 0, 1, 2, ..., j = 0, 1, 2, ...}. 

Chen Zehua 


Example 6.11 (continued) 

Let θ = log(λ 2 /λ 1 ). 

Then hypotheses such as λ 1 = λ 2 and λ 1 ≥ λ 2 are equivalent to 

θ = 0 and θ ≤ 0, respectively. 

The p.d.f. of X is in a multiparameter exponential family with 

ϕ = log λ 1 , Y = X 2 , and U = X 1 + X 2 . 

Thus, Theorem 6.4 applies. 

To obtain various tests in Theorem 6.4, it is enough to derive the 

conditional distribution of Y = X 2 given U = X 1 + X 2 = u. 

Using the fact that X 1 + X 2 has the Poisson distribution 

P(λ 1 + λ 2 ), one can show that 

( u 

P(Y = y|U = u) = p 

y) 

y (1−p) u−y I {0,1,...,u} (y), u = 0, 1, 2, ..., 

where p = λ 2 /(λ 1 + λ 2 ) = e θ /(1 + e θ ). 

This is the binomial distribution Bi(p, u). 

On the boundary set ¯Θ 01 , θ = θ j (a known value) and the 

distribution P Y |U=u is known. 

Chen Zehua 



Consider next the binomial problem in which X j , j = 1, 2, are 

independently distributed as the binomial distributions Bi(p j , n j ), 

j = 1, 2, respectively, where n j ’s are known but p j ’s are unknown. 

The p.d.f. of X = (X 1 , X 2 ) is 

( )( ) 

n1 n2 

{ 

(1−p 1 ) n 1 

(1−p 2 ) n 2 

exp x 2 log p 2(1−p 1 ) 

x 1 x 

p 1 (1−p 2 ) + (x 1 + x 2 ) log 

2 

w.r.t. the counting measure on 

{(i, j) : i = 0, 1, ..., n 1 , j = 0, 1, ..., n 2 }. 

This p.d.f. is in a multiparameter exponential family with 

θ = log p 2(1 − p 1 ) 

p 1 (1 − p 2 ) , Y = X 2, and U = X 1 + X 2 . 

Thus, Theorem 6.4 applies. 

Note that hypotheses such as p 1 = p 2 and p 1 ≥ p 2 are equivalent 

to θ = 0 and θ ≤ 0, respectively. 

} 

p 1 

(1−p 1 ) 

Chen Zehua 



Using the joint distribution of (X 1 , X 2 ), one can show (exercise) 

that 

( )( ) 

n1 n2 

P(Y = y|U = u) = K u (θ) 

e θy I A (y), u = 0, 1, ..., n 1 +n 2 , 

u − y y 

where 

A = {y : y = 0, 1, ..., min{u, n 2 }, u − y ≤ n 1 } 

and 

K u (θ) = 

⎡ 

⎤ 

⎣ ∑ ( )( ) 

−1 

n1 n2 

e θy ⎦ . 

u − y y 

y∈A 

If θ = 0, this distribution reduces to a known distribution: the 

hypergeometric distribution HG(u, n 2 , n 1 ) (Table 1.1, page 18). 

Chen Zehua

ST5224: Advanced Statistical Theory II - The Department of Statistics ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?