27/02/2024, 23:46 Likelihood Ratio Tests

8.4.5 Likelihood Ratio Tests

So far we have focused on specific examples of hypothesis testing problems. Here, we would like to
introduce a relatively general hypothesis testing procedure called the likelihood ratio test. Before doing so,
let us quickly review the definition of the likelihood function, which was previously discussed in Section
8.2.3.

Review of the Likelihood Function:

Let X 1, X 2, X 3, . . . , X n be a random sample from a distribution with a parameter θ. Suppose that we have
observed X 1 = x 1, X 2 = x 2, ⋯, X n = x n.

- If the X i's are discrete, then the likelihood function is defined as

L(x 1, x 2, ⋯, x n; θ) = P X X ⋯ X (x 1, x 2, ⋯, x n; θ).
1 2 n

- If the X i's are jointly continuous, then the likelihood function is defined as

L(x 1, x 2, ⋯, x n; θ) = f X X ⋯ X (x 1, x 2, ⋯, x n; θ).
1 2 n

Likelihood Ratio Tests:

Consider a hypothesis testing problem in which both the null and the alternative hypotheses are simple. That
is

H 0: θ = θ 0,

H 1: θ = θ 1.

Now, let X 1, X 2, X 3, . . . , X n be a random sample from a distribution with a parameter θ. Suppose that we
have observed X 1 = x 1, X 2 = x 2, ⋯, X n = x n. One way to decide between H 0 and H 1 is to compare the
corresponding likelihood functions:

l 0 = L(x 1, x 2, ⋯, x n; θ 0), l 1 = L(x 1, x 2, ⋯, x n; θ 1).

More specifically, if l 0 is much larger than l 1, we should accept H 0. On the other hand if l 1 is much larger, we
l0
tend to reject H 0. Therefore, we can look at the ratio l to decide between H 0 and H 1. This is the idea behind
1
likelihood ratio tests.
Likelihood Ratio Test for Simple Hypotheses

Let X 1, X 2, X 3, . . . , X n be a random sample from a distribution with a parameter θ. Suppose that we have
observed X 1 = x 1, X 2 = x 2, ⋯, X n = x n. To decide between two simple hypotheses

H 0: θ = θ 0,
Processing math: 100%

https://www.probabilitycourse.com/chapter8/8_4_5_likelihood_ratio_tests.php 1/5
27/02/2024, 23:46 Likelihood Ratio Tests

H 1: θ = θ 1,

we define

L(x 1, x 2, ⋯, x n; θ 0)
λ(x 1, x 2, ⋯, x n) = .
L(x 1, x 2, ⋯, x n; θ 1)

To perform a likelihood ratio test (LRT), we choose a constant c. We reject H 0 if λ < c and accept it if λ ≥ c.
The value of c can be chosen based on the desired α.
Let's look at an example to see how we can perform a likelihood ratio test.

Example 8.30
Here, we look again at the radar problem (Example 8.23). More specifically, we observe the random variable
X:

X = θ + W,

1
where W ∼ N(0, σ 2 = 9 ). We need to decide between

H 0: θ = θ 0 = 0,

H 1: θ = θ 1 = 1.

Let X = x. Design a level 0.05 test (α = 0.05) to decide between H 0 and H 1.

Solution
1
If θ = θ 0 = 0, then X ∼ N(0, σ 2 = 9 ). Therefore,

3 9x 2
L(x; θ 0) = f X(x; θ 0) = e− 2 .
√2π
1
On the other hand, if θ = θ 1 = 1, then X ∼ N(1, σ 2 = 9 ). Therefore,

3 9 ( x − 1 )2
L(x; θ 1) = f X(x; θ 1) = e− 2 .
√2π
Therefore,

{ }
L(x; θ 0) 9x 2 9(x − 1) 2
λ(x) = = exp − +
L(x; θ 1) 2 2

= exp
{ 9(1 − 2x)
2 } .

Processing math: 100%

https://www.probabilitycourse.com/chapter8/8_4_5_likelihood_ratio_tests.php 2/5
27/02/2024, 23:46 Likelihood Ratio Tests

Thus, we accept H 0 if

exp { 9(1 − 2x)

2 } ≥ c,

where c is the threshold. Equivalently, we accept H 0 if

x≤
1
2 ( 1−
2
9 )
lnc .

1
( 2
)
Let us define c ′ = 2 1 − 9 lnc , where c ′ is a new threshold. Remember that x is the observed

value of the random variable X. Thus, we can summarize the decision rule as follows. We accept
H 0 if

X ≤ c ′.

How to do we choose c ′ ? We use the required α.

P(type I error) = P(Reject H 0 | H 0)

= P(X > c ′ | H 0)

= P(X > c ′ ) (where X ∼ N ( ))

0,
1
9

= 1 − Φ(3c ′ ).

Letting P(type I error) = α, we obtain

1
c′ = Φ − 1(1 − α).
3

Letting α = 0.05, we obtain

1
c′ = Φ − 1(.95) = 0.548
3

As we see, in this case, the likelihood ratio test is exactly the same test that we obtained in
Example 8.23.

How do we perform the likelihood ratio test if the hypotheses are not simple? Suppose that θ is an unknown
parameter. Let S be the set of possible values for θ and suppose that we can partition S into two disjoint sets
S 0 and S 1. Consider the following hypotheses:

H 0: θ ∈ S 0,

H 1: θ ∈ S 1.
Processing math: 100%

https://www.probabilitycourse.com/chapter8/8_4_5_likelihood_ratio_tests.php 3/5
27/02/2024, 23:46 Likelihood Ratio Tests

The idea behind the general likelihood ratio test can be explained as follows: We first find the likelihoods
corresponding to the most likely values of θ in S 0 and S respectively. That is, we find

l 0 = max {L(x 1, x 2, ⋯, x n; θ) : θ ∈ S 0},

l = max {L(x 1, x 2, ⋯, x n; θ) : θ ∈ S}.

(To be more accurate, we need to replace max by sup .) Let us consider two extreme cases. First, if l 0 = l,
then we can say that the most likely value of θ belongs to S 0. This indicates that we should not reject H 0. On
l0
the other hand, if l is much smaller than 1, we should probably reject H 0 in favor of H 1. To conduct a
l0 l0 l0
likelihood ratio test, we choose a threshold 0 ≤ c ≤ 1 and compare l to c. If l ≥ c, we accept H 0. If l < c,
we reject H 0. The value of c can be chosen based on the desired α.
Likelihood Ratio Tests

Let X 1, X 2, X 3, . . . , X n be a random sample from a distribution with a parameter θ. Suppose that we have
observed X 1 = x 1, X 2 = x 2, ⋯, X n = x n. Define

sup {L(x 1, x 2, ⋯, x n; θ) : θ ∈ S 0}
λ(x 1, x 2, ⋯, x n) = .
sup {L(x 1, x 2, ⋯, x n; θ) : θ ∈ S}

To perform a likelihood ratio test (LRT), we choose a constant c in [0, 1]. We reject H 0 if λ < c and accept it
if λ ≥ c. The value of c can be chosen based on the desired α.
← previous
next →

The print version of the book is available on Amazon.

Practical uncertainty: Useful Ideas in Decision-Making, Risk, Randomness, & AI

Processing math: 100%

https://www.probabilitycourse.com/chapter8/8_4_5_likelihood_ratio_tests.php 4/5
27/02/2024, 23:46 Likelihood Ratio Tests

Processing math: 100%

https://www.probabilitycourse.com/chapter8/8_4_5_likelihood_ratio_tests.php 5/5