1. Show that P (A ∩ B) ≥ P (A) + P (B) − 1.

Ans. P (A) + P (B) = P (A ∩ B) + P (A ∪ B) ≤ P (A ∩ B) + 1.

2. Show that the events whose probability is either 0 or 1 form a σ-field.
Ans. Let F := {A : P (A) = 0 or 1}. Then P (ϕ) = 0, P (Ω) = 1 =⇒
ϕ, Ω ∈ F. If P (A) = 0, P (Ac ) = 1 and vice versa, so A ∈ F =⇒ Ac ∈ F.
If disjoint sets Ai , i ≥ 1, are in F, P (∪i Ai ) = 1 if P (Ai ) = 1 for at least
one i, zero otherwise, so ∪i Ai ∈ F.
3. Let (Ω, F, P ) be a probability space and let A denote the collection of
sets A ⊂ R with the property:
(†) for any ϵ > 0, there exist a finite union Gϵ of open intervals such
that P (A∆Gϵ ) < ϵ.
Show that A is a σ-field that contains B(R). (Recall that A∆B :=
(A ∩ B c ) ∪ (B ∩ Ac ).)
4. If X, Y are real random variables, show that so are aX for a ∈ R,
X + Y , XY , and XY
I{Y ̸= 0}.
Ans. For r ∈ R, a > 0,, {aX < r} = {X < a/r} ∈ F, so aX is a
random variable. For a = 0, X ≡ 0 which is trivially a random variable.
{X + Y < r} = ∪n≥1 ∪q ({X ≤ q} ∩ {Y < r − q − n1 }) where the union
is over all rational q. Thus {X + Y < r} ∈ F.
{XY < r} = ∪n≥1 ∪q ({X < q} ∩ {Y < r/q − n1 }) where the union is
over all strictly positive rational q. Thus {XY < r} ∈ F.
{1/Y < r} = {Y ≤ 1/r}c ∈ F, so 1/Y is a random variable. Now use
the preceding claim.
5. (a) Prove the inclusion exclusion principle:
 
n
P (∪ni=1 Ai ) = (−1)k−1
X X
P (∩i∈I Ai ) .
k=1 I⊂{1,···,n},|I|=k

(b) Prove the Bonferroni inequalities:

k
P (∪ni=1 Ai ) ≤ (−1)j−1
X X
P (∩ℓ∈I Aℓ )
j=1 I⊂{1,···,n},|I|=j

1
 for odd k, 3 ≤ k ≤ n, and,
k
(∪ni=1 Ai ) (−1)j−1
X X
P ≥ P (∩ℓ∈I Aℓ )
j=1 I⊂{1,···,n},|I|=j

for even k, 2 ≤ k ≤ n.
Ans. Prove by induction.

6. Let P and Q be two probability measures defined on the same sample

space Ω and σ-algebra F.

(a) Suppose that P (A) = Q(A) for all A ∈ F with P (A) ≤ 1/2. Prove
that P = Q, i.e., that P (A) = Q(A) for all A ∈ F.

Ans. If P (A) ≤ 1/2, P (A) = Q(A). If P (A) > 1/2, P (Ac ) ≤ 1/2 and
P (Ac ) = Q(Ac ), so P (A) = 1 − P (Ac ) = 1 − Q(Ac ) = Q(A). Hence
P = Q.

(b) Give an example where P (A) = Q(A) for all A ∈ F with P (A) <
1/2, but such that P ̸= Q, i.e., that P (A) ̸= Q(A) for some A ∈ F.

Ans. Let Ω := {1, 2}, F := {ϕ, {1}, {2}, Ω} and

P ({1}) = P ({2}) = 1/2, Q({1}) = 1/3 = 1 − Q({2}).

7. Let X be an exponential random variable with parameter 1. Find the

distribution function of the random variable X 2 .

2
√ √
Ans. Since
√ X ≥ 0 a.s., P (X ≤ x) = P (X ≤ x) = 1 − P (X > x),
where x denotes the positive √square-root. Hence the distribution
function of X 2 is F (x) := 1 − e− x for x ≥ 0 and = 0 for x < 0.

8. Let X be a random variable with P(X >0) > 0. Prove that there is a
δ > 0 such that P (X ≥ δ) > 0.

2
 Ans. If P (X > δ) = 0 ∀ δ > 0, then P (X > 0) = ∪n≥1 P (X > n1 ) = 0,
a contradiction.

9. Suppose that F is the distribution function of a random variable X.

Show that the set D = {x ∈ R : F is discontinuous at x} is countable.

Ans. Since F is monotone non-decreasing, at each discontinuity x,

F (x) − F (x− ) > 0 =⇒ D = ∪n≥1 {F (x) − F (x− ) ≥ n1 }. Since F (x) −
F (x− ) ≥ n1 for at most n points, each of these sets is finite, so their
union is countable.

10. Suppose that F is the distribution function of a random variable X. If

F is a continuous function, show that the random variable Y = F(X)
has a uniform distribution on [0, 1], i.e, P(Y ≤ y) = y for all y ∈[0, 1].

Ans. Done in class.

11. Let Xn , 1 ≤ n ≤ ∞, be integrable real random variables. Show that

E [|Xn − X∞ |] → 0 =⇒ {Xn , 0 ≤ n ≤ ∞} are uniformly integrable.

Ans. It is easy to see that if X∞ is integrable, then {Xn } are u.i. ⇐⇒

{Xn −X∞ } are u.i. Thus we can take X∞ ≡ 0 without loss of generality.
Then E[|Xn |] → 0. If {Xn } are not u.i., then there exist η > 0, a(k) ↑
∞ and n(k) ↑ ∞ such that lim inf k↑∞ E[|Xn(k) |I{|Xn(k) | > a(k)}] ≥ η.
But lim inf k↑∞ E[|Xn(k) |I{|Xn(k) | > a(k)}] ≤ limk↑∞ E[|Xn(k) |] = 0, a
contradiction. Hence {Xn } are u.i.

3
 Exercise 2

1. Let Xn , n ≥ 1, be real random variables satisfying Xn+1 ≥ Xn a.s. for

all n ≥ 1. Suppose Xn → X in probability. Show that Xn → X a.s.

Ans. Since Xn → X in probability, there is a subsequence Xn(k) , k ≥ 1,

with n(k) ↑ ∞ such that Xn(k) ↑ a.s. Let n(1) = 1 without loss of
generality. Then for each m ≥ 1, there is some k such that n(k) ≤ m <
n(k + 1). Then

Xn(k) − X ≤ Xn − X ≤ Xn(k+1) − X ∀ k.

The leftmost and rightmost terms in this inequality → 0 a.s. It follows

that Xm ↑ X a.s.

2. If Xn → X in law, show that f (Xn ) → f (X) in law for all continuous f .

Ans. Let g : R 7→ R be bounded continuous. Then h : x 7→ g(f (x)) is

also bounded continuous. Since Xn → X in law, E[h(Xn )] → E[h(X)].
That is E[g(f (Xn ))] → E[g(f (X))]. Since this is true for any bounded
continuous g, f (Xn ) → f (X) in law.

3. Xn , n ≥ 1, are real random variables defined on a common probability

space and Xn → a constant C in law. Show that Xn → C in probability.

Ans. Let  > 0 and A := (C − , C + )c . Then the boundary of

A is ∂A := {C − , C + }. Letting δC denote the probability mea-
sure concentrated at the single point C and µn := the law of Xn
for n ≥ 1, we have µn → δC in P(R). Since δC (∂A) = 0, we have
P (Xn ∈ A) → P (C ∈ A) = 0. That is, P (|Xn − C| ≥ ) → 0 as
desired.

4. Recall the metric on P(Rd ) given by

d(µ, ν) := inf E[kX − Y k ∧ 1]

1
 where the infimum is over all pairs of random variables (X, Y ) such
that the law of X is µ and the law of Y is ν. Show that µn → µ in
P(Rd ) if and only if d(µn , µ) → 0.

Ans. If µn → µ, by Skorohod’s theorem there exist on some probability

space random variables {Xn , n ≥ 1, X} such that the law of Xn is µn
for all n, the law of X is µ, and furthermore, Xn → X a.s., implying
kXn − Xk ∧ 1 → a.s. So by the bounded convergence theorem,

E [kXn − Xk ∧ 1] → 0

a.s., which implies d(µn , µ) → 0. Conversely, let d(µn , µ) → 0. Let

n ↓ 0 and for each n, pick random variables Xn , Zn such that the law
of Xn is µn , the law of Zn is µ, and

E [kXn − Zn k ∧ 1] ≤ d(µn , µ) + n .

Let f ∈ Cb (Rd ) be uniformly continuous with K > 0 a bound on |f (·)|.

Given η > 0, pick δ ∈ (0, 1) such that kx−yk < δ =⇒ |f (x)−f (y)| < η.
Then
Z Z
f dµn − f dµ
= |E[f (Xn )] − E[f (Zn )]|
≤ E[|f (Xn ) − f (Zn )|I{kXn − Zn k < δ}] +
E[|f (Xn ) − f (Zn )|I{kXn − Zn k ≥ δ}]
≤ η + 2KP (kXn − Zn k ≥ δ)
≤ η + 2KP (kXn − Zn k ∧ 1 ≥ δ)
2KE[kXn − Zn k ∧ 1]
≤ η+
δ
2K(d(µn , µ) + n )
≤ η+
δ
→ η as n ↑ ∞.

f dµn → f dµ, so µn → µ.
R R
Since η > 0 is arbitrary,

5. Let Xn , Yn , n ≥ 1, be real random variables on some probability space

such that Xn → some Z in law and |Xn − Yn | → 0 in probability. Show

2
 that Yn → Z in law.

Ans. Let µn , νn , µ denote the laws of Xn , Yn , Z resp. Let d(β, γ) :=

inf E[|U − V | ∧ 1] denote the metric defined in the preceding problem
and in class, where the infimum is over all pairs of random variables
(U, V ) such that the law of U is β and the law of V is γ. Recall also
that E[|X − Y | ∧ 1] is a metric on the space of all random variables
on some probability space compatible with convergence in probability.
Then d(µn , µ) → 0. Also,

d(µn , νn ) ≤ E[|Xn − Yn | ∧ 1] → 0

because Xn − Yn → 0 in probability. Thus by triangle inequality,

d(νn , µ) ≤ d(µn , µ) + d(νn , µn ) → 0.

6. Prove Scheffe’s theorem.

Ans. Let pn (·), 1 ≤ n ≤ ∞, be probability densities such that pn (x) →

p∞ (x) for a.s. x w.r.t. the Lebesgue measure. Then
Z
0 ≤ (p∞ (x) − pn (x))+ ≤ p∞ (x), p∞ (x)dx = 1 < ∞,

together imply,R by the dominated convergence theorem for Lebesgue

measure, that (p∞ (x) − pn (x))+ dx → 0. Hence
Z Z Z
|p∞ (x) − pn (x)|dx = +
(p∞ (x) − pn (x)) dx + (p∞ (x) − pn (x))− dx
Z Z
= 2 (p∞ (x) − pn (x))+ dx − (p∞ (x) − pn (x))dx → 0 − (1 − 1) = 0.

7. If µ ∈ P(Rd ), show that there exists a sequence µn ∈ P(Rd ), n ≥ 1,

such that µn is concentrated on finitely many points for each n and
µn → µ in P(Rd ).

Ans. Let X be a random variable with law µ. By combining the

simple function approximations of X + , X − , construct simple function
approximations Xn , n ≥ 1, of X such that Xn → X pointwise. Let

3
 µn be the law of Xn for n ≥ 1. Then for f ∈ RCb (R), f (Xn ) → f (X)
a.s., hence f dµn = E[f (Xn )] → E[f (X)] = f dµ by bounded con-
R

vergence theorem. Thus µn → µ. Since µn are concentrated on finitely

many points, we are done.

8. If ϕ is a characteristic function, show that

q
|1 − ϕ(t)| ≤ 2(1 − Re(ϕ(x))).

h i
Ans. Let ϕ(t) = E eitX = E[cos(tX) + isin(tX)]. Then

|1 − ϕ(t)|2
= |1 − E[cos(tX)] + iE[sin(tX)]|2
≤ E[|1 − cos(tX) + isin(tX)|2 ]
= E[(1 − cos(tX))2 + sin2 (tX)]
= E[1 + cos2 (tX) + sin2 (tX) − 2cos(tX)]
= 2(1 − E[cos(tX)])
= 2 − (E[cos(tX) + isin(tX)] + E[cos(tX) − isin(tX)])
= 2 − (ϕ(t) + ϕ(t))
= 2(1 − Re(ϕ(t))).

Xn
9. Xn , n ≥ 1, are i.i.d. real random variables. Show that n
→ 0 a.s. if
and only if E[|Xn |] < ∞.

Ans. E[|Xn |] < ∞ ⇐⇒ E[d|Xn |e] < ∞ ⇐⇒ E[b|Xn |c] < ∞. Hence,
letting dxe, bxc denote resp. the smallest integer greater than and the
largest integer less than x, we have
∞
X ∞
X
E[|Xn |] < ∞ ⇐⇒ P (|Xn | ≥ m) < ∞ ⇐⇒ P (|Xm | ≥ m) < ∞)
m=0 m=0

∞
X
⇐⇒ P (|Xm /m| ≥ 1) < ∞.
m=0

Then
E[|Xn |] < ∞ =⇒ E[k|Xn |] < ∞ ∀ k ≥ 1 =⇒

4
 X
P (|Xm /m| ≥ 1/k) < ∞ ∀ k ≥ 1 =⇒ |Xm |/m < 1/k
m

for sufficiently large m a.s., for each k. That is, |Xn |/n → 0 a.s.
Similarly,
X
E[|Xn |] = ∞ =⇒ P (|Xm | ≥ 1) = ∞ =⇒ |Xm |/m ≥ 1 i.o.,
m

a.s. Hence P (|Xm |/m → 0) = 0.

10. (a) Let f, g : R 7→ R be both non-increasing or non-decreasing. Given

real random variables X, Y on some probability space such that the
law of (X, Y ) = the law of (Y, X), show that

E[f (X)g(X)] ≥ E[f (X)g(Y )].

(Hint: Show that (f (x) − f (y))(g(x) − g(y)) ≥ 0 ∀ x, y.)

Ans. Since f (x) ≥ f (y) ⇐⇒ g(x) ≥ g(y), we have

(f (x) − f (y))(g(x) − g(y)) ≥ 0 ∀ x, y.

Also, law of (X, Y ) = law of (Y, X) =⇒ law of X = law of Y . Hence

E [(f (X) − f (Y ))(g(X) − g(Y ))] ≥ 0

=⇒ E[f (X)g(X)] + E[f (Y )g(Y )] ≥ E[f (X)g(Y )] + E[f (Y )g(X)]
=⇒ 2E[f (X)g(X)] ≥ 2E[f (X)g(Y )],

which implies the claim.

(b) Show that E[f (X)g(X)] ≥ E[f (X)]E[g(X)].

Ans. Let (X, Y ) be independent identically distributed. By the above,

E[f (X)g(X)] ≥ E[f (X)g(Y )] = E[f (X)]E[g(Y )] = E[f (X)]E[g(X)].

5
 Exercise 3

1. Let (Xn , Fn ) be a submartingale. Show that if f is a convex increasing

function such that E[|f (Xn )|] < ∞ ∀n, then (f (Xn ), Fn ) is a sub-
martingale.

Ans. Follows from

E[f (Xn+1 )|Fn ] ≥ f (E[Xn+1 |Fn ]) ≥ f (Xn ) a.s.,

where the first inequality is the conditional Jensen’s inequality and the
second inequality follows from the submartingale property of {Xn },
viz., E[Xn+1 |Fn ] ≥ Xn a.s., and the monotone increase of f .

2. Show that if (Mn , Fn ) is a martingale and {Zn } is a bounded real valued

Pn−1
process adapted to {Fn }, then X0 = 0, Xn := m=0 Zm ∆Mm+1 , n ≥ 1,
defines an {Fn }-martingale.

Ans. E [Xn+1 − Xn |Fn ] = E [Zn ∆Mn+1 |Fn ] = Zn E [∆Mn+1 |Fn ] = 0.

The claim follows.

3. Let (Mn , Fn ), n ≥ 0, be a zero mean martingale with bounded incre-

ments. Find an increasing predictable process {Bn } such that

(exp(Mn − Bn ), Fn ), n ≥ 0,

is a martingale.
h i
n−1
log E e∆Mm |Fm−1 , m ≥ 0, with F−1 :=
P
Ans. Take Bn := m=0
{φ, Ω}, M−1 = 0. Then {Bn } is predictable and
n
e∆Mm
!
Mn −Bn
Y
Zn := e = .
m=0 E [e∆Mm |Fm−1 ]

1
 Therefore
e∆Mn+1
" #
E [Zn+1 |Fn ] = Zn E Fn = Zn .
E [e∆Mn+1 |Fn ]

Also, by the conditional Jensen’s inequality,

h i
h i E ∆Mn+1 |Fn
log E e∆Mn+1 |Fn ≥ log e = E [∆Mn+1 |Fn ] = 0.

Thus {Bn } is increasing.

4. Let Xn , n ≥ 1, be i.i.d. with P (Xn = 1) = P (Xn = −1) = 21 . Let

S0 := 0, Sn := nm=1 Xm . Let a, b be positive integers. Calculate
P

P (Sn hits a before it hits −b).

Ans. Let T := min{n ≥ 0 : Xn = a or − b}. Then for N ≥ 1,

E [ST ∧N ] = 0 by the optional sampling theorem. Since |ST ∧N | ≤ a ∨ b,
dominated convergence theorem leads to

E [ST ] = lim E [ST ∧N ] = 0.

N ↑∞

Let pa = P (Sn hits a before it hits −b) and pb = P (Sn hits −b before
it hits a). Since lim supn↑∞ Sn = − lim inf n↑∞ Sn = ∞ (e.g., by LIL), it
follows that pa + pb = 1. Also, E [ST ] = apa − bpb = 0. Solving the two
b a
equations, pa = a+b , pb = a+b .

5. {Xn , n ≥ 0} is a process taking values in the node set V of a finite

connected undirected graph. Suppose there exists an α > 0 such that
P (Xn+1 = j|Xm = im , m ≤ n) ≥ α whenever there is an edge between
in and j. Show that P (Xn = i i.o.) = 1 ∀i ∈ V .

Ans. Since
P P
(i) n I{Xn = i} and n P (Xn = i|Xm , m < n) converge or diverge
together a.s.,
(ii) n P (Xn = i|Xm , m < n) ≥ α n I{Xn = j} for any j such that
P P

there is an edge between i and j, and,

2
 (iii) the graph is connected,
it follows that Xn = i i.o. for some i ∈ S implies Xn = i i.o. for all
i ∈ S. But i∈S n I{Xn = i} = n i∈S I{Xn = i} = n 1 = ∞, so
P P P P P

Xn = i i.o. for at least one i ∈ S, hence for all i ∈ S.

6. Let Xn , n ≥ 0, be real random variables and Fn := σ(Xm , m ≤ n)

for n ≥ 0. Let T be a stopping time w.r.t. {Fn }. Show that FT =
σ(XT ∧n , n ≥ 0). (Assume that all σ-fields are complete w.r.t. the un-
derlying probability measure.)
Ans. Let F T := σ(XT ∧n , n ≥ 0). If A ∈ FT , A ∩ {T = n} ∈ Fn ,
therefore is of the form {[X0 , · · · , Xn ] ∈ Bn0 } for some Borel set Bn0 ⊂
Rn+1 . The latter equals {[X0 , · · · , Xn , Xn , Xn , · · ·] ∈ Bn } ⊂ {T = n}
for Bn := Bn0 × R∞ ⊂ R∞ . In turn, {T = n} ∈ Fn , so it is of the form

{T = n} = {[X0 , · · · , Xn ] ∈ Cn0 }

for some Cn0 ∈ Rn+1 . Let Cn := Cn0 × R∞ ⊂ R∞ . Then

{T = n} = {[XT ∧0 , XT ∧1 , · · ·] ∈ Cn } ∈ F T .

Also,

A = ∪n (A ∩ {T = n}) = ∪n {[X0 , · · · , Xn , Xn , Xn , · · ·] ∈ Bn }

= ∪n ({[X0 , · · · , Xn , Xn , Xn , · · ·] ∈ Bn } ∩ {T = n})
= ∪n ({[XT ∧0 , · · · , XT ∧n , · · ·] ∈ Bn } ∩ {T = n}).
Thus A ∈ F T . Conversely, let A ∈ F T . Then it is of the form A =
{[XT ∧0 , XT ∧1 , · · ·] ∈ B} for some Borel B ⊂ R∞ . Then

A ∩ {T = n} = {[X0 , · · · , Xn , Xn , Xn , · · ·] ∈ B} ∩ {T = n} ∈ Fn ,

so A ∈ FT .

7. Let (Xn , Fn ), n ≥ 0, be a non-negative supermartingale. Show that

Xn+i = 0 ∀i ≥ 0 a.s. on the set {Xn = 0}.

Ans. Since 0 ≤ E [Xn+i |Fn ] ≤ Xn a.s., E [Xn+i |Fn ] I{Xn = 0} =

0 a.s., that is, E [Xn+i I{Xn = 0}|Fn ] = 0 a.s. Taking expectation,

3
 E [Xn+i I{Xn = 0}] = 0. Since Xn+i I{Xn = 0} ≥ 0 a.s., it follows from
Chebyshev inequality that Xn+i I{Xn = 0} = 0 a.s., which proves the
claim.

8. Let X, Y be real random variables with X integrable. Consider aprox-

imations Zn , n ≥ 1, of E[X|Y ] defined by
 h n oi 
k k+1
∞ E XI ≤Y <
( )
X 2n 2n k k+1
Zn :=    I
n
≤Y < .
k=−∞ P k
≤Y < k+1 2 2n
2n 2n

Show that Zn → E[X|Y ] a.s.

Ans. Let Fn := σ(An ) for An := {{ 2kn ≤ Y < k+1 2n

}, −∞ < k <
∞}, n ≥ 1. Then by construction, Zn = E [X|Fn ] , n ≥ 1. Since
Fn ⊂ Fn+1 ∀n and ∨n Fn = σ(Y ), convergence of regular martingales
implies that Zn → E [X|Y ] a.s.

9. (i) Show that a sequence of real integrable random variables {Mn } is

a martingale with respect to Fn := σ(Mk , k ≤ n), n ≥ 0, if and only
if ∀ n > m ≥ 0, E [(Mn − Mm )g(M0 , · · · , Mm )] = 0 for all bounded
continuous g : Rm+1 7→ R.

Ans. (=⇒) Since E[Mn |Fm ] = 0,

E [(Mn − Mm )g(M0 , · · · , Mm )] = E [E[Mn − Mm |Fm ]g(M0 , · · · , Mm )]
= 0.

(⇐=) We have
0 = E [(Mn − Mm )g(M0 , · · · , Mm )]
= E [E[Mn − Mm |Fm ]g(M0 , · · · , Mm )]
for all g as above. Since we can approximate indicator functions of open
balls in a bounded fashion by continuous functions, we can extend this
to g = indicator of an open ball in Rm+1 . Then we have
Z Z
E [Mn − Mm |Fm ] dP = 0 = 0dP.
C C

4
 for all open balls C ⊂ Rm+1 . Since the sets C for which the above
holds form a σ-field containing all open balls, it includes B(Rm+1 ).
By a.s. uniqueness of conditional expectation, it then follows that
E [Mn − Mm |Fm ] = 0 a.s.

(ii) Show that if {(Mni , σ(Mm i

, m ≤ n))} for 1 ≤ i < ∞ are martin-
gales such that {Mn , i ≥ 1} are u.i. for each n and (M1i , M2i , · · ·) →
i

(M1∞ , M2∞ , · · ·) in law, then

{(Mn∞ , σ(Mm
∞
, m ≤ n))}

is also a martingale.

Ans. We have E [(Mni − Mm i

)g(M0i , · · · , Mmi
)] = 0 for all 1 ≤ i < ∞
m+1
and all bounded continuous g : R 7→ R. Letting i ↑ ∞ and using
uniform integrability, we get E [(Mn∞ − Mm ∞
)g(M0∞ , · · · , Mm
∞
)] = 0 for
m+1
all bounded continuous g : R 7→ R. The claim follows from (i).

10. Let {Xn }, {Yn } be square integrable {Fn }-martingales. Define

1
hX, Y in := (hX + Y in − hX − Y in ) , n ≥ 0.
4
Show that Xn Yn − hX, Y in , n ≥ 0, is an {Fn }-martingale.

Ans. Follows from the fact Xn Yn = 41 ((Xn + Yn )2 − (Xn − Yn )2 ).

11. Let (Ω, F, P ) be a probability space and Fn ⊂ F, n ≥ 0, a family of

increasing sub-σ-fields of F such that F = ∨n Fn . Let (Λn , FnR), n ≥ 0,
be a non-negative martingale with mean 1. Define Q(A) := A Λn dP
for A ∈ Fn , n ≥ 0.R Let Λ∞ = limn↑∞ Λn and define a measure Q e on
(Ω, F) by Q(A) = A Λ∞ dP for A ∈ F. Show that Q is a probability
e e
measure that uniquely extends Q to F if and only if (Λn , Fn ), n ≥ 0, is
a regular martingale.

Ans. (⇐=) The consistency of Q on Fn , n ≥ 1, is already done in class.

Thus we only need to prove that E[Λ∞ ] = 1 and Λn = E[Λ∞ |Fn ] ∀n.

5
The latter follows because (Λn , Fn ) is a regular martingale. It also fol-
lows that E[|Λn − Λ∞ |] → 0, from which we have E[Λ∞ ] = 1.

(=⇒) If Q e is a probability measure, E[Λ ] = 1 Since Λ → Λ a.s.

∞ n ∞
and E[Λn ] = 1 → 1 = E[Λ∞ ], it follows that E[|Λn − Λ∞ |] → 0, so
Λn , n ≥ 1, are u.i., i.e., (Λn , Fn ), n ≥ 1, is a regular martingale.

6
 Exercise 4

1. Let {Xn } be a random process taking values in a finite state space S.

Show that it is a time-homogeneous Markov chain if and only if there
exists a transition probability function (i, j) ∈ S × S 7→ p(j|i) ∈ [0, 1]
such that for any f : S 7→ R, Mn := ni=1 (f (Xi ) − j f (j)p(j|Xi−1 )),
P P

n ≥ 1, is a martingale w.r.t. the σ-fields σ(Xm , m ≤ n), n ≥ 1.

Ans. Clearly, if {Xn } is a Markov chain with transition probabilities
p(·|·), then
X
E[Mn+1 − Mn |Fn ] = E[f (Xn+1 ) − p(j|Xn )f (j)|Fn ]
j

= E[f (Xn+1 ) − E[f (Xn+1 )|Fn ]|Fn ] = 0,

so (Mn , Fn ), n ≥ 1, is a martingale. Conversely, if (Mn , Fn ), n ≥ 1, is
a martingale, then E[∆Mn+1 |Fn ] = 0 implies
X
E[f (Xn+1 )|Fn ] = p(j|Xn )f (j) ∀f : S 7→ R,
j

so {Xn } is a Markov chain with transition probabilities p(·|·).

2. Given a Markov chain Xn , 0 ≤ n ≤ N < ∞, with transition probabil-

ities p(j|i), i, j ∈ S and initial distribution λ, calculate the transition
probabilities of the time reversed chain XN −n , 0 ≤ n ≤ N .
Ans. Let Yn := XN −n and πn (i) := P (Xn = i) = i0 pn (i|i0 )λ(i0 ).
P

Then
P (Yn = i, Yn+1 = j)
P (Yn+1 = j|Yn = i) =
P (Yn = i)
P (XN −n = i, XN −n−1 = j)
=
P (XN −n = i)
P (XN −n−1 = j)p(XN −n = i|XN −n−1 = j) πN −n−1 (j)p(i|j)
= = .
πN −n (i) πN −n (i)

1
3. Show that for a Markov chain {Xn },
P (Xn = i|Xn−1 , Xn−2 , Xn+1 , Xn+2 ) = P (Xn = i|Xn−1 , Xn+1 ).

Ans. On the set {Xn−1 = j, Xn−2 = k, Xn+1 = s, Xn+2 = r}, the left
hand side equals
P (Xn−1 = j, Xn−2 = k, Xn = i, Xn+1 = s, Xn+2 = r)
P (Xn−1 = j, Xn−2 = k, Xn+1 = s, Xn+2 = r)
p(j|k)p(i|j)p(s|i)p(r|s)
=P 0 0
=
i0 p(j|k)p(i |j)p(s|i )p(r|s)
p(i|j)p(s|i)
0 0
= P (Xn = i|Xn−1 = j, Xn+1 = s).
i0 p(i |j)p(s|i )
P

4. Let {Xn } be an irreducible positive recurrent Markov chain on a count-

able state space. Show that Ei [τj ] < ∞ for i 6= j in S, where τj is the
first hitting time of j.
Ans. By strong Markov property,
∞ > Ej [τj ] ≥ Ej [I{τi < τj }τj ] = Ej [E[I{τi < τj }τj |Fτi ]]
= Ej [I{τi < τj }E[τj |Fτi ]] = Ej [I{τi < τj }E[τj |Xτi ]]
= Ei [τj ]Pj (τi < τj ).
The claim follows from the fact that there is a path of probability > 0
from j to i, so Pj (τi < τj ) > 0.
5. Let {Xn } be an irreducible Markov chain on a finite state space S and
A ⊂ S a nonempty strict subset of S. Let τ := min{n ≥ 0 : Xn ∈ A}.
Show that E [τ ] < ∞.
Ans. For i ∈ A, τ = 0. For each state i ∈
/ A, there is a path of strictly
positive probability from i to A. Since S\A is finite, it follows that
there exists an N ≥ 1 and κ ∈ (0, 1) such that
max P (τ > N ) < κ.
i∈A
/

Using Markov property, it follows by induction that

τ
  
P (τ > nN ) < κn =⇒ P ≥ n < κn .
N
hl mi
τ
Summing over n ≥ 0, E N
< ∞, hence E[τ ] < ∞.

2
6. Let {Xn } be an irreducible Markov chain on a finite state space S and
let A, B ⊂ S, satisfying A ∩ B = φ and A ∪ B 6= S. Let τ := min{n ≥
0 : Xn ∈ A ∪ B}. Show that V (i) = Pi (Xτ ∈ A), i ∈ S, is characterized
by the linear system
X
V (i) = p(j|i)V (j), i ∈
/ A ∪ B,
j

V (i) = 1 for i ∈ A, V (i) = 0 for i ∈ B.

Ans. Since τ = 0 when X0 = i ∈ A ∪ B, V (i) = 1, i ∈ A, and

= 0, i ∈ B. For i ∈
/ A ∪ B,
X
V (i) = Ei [E[I{Xτ ∈ A}|X1 ]] = p(j|i)V (j).
j

7. Let {Xn } be a Markov chain with state space S and P the probability
measure induced on (S ∞ , F := B(S ∞ )) by [X0 , X1 , X2 , · · ·]. Let Fn :=
B(S n ) and Pn := the restriction of P to (S ∞ , Fn ) for n ≥ 0. Clearly,
F = ∨n Fn . Let (Λn , Fn ), n ≥ 0, be a non-negative martingale with
mean 1. Define a measure Q onR (S ∞ , F) by: the restriction Qn of Q to
(S ∞ , Fn ) is given by Qn (A) = A Λn dP, A ∈ Fn .
(a) Show that for any bounded random variable Y on (S ∞ , Fn , P ) and
m < n,
EP [Y Λn |Fm ]
EQ [Y |Fm ] = .
EP [Λn |Fm ]
Ans. For A ∈ Fm , by the definition of Q,
! !
Z
EP [Y Λn |Fm ] Z
EP [Y Λn |Fm ]
dQ = Λn dP
A EP [Λn |Fm ] A EP [Λn |Fm ]
" ! #
Z
EP [Y Λn |Fm ]
= E Λn Fm dP
A EP [Λn |Fm ]
!
Z
EP [Y Λn |Fm ]
= E[Λn |Fm ]dP
A EP [Λn |Fm ]
Z Z Z
= EP [Y Λn |Fm ]dP = Y Λn dP = Y dQ.
A A A
The claim follows from the a.s. uniqueness of conditional expec-
tation.

3
 (b) Using the above result, show that under Q, {Xn } is a Markov
Qn−1
chain if and only if Λn is of the form m=0 λm (Xm , Xm+1 ) for
some λ : S 2 7→ R+ .
Ans. If Λn are of this form, then
EP [f (Xn+1 ) nm=0 λm (Xm , Xm+1 )|Fn ]
Q
EQ [f (Xn+1 )|Fn ] =
EP [ nm=0 λm (Xm , Xm+1 )|Fn ]
Q

EP [f (Xn+1 )λn (Xn , Xn+1 )|Fn ]

=
EP [λn (Xn , Xn+1 )|Fn ]
EP [f (Xn+1 )λn (Xn , Xn+1 )|Xn ]
= ,
EP [λn (Xn , Xn+1 )|Xn ]
which depends on Xn alone. So {Xn } is Markov under Q. Con-
versely, if {Xn } is Markov under Q with transition probabilities
(say) qn (·|·), then take λn (i, j) := qp(j|i)
n (j|i)
, i, j ∈ S.

8. Let {Xn } be an irreducible Markov chain on a finite state space S with

stationary distribution π. Let i0 ∈ S and τ := min{n > 0 : Xn = i0 }.
Let f : S 7→ R and βf := i π(i)f (i). Define
P

" τ −1 #
X
V (i) = Ei (f (Xm ) − βf ) , i ∈ S.
m=0

(a) Show that (V (·), βf ) satisfy the Poisson equation

X
V (i) = f (i) − βf + p(j|i)V (j), i ∈ S.
j

Ans. First, note that V (i0 ) = 0, because

hP i
τ −1
X E i0 m=0 f (Xm )
βf = π(i)f (i) = .
i Ei0 [τ ]

For i 6= i0 , we have
" τ −1 #
X
V (i) = Ei (f (Xm ) − βf )
m=0
−1
τX
" #
= f (i) − βf + Ei I{τ > 1} (f (Xm ) − βf )
m=1

4
 " " τ −1 ##
X
= f (i) − βf + Ei I{τ > 1}E (f (Xm ) − βf )|X1
m=1
= f (i) − βf + Ei [I{τ > 1}V (X1 )]
= f (i) − βf + Ei [V (X1 )]
(because V (i0 ) = 0)
X
= f (i) − βf + p(j|i)V (j).
j

(b) Show that if (V 0 (·), β 0 ) satisfy this equation, then β 0 = βf and

V 0 − V ≡ a constant.
Ans. Multiply both sides of the equation

V 0 (i) = f (i) − β 0 + p(j|i)V 0 (j)

X

P
by π(i), sum over i, and use the fact that i π(i)p(j|i) = π(j)
to obtain i π(i)f (i) = β 0 , so β 0 = βf . Thus subtracting the
P

corresponding equations for V, V 0 we have

V (i) − V 0 (i) = p(j|i)(V (j) − V 0 (j)) ∀i.

X

Since the unique right eigenvector of P (modulo a scalar multiple)

is [1, 1, · · · , 1]T , the second claim follows.