First Course in Stochastic Processes
First Course in Stochastic Processes
First Course in Stochastic Processes
SECOND EDITION
SAMUEL KARLIN
STANFORD UNIVERSITY AND THE WEIZMANN INSTITUTE OF SCIENCE
HOWARD M. TAYLOR
CORNELL UNIVERSITY
ACADEMIC PRESS
New York
San Francisco
London
CONTENTS
. ..
.. ..
.. ..
.. ..
.. ..
xi xv
Chapter 1 ELEMENTS OF STOCHASTIC PROCESSES 1. Review of Basic Terminology and Properties of Random Variables and Distribution Functions .. .. .. .. 2. Two Simple Examples of Stochastic Processes .. .. 3. Classification of General Stochastic Processes .. .. 4. Denning a Stochastic Process .. .. .. .. Elementary Problems .. .. .. .. Problems .. .. .. .. .. Notes .. .. .. .. .. .. References .. .. .. .. ..
.. .. .. .. .. .. .. ..
1 20 26 32 33 36 44 44
Chapter 2 MARKOV CHAINS 1. Definitions .. .. .. 2. Examples of Markov Chains .. .. 3. Transition Probability Matrices of a Markov Chain .. .. .. .. .. .. .. .. .. 45 47 58
vi 4. 5. 6. 7. Classification of States of a Markov Chain . . Recurrence .. .. . . Examples of Recurrent Markov Chains .. More on Recurrence . . .. .. Elementary Problems .. .. Problems .. .. .. Notes .. .. .. .. References .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
CONTENTS .. .. .. .. .. .. .. 59 62 67 72 73 77 79 80
CONTENTS
vii
Chapter 6 MARTINGALES
1. Preliminary Definitions and Examples .. .. .. 2. Supermartingales and Submartingales .. .. .. 3. The Optional Sampling Theorem .. .. .. 4. Some Applications of the Optional Sampling Theorem . . .. 5. Martingale Convergence Theorems .. .. .. 6. Applications and Extensions of the Martingale Convergence Theorems 7. Martingales with Respect to <r-Fields .. .. .. 8. Other Martingales .. .. .. .. .. Elementary Problems .. .. .. .. Problems .. .. .. .. .. Notes .. .. .. .. .. .x. References .. .. .. .. .. .. .. ., .. .. .. .. .. .. .. .. .. 238 248 253 263 278 287 297 313 325 330 339 339
viii
Elementary Problems Problems .. Notes .. .. References . . .. .. .. . . .. .. ... .. .. .. .. .. .. .. .. ..
CONTENTS
.. .. .. .. 383 386 391 391
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.
Discrete Time Branching Processes .. .. Generating Function Relations for Branching Processes Extinction Probabilities .. .. .. Examples .. .. .. .. Two-Type Branching Processes .. .. .. Multi-Type Branching Processes .. .. Continuous Time Branching Processes .. .. Extinction Probabilities for Continuous Time Branching Processes Limit Theorems for Continuous Time Branching Processes Two-Type Continuous Time Branching Process .. Branching Processes with General Variable Lifetime .. Elementary Problems .. .. .. Problems .. .. .. .. Notes .. .. .. .. .. Reference .. .. .. ..
.. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
392 394 396 400 404 411 412 416 419 424 431 436 . . 4 3 8 .. 442 .. 442
.. .. .. .. .. .. .. .. .. .. .. ..
CONTENTS
ix
Appendix REVIEW OF MATRIX ANALYSIS 1. The Spectral Theorem .. .. 2. The Frobenius Theory of Positive Matrices . . .. .. .. .. .. .. 536 542
Index
..
..
..
..
553