Understanding Markov Chains: Examples and Applications

This book provides an undergraduate-level introduction to discrete and continuous-time Markov chains and their applications, with a particular focus on the first step analysis technique and its applications to average hitting times and ruin probabilities. It also discusses classical topics such as recurrence and transience, stationary and limiting distributions, and branching processes.

Format: Paperback / softback
Length: 372 pages
Publication date: 15 August 2018
Publisher: Springer Verlag, Singapore

This comprehensive textbook offers a thorough introduction to discrete and continuous-time Markov chains, with a special emphasis on the first-step analysis technique and its practical applications in the fields of average hitting times and ruin probabilities. It delves into classical topics such as recurrence and transience, stationary and limiting distributions, and branching processes, providing a solid foundation for understanding these complex phenomena.

The book begins by examining two significant examples, gambling processes and random walks, in great detail, before presenting the general theory of Markov chains in subsequent chapters. It also introduces discrete-time martingales and their connection to ruin probabilities and mean exit times, along with a comprehensive chapter on spatial Poisson processes.

To enhance comprehension, the concepts are illustrated through numerous examples, 138 exercises, and 9 problems with their solutions. This textbook is an invaluable resource for undergraduate students studying probability theory, statistics, and applied mathematics, as well as professionals seeking to expand their knowledge in these areas.

Weight: 608g
Dimension: 155 x 235 x 23 (mm)
ISBN-13: 9789811306587
Edition number: 2nd ed. 2018