MATH 466

Credit: 3 OR 4 hours.

Systematic discussion of discrete-time Markov chains, continuous-time Markov chains and discrete-time martingales. Topics include strong Markov properties, recurrence and transience, invariant distributions, convergence and ergodicity, time reversal, Q-matrices, holding time, forward and backward equations, martingales and potential theory, queuing networks, Markov decision processes, Markov Chain and Monte Carlo techniques. Unlike other campus stochastic processes courses, this course will emphasize the fundamental mathematical constructions underlying the theory of Markov chains, such as Laplace operators, martingales, and harmonic functions.

3 undergraduate hours. 3 or 4 graduate hours. Prerequisite: MATH 241, MATH 416, and MATH 461. Priority registration will be given to students in the Mathematics + Data Science major.