5.6 Markov Chains

• Define Markov Chain, initial probability vector, transition matrix, steady state vector
• Applications
• p. 353# 1-3, 6-8