Stochastic Methods

1. Review of Probability Theory

• Probability spaces, sigma-algebras, properties of probability
• Conditional probability and independence
• Random variables, expected value, and conditional expected value
• Law of large numbers and Chernoff Bounds
• Approximation of probability distributions.

2. Markov Chains and Random Walks

• Discrete-time Markov chains: definition and classification of states
• Stationary distributions and convergence results.