This lecture series discusses how the concept of probability can be used to handle, control, and exploit uncertainty in the real-world. It is an undergraduate-level lecture series on probability, but is entirely different from the usual courses on probability theory. The lectures cover the basics of probability theory including the relevant mathematics, but instead of focusing on mathematics, the lectures explain how probability theory can help understand real-world uncertainty using various examples.
Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.
The examples are used to describe how uncertainty can be exploited to implement modern randomized algorithms such as Markov chain Monte Carlo and deep learning.
The first part of the series (three weeks) discusses the basics of probability theory such as the mathematical formulation of probability, random variables, expectation, and variance in a creative way as a means to quantify uncertainty.
The second part of the series (five weeks) introduces a few universal principles of probability theory. Standard theorems in probability theory such as the law of large numbers and the central limit theorems are introduced as fundamental examples of universal principles, and hence, are discussed from a unique perspective. These universal principles are used to explain uncertainty in the real-world, and numerous interesting examples are introduced for illustration.
The third part of the series (four weeks) introduces the concept of Markov chain and then discusses various randomized algorithms as examples of Markov chains. For example, riffle shuffle of playing cards, Markov chain Monte Carlo, and deep learning algorithms are discussed based on the modern theory of Markov chains.
The lecture series requires knowledge of calculus, but knowledge of higher mathematics and probability is not a pre-requisite.
What you'll learn
- Basic probability theory including random variable, expectation, and variance
- Universal principles in probability theory such as law of large numbers, central limit theorem, and large deviation principles, and their applications
- Heavy-tailed phenomenon
- Theory random processes and applications to real world problem
- Theory of Markov chains and applications to simulation, randomization, and deep learning.
Syllabus
Lecture 1. Uncertainty: Control vs Exploit
1) A toy example
2) Control the uncertainty
3) Exploit the uncertainty
Lecture 2. Quantification of Uncertainty (1): Probability and Random Variables
1) Mathematical formulation of probability
2) Random variables
3) Independence
Lecture 3. Quantification of Uncertainty (2): Expectation and Variance
1) Expectation
2) Variance and standard deviation
3) Applications
Lecture 4. Universal Principle (1): Law of large numbers
1) Introduction to universality
2) Law of large numbers
3) Proof of law of large numbers
4) Applications
Lecture 5. Universal Principle (2): Central limit theorem
1) Central limit theorem
2) Applications to statistics
Lecture 6. Universal Principle (3): More on fluctuation
1) Heavy-tailed random variables
2) Large deviation principles
Lecture 7. Universal Principle (4): Random processes
1) Introduction to random processes
2) Simple random walk on a line
3) Applications to gambling
Lecture 8. Universal Principle (5): Universality of random processes
1) Universality in random walks
2) Galton-Watson tree
Lecture 9. How to use uncertainty? (1): Introduction to Markov Chains
1) Markov processes
2) Markov chains
3) Examples
Lecture 10. How to use uncertainty? (2): Universal principles of Markov chains
1) Stationary distribution
2) Universal principles for Markov chains
Lecture 11. How to use uncertainty? (3): MCMC and Cutoff phenomenon
1) Markov chain Monte Carlo (MCMC)
2) Markov chain mixing theory
3) Cutoff phenomenon
Lecture 12. How to use uncertainty? (4): Stochastic optimizations and deep learning
1) Gradient descent
2) Stochastic gradient descent
3) Mini-batch gradient descent
