In a previous post, I mentioned some books that are useful if you want to eventually be able to read more advanced papers/books in statistics and machine learning. The fundamentals of machine learning, even if you’re interested in deep learning, are the same as in statistics: concepts like bias and variance, significance, probability, etc.
When learning these topics, without a regular course schedule and an instructor/TA to ask questions of, it can be difficult to follow the math and/or keep yourself motivated. There are many, many massive open online courses (MOOCs) on statistics and machine learning. However, most of them don’t improve your ability to work through the math. Here I’ll list a few that do: most are at the introductory to intermediate level.
Probability:
Harvard 110x: Introduction to Probability: this is a classic intro to mathematical probability course without measure theory, similar in level to Sheldon Ross’s book.
Statistics:
Econometrics: Methods and Application: This is an excellent course focused primarily on linear regression. Linear regression is a lot deeper than most people think: there are strong assumptions made in the ordinary least squares model, that when violated, lead to poor inference and prediction results. This course has a nice balance between derivations and applications, and shows you the ‘right’ way to do linear regression and use well-studied statistical models in general: start with a simple model, understand the assumptions, check for violations, and finally extend the model to handle the violations.
Mathematical Biostatistics Boot Camp Part 1 and Part 2: This goes over classical mathematical statistics. The material is excellent. That said, the homework problems are potentially a bit too easy compared to an on-campus course at a top school.
Linear Algebra:
As far as I can tell, there are no mathematical linear algebra MOOCs!!! The best starting place for an engineering oriented linear algebra course is probably Gilbert Strang’s famous course. However, I suspect one will want more practice after that, and that doing a MOOC in a topic that uses lots of linear algebra (some area of signal processing, linear regression course, something else) would be very helpful. I will update this post if I have a clearer picture.
Real Analysis:
There is a dearth of pure math MOOCs, and currently no real analysis MOOC. While MIT has an OCW of real analysis, the reality is that only highly talented students can self-teach themselves real analysis from non-interactive course materials. There is a need for a way to scale up teaching of proof skills. Traditionally the most talented students self-teach the initial proof skills, while the rest of us go to office hours, explain our reasoning, and the professor or TA pokes holes in our reasoning. This isn’t scalable though!