What should someone starting a PhD in machine learning know?

I saw a question like this Quora, and have been meaning to start a blog so decided to answer it here.  I’m not going to call myself an expert, but I’ll chime in on this anyway since everyone has an opinion :).  Michael Jordan has a great reading list (https://honglangwang.wordpress.com/2014/12/30/machine-learning-books-suggested-by-michael-i-jordan-from-berkeley/) on what you need to be an expert, but if you don’t have the training, you won’t be able to pick up any of those books and get anywhere.  So here’s my list, which should allow you to get started on Michael Jordan’s list comfortably.  Update: a friend doing a stat PhD at Cornell told me that during her undergrad at UC Berkeley, she used exactly these books.

1.  Sheldon Ross: A First Course on Probability
 

2.  John A. Rice: Mathematical Statistics and Data Analysis

3.  Walter Rudin: Principles of Mathematical Analysis

4.  Otto Bretscher: Linear Algebra with Applications

Mastering those four books will give you the foundations to study other topics as you need them, and you won’t feel lost if you start reading a book on convex optimization or numerical linear algebra or measure theoretic probability, all of which could play a role down the line.  Some people may disagree that 3 is important, but I’d argue that it is.  There are a whole lot of papers/books where small tricks from real analysis get used or the paper gets written in a way that uses just a hint of measure theory, and having gone through that book will really help.