Advice for students of machine learning——Written by David Mimno

    One of my students recently asked me for advice on learning ML. Here’swhat I wrote. It’s biased toward my own experience, but should generalize.    

    My current favorite introduction is Kevin Murphy’s book (MachineLearning). You might also want to look at books by Chris Bishop(Pattern Recognition), Daphne Koller (Probabilistic Graphical Models),and David MacKay (Information Theory, Inference and Learning Algorithms).

   Anything you can learn about linear algebra and probability/statisticswill be useful. Strang’s Introduction to Linear Algebra, Gelman, Carlin, Stern and Rubin’s BayesianData Analysis, and Gelman and Hill’s Data Analysis using Regression and Multilevel/Hierarchical models are some of my favorite books.

   Don’t expect to get anything the first time. Read descriptions of thesame thing from several different sources.

   There’s nothing like trying something yourself. Pick a model andimplement it. Work through open source implementations andcompare. Are there computational or mathematical tricks that makethings work?

   Read a lot of papers. When I was a grad student, I had a 20 minute busride in the morning and the evening. I always tried to have aninteresting paper in my bag. The bus isn’t the important part — whatwas useful was having about half an hour every day devoted to reading.

   Pick a paper you like and “live inside it” for a week. Think about itall the time. Memorize the form of each equation. Take long walks andtry to figure out how each variable affects the output, and howdifferent variables interact. Think about how you get from Eq. 6 toEq. 7 — authors often gloss over algebraic details. Fill them in.

   Be patient and persistent. Remember von Neumann: “in mathematics youdon’t understand things, you just get used to them.”