A review of Matrix Methods in Data Analysis

During the COVID-related work-from-home period, I decided to revisit the mathematics underpinning statistical learning. The MIT 18.065 course seemed ideal: it covers fundamentals rigorously and builds neural networks from first principles. The textbook offers solid references and problems on key topics, though it loses focus in probability and alternative statistical methods. This post documents my experience working through the lectures and problems over roughly thirty days.

I spent to hours daily watching lectures and solving problems. Early on, I found the lectures clear but pitched above my level, so I bought the textbook Linear Algebra and Learning from Data. Each lecture pairs with sections in the textbook, though the problems differ slightly. With textbook reviews and online help for difficult questions, I worked through them all.

The course divides into four areas: linear algebra, probability, optimisation, and deep learning. Strang is an exceptional teacher. I’d first seen him at a Julia conference, so witnessing his teaching prowess directly was a pleasure. He walks through the basics of linear algebra and numerical methods (solving real problems), then introduces probability. The probability section is brief; the textbook could include more statistics problems. Yet this is minor—the course’s focus is teaching neural networks through linear algebra, not comprehensive statistics.

The optimisation and deep learning sections excel: they break complex topics into manageable pieces. Problem sets reinforce earlier material and deepen understanding. Abundant references (mostly open access) and project ideas support further exploration. The course culminates in a project investigating one aspect of the material.

The main weakness is limited coverage of methods beyond neural networks. While NNs are essential, k-means feels tacked on. Yet excellent resources exist for alternative methods (such as Elements of Statistical Learning), and expanding here would dilute the course’s focus.

I recommend this course to anyone with time to invest. The textbook is clear, with solid problems and ample references for exploring applied mathematics. The video lectures excel at teaching linear algebra, optimisation, and neural networks.