Linear Algebra for Data Science Cookbook

A review of mathematical concepts helpful for data science.

mathematicsdata science

These notes cover key linear algebra topics from the MIT Opencourseware course on Matrix Methods in Data Analysis, Signal Processing, and Machine Learning, using Linear Algebra and Learning from Data as the main reference. All ideas derive from these sources.

Summary

Positive definite symmetric matrices are particularly useful because they have positive eigenvalues and orthogonal eigenvectors . Any such matrix can be written as a combination of simple rank-one projections onto those eigenvectors.

If , then captures the greatest variation in a sample covariance matrix.

The singular value decomposition (SVD) extends these ideas to all matrices. It decomposes matrix into two sets of singular vectors (s and s) and singular values , which play the same role as eigenvalues .

Rank-one pieces of retain their importance ordering as decreases. The Eckart-Young Theorem states that for a rank- approximation, stop at —complementing our understanding of the symmetric matrix .

Topics from Part 1 of Linear Algebra and Learning from Data

  1. Multiplication using columns of
  2. Matrix-Matrix multiplication
  3. The four fundamental subspaces
  4. Elimination and
  5. Orthogonal matrices and subspaces
  6. Eigenvalues and eigenvectors
  7. Symmetric positive definite matrices
  8. Singular values and singular vectors in the SVD
  9. Principal components and the best low rank matrix
  10. Norms of vectors and functions and matrices