Linear Algebra
Vectors and Matrices
Vectors
- Vector representation
- Vector operations (addition, scalar multiplication)
- Dot product
- Cross product
- Vector norms (L1, L2, infinity norm)
- Unit vectors
Matrices
- Matrix representation
- Matrix operations (addition, subtraction, scalar multiplication)
- Matrix multiplication
- Matrix transpose
- Identity matrix
- Diagonal matrices
Matrix Operations
Basic Operations
- Matrix addition and subtraction
- Scalar multiplication
- Matrix multiplication
- Matrix transpose
- Matrix inverse
- Determinant calculation
Special Matrices
- Square matrices
- Symmetric matrices
- Orthogonal matrices
- Positive definite matrices
- Sparse matrices
Eigenvalues and Eigenvectors
Concepts
- Definition of eigenvalues and eigenvectors
- Characteristic polynomial
- Finding eigenvalues
- Finding eigenvectors
- Geometric interpretation
Applications
- Principal Component Analysis (PCA)
- Dimensionality reduction
- Matrix diagonalization
- Power iteration method
SVD and PCA
Singular Value Decomposition (SVD)
- SVD decomposition
- Left and right singular vectors
- Singular values
- Applications of SVD
- Truncated SVD
Principal Component Analysis (PCA)
- Covariance matrix
- Eigenvalue decomposition of covariance matrix
- Principal components
- Variance explained
- Dimensionality reduction with PCA
- PCA implementation
Interview Questions
- What is the geometric interpretation of eigenvalues and eigenvectors?
- Explain the difference between SVD and eigenvalue decomposition.
- How does PCA use linear algebra?
- What is the relationship between matrix rank and linear independence?
- When would you use SVD vs PCA?
Coding Practice
- Implement matrix multiplication from scratch.
- Write a function to compute eigenvalues and eigenvectors.
- Implement PCA from scratch using numpy.
- Create a function to perform SVD decomposition.
- Write code to visualize principal components.
Resources
- Linear Algebra Textbook: "Introduction to Linear Algebra" by Gilbert Strang
- Khan Academy Linear Algebra: https://www.khanacademy.org/math/linear-algebra
- NumPy Linear Algebra: https://numpy.org/doc/stable/reference/routines.linalg.html