Linear Algebra
- Linear algebra is the foundation of the course, and everyone should be familiar with its concepts.
- Direct methods for solving sets of equations, such as Gaussian elimination, factor the coefficient matrix into easily invertible matrices.
- Sparse matrix factorization deals with efficient methods for factorizing and solving sparse matrices.
- The Cholesky factorization is used to solve positive definite systems of equations efficiently.
- Sparse Cholesky factorization is used to solve positive definite systems of equations efficiently.
- Permuting the matrix before factorization can significantly affect the sparsity of the resulting L factor.
- Finding a good permutation is crucial for achieving efficient factorization and solving times.
- Cholesky factorization fails if the matrix is not positive definite.
- LDL transpose factorization is a method for solving non-singular symmetric matrices.
- The sure complement method is used to solve matrix equations efficiently in certain cases.
- Sparse matrices with a banded structure and dense rows and columns can be solved efficiently using block elimination.
- The sparsity pattern of a matrix can indicate the complexity of solving the associated system of equations.
- The Matrix Inversion Lemma provides a way to efficiently invert a matrix that is a perturbation of a diagonal matrix.
- Un-elimination can be an effective technique for solving certain types of linear systems.
- Sparse solvers can efficiently solve large systems of linear equations, making many traditional methods obsolete.
Unconstrained Minimization
- Unconstrained minimization involves finding the minimum value of a smooth, twice continuously differentiable function without any constraints.
- Iterative methods are used to solve unconstrained minimization problems since analytical solutions are generally not available.
- Stopping criteria are used to determine when the iterative process should stop.
- Descent methods are widely used for convex optimization and involve finding a descent direction and choosing a step length.
- Gradient descent is the most intuitive iterative method for unconstrained minimization.
- Gradient descent exhibits linear convergence, meaning that each iteration reduces the error by a constant factor.
