Convex Optimization
- Convex problems can be solved efficiently and reliably, and have several advantages such as finding exact solutions and potential for embedding in control systems.
- Many applications can be formulated as convex problems, but most real-world problems are not truly convex.
- Convex optimization problems have identical structures across different fields and can be solved efficiently using tools like CVX PI.
- Understanding the dual of a convex problem can provide insights and practical benefits.
- Advanced techniques like repeatedly forming and solving convex approximations can be used to solve complex problems.
Sparse Signal Reconstruction
- Sparse signal reconstruction involves finding a sparse representation of a signal from noisy measurements.
- L1 reconstruction, also known as LASSO, is an effective method for sparse signal reconstruction.
- Total variation denoising is an application of sparse signal reconstruction to image denoising.
- Total variation regularization can be used to reduce noise while preserving sharp edges in images.
- Iterated weighted L1 regularization can improve the performance of total variation regularization.
Cardinality Constraint
- The cardinality constraint can be applied to various scenarios, such as piecewise constant fitting or piecewise linear fitting.
- The L1 norm heuristic is commonly used as a convex relaxation for cardinality constrained problems.
- Polishing is a technique where L1 heuristic is used to guess the sparsity pattern of the solution, and then the problem is resolved with only the non-zero entries.
- L1 penalty keeps up the pressure to make the coefficients small all the way up until zero, which explains its effectiveness as a sparsifying regularizer.
- There are other sparsifying penalties, such as the Huber penalty and the buru penalty, which combine different types of penalties to achieve sparsity.
Other Topics
- Sparse solutions of linear inequalities can be found using the L1 heuristic, but it doesn't always find the global solution.
- Time-varying AR models can be fit using a sparsity-inducing regularizer on the differences in the coefficients.
- The dual of the spectral norm is the sum of the singular values and it can be used as a low-rank regularizer.
- Sparse Bayesian models can be estimated by maximizing the log-likelihood minus the dual norm of the precision matrix.
- Subgradient methods and stochastic subgradient methods are covered in other courses such as 364b.
- Distributed convex optimization allows multiple parties to collaborate to solve a problem without sharing their data.
