Optimization Concepts
- Avoid treating CVX Pi coding assignments as a "monkey at a typewriter" exercise.
- Focus on understanding the problem and the underlying concepts rather than blindly typing in code.
- The course design emphasizes real-world problem-solving rather than theoretical concepts.
Duality in Optimization
- Duality involves converting a problem into an equivalent problem, known as its dual.
- Strong duality assumes that the optimal values of the primal and dual problems are the same.
- Simple transformations can lead to radically different duals.
- Patterns can be observed in dual problems, such as the appearance of conjugates and adjoints.
- Partial dualization involves dualizing only a subset of the constraints.
Box-Constrained Problems
- The box-constrained problem has an equivalent formulation where the box constraints are implicit in the objective function.
- The Lagrangian dual of the box-constrained problem can be derived analytically, and the optimal value of the dual problem is related to the primal problem.
Generalized Inequalities
- For problems with generalized inequalities, the Lagrangian dual is formed by taking the inner product of the inequality constraints with non-negative vectors in the dual cone.
Semi-Definite Programming
- Semi-definite programming involves minimizing a linear function subject to linear matrix inequalities. The Lagrangian dual of a semi-definite program is another semi-definite program.
Duality for Feasibility Problems
- Duality for feasibility problems, also known as theorems of the alternative, involves formulating a feasibility problem as minimizing zero subject to constraints. The optimal value of the primal problem is either zero or infinity, and the dual problem can be derived accordingly.
Penalty Functions
- Penalty functions are used to approximate norms in optimization problems.
- The L1 approximation is a penalty function that uses the absolute value of the residual.
- The dead zone linear penalty function has a "dead zone" where the penalty is zero, and outside of that zone, the penalty grows linearly.
- The log barrier penalty function closely matches the quadratic penalty function for small residuals but increases more rapidly for larger residuals.
- The choice of penalty function depends on the specific problem being solved.
Least Norm Problems
- Least Norm problems involve minimizing a norm subject to linear constraints.
- In estimation problems, the goal is to find the most plausible solution when there are more unknowns than measurements.
- In design problems, the goal is to minimize a cost function subject to constraints.
Regularized Approximation
- Regularized approximation involves balancing the fit to the data with the size of the solution.
Applications
- Applications of these concepts include minimum fuel trajectory design and missile guidance systems.
- The speaker provides an example of optimal input design in control engineering, where the goal is to minimize tracking error, input magnitude, and input smoothness simultaneously.
- The video discusses a control system that tracks an input signal.
- The goal is to design a controller that tracks the input signal well while minimizing tracking error, input wiggliness, and input size.
