Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Conic Linear Programming Yinyu Ye Department of Management Science and Engineering Stanford University Stanford CA 94305 U S A http www stanford edu yyye 1 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Conic LP CLP minimize c x subject to ai x bi i 1 2 m x C where C is a convex cone Linear Programming LP c ai x Rn and C Rn Second Order Cone Programming SOCP c a i x Semidefinite Programming SDP c ai x Rn and C SOC Mn and C Mn Note that cone C can be a product of many different convex cones 2 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Convex Optimization or Convex Programming Convex Optimization minimize a convex function over a convex constraint set region An important fact for CO any local minimizer is a global minimizer CO where ci x i minimize c0 x subject to ci x bi i 1 2 m 0 1 m are convex functions of x Proof Let x be a local minimizer and x be the global minimizer such that c0 x c0 x Let x x 1 x Then it is feasible and c0 x c0 x 1 c0 x c0 x 0 This contradicts to x being a local minimizer since can be small enough such that x is in the neighborhood of x 3 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Convex Optimization is equivalent to CLP The convex program can be rewritten as CO minimize subject to c0 x 0 ci x bi 0 i 1 2 m Thus it is sufficient to consider convex optimization in a form CO where ci x i minimize cT x subject to ci x 0 i 1 2 m 1 m are convex functions of x Consider set Ci t x t 0 tci x t 0 It is a convex cone 4 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Convex Optimization is equivalent to CLP continued Then CO can be equivalently written as minimize 0 c t x subject to 1 0 t x 1 t x C1 Cm This is a Conic LP We now develop theories for CLP 5 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Dual of Conic LP The dual problem to CLP minimize c x subject to ai x bi i 1 2 m x C is CLD maximize subject to bT y m i yi ai s c s C where y Rm are the dual variables s is called the dual slack vector matrix and C is the dual cone of C Theorem 1 Weak duality theorem c x bT y x s 0 for any feasible x of CLP and y s of CLD 6 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Self Dual Cones Again Frequently C C that is they are self dual The dual of the n dimensional non negative orthant Rn x Rn x 0 is Rn it is self dual n The dual of the positive semi definite matrices cone in M n Mn is M it is self dual The dual of the second order cone t x second order cone it is self dual R n 1 t x is also the 7 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 SOCP Examples minimize subject to 2 1 x 1 1 1 x 1 x SOC 1 Dual maximize subject to y 2 1 1 y 1 s SOC 1 1 8 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 SDP Examples minimize 2 5 5 1 subject to 1 5 5 1 X X 1 X 0 Dual maximize y subject to 2 5 5 1 y 1 5 5 1 S 0 9 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Farkas Lemma for General Cones Given ai i 1 m and b Rm Then the system x ai x bi i 1 m x C has a feasible m T solution x if and only if that i yi ai C and b y 0 has no feasible solution y It is necessary but not sufficient Let s write equations in a compact form Ax a1 x am x Rm and m AT y yi ai i 10 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Alternative Systems for General Cones Alternative System Pair I Ax b x C and AT y C bT y 1 Alternative System Pair II Ax 0 x C c x 1 0 and c AT y C 11 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 An SDP Cone Example when Alternative System failed C M2 a1 and 1 0 0 0 a2 b 0 2 0 1 1 0 12 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 When Farkas Lemma Holds for General Cones Let C be a closed convex cone in the rest of the course If there is y such that AT y int C then Alternative System Pair I is true Ax b x C and AT y C bT y 1 And if there is x such that AT x 0 x int C then Alternative System Pair II is true Ax 0 x C c x 1 0 and c AT y C 13 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 Conic Linear Programming in Compact Form CLP minimize c x subject to Ax b x C CLD maximize subject to bT y AT y s c s C Denote by Fp and Fd the primal and dual feasible sets respectively 14 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 CLP Duality Theories The weak duality theorem shows that a feasible solution to either problem yields a bound on the value of the other problem We call c x b T y the duality gap Corollary 1 Let x Fp and y s Fd Then c x bT y implies that x is optimal for CLP and y s is optimal for CLD Is the reverse also true That is given x optimal for CLP then there is y s feasible for CLD and c x bT y This is called the Strong Duality Theorem and it is true for LP but it is False in general cases 15 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 16 SDP Example with a Duality Gap 0 1 c 1 0 0 0 and 0 0 0 a1 0 0 0 0 0 0 1 0 a2 1 0 0 0 b 0 10 1 0 0 0 0 2 Yinyu Ye MS E Stanford MS E310 Lecture Note 14 When Strong Duality Theorems Holds for CLP Theorem 2 Strong duality theorem Let F p and Fd be non empty and at least one of them has an interior Then x is optimal for CLP and y s is optimal for CLD if any only if c x bT y There are cases that …