Yinyu Ye MS E Stanford MS E310 Lecture Note 15 MS E314 Conic Linear Programming Yinyu Ye Department of Management Science and Engineering Stanford University Stanford CA 94305 U S A http www stanford edu yyye 1 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Conic LP CLP minimize c x subject to ai x bi i 1 2 m x C where C is a convex cone Linear Programming LP c ai x Rn and C Rn Second Order Cone Programming SOCP c a i x Semidefinite Programming SDP c ai x Rn and C SOC Mn and C Mn Note that cone C can be a product of many different convex cones 2 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 LP SOCP and SDP Examples minimize 2x1 x2 x3 subject to x1 x2 x3 1 x1 x2 x3 0 minimize 2x1 x2 x3 subject to x1 x2 x3 1 x22 x23 x1 minimize 2x1 x2 x3 subject to x1 x2 x3 1 x x2 1 0 x2 x3 3 Yinyu Ye MS E Stanford where for SDP c MS E310 Lecture Note 15 2 5 5 1 and a1 1 5 5 1 Or minimize 2x1 x2 x3 2x4 x5 x6 subject to x1 x2 x3 x4 x5 x6 1 x1 x2 x3 0 x4 x5 x6 SOCP 4 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Convex Optimization or Convex Programming Convex Optimization minimize a convex function over a convex constraint set region An important fact for CO any local minimizer is a global minimizer CO where ci x i minimize c0 x subject to ci x bi i 1 2 m 0 1 m are convex functions of x Proof Let x be a local minimizer and x be the global minimizer such that c0 x c0 x Let x x 1 x Then it is feasible and c0 x c0 x 1 c0 x c0 x 0 This contradicts to x being a local minimizer since can be small enough such that x is in the neighborhood of x 5 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Convex Optimization is equivalent to CLP The convex program can be rewritten as CO minimize subject to c0 x 0 ci x bi 0 i 1 2 m Thus it is sufficient to consider convex optimization in a form CO where ci x i minimize cT x subject to ci x 0 i 1 2 m 1 m are convex functions of x Consider set Ci t x t 0 tci x t 0 It is a convex cone 6 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Convex Optimization is equivalent to CLP continued Then CO can be equivalently written as minimize 0 c t x subject to 1 0 t x 1 t x C1 Cm This is a Conic LP We now develop theories for CLP 7 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Dual of Conic LP The dual problem to CLP minimize c x subject to ai x bi i 1 2 m x C is CLD maximize subject to bT y m i yi ai s c s C where y Rm are the dual variables s is called the dual slack vector matrix and C is the dual cone of C Theorem 1 Weak duality theorem c x bT y x s 0 for any feasible x of CLP and y s of CLD 8 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Self Dual Cones Again Frequently C C that is they are self dual The dual of the n dimensional non negative orthant Rn x Rn x 0 is Rn it is self dual n The dual of the positive semi definite matrices cone in M n Mn is M it is self dual The dual of the second order cone t x second order cone it is self dual R n 1 t x is also the 9 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 SOCP Examples minimize subject to 2 1 x 1 1 1 x 1 x SOC 1 Dual maximize subject to y 2 1 1 y 1 s SOC 1 1 10 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 SDP Examples minimize subject to 2 5 5 1 1 5 5 1 X X 1 X 0 Dual maximize y subject to 2 5 5 1 y 1 5 5 1 S 0 11 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Farkas Lemma for General Cones Given ai i 1 m and b Rm Then the system x ai x bi i 1 m x C has a feasible m T solution x if and only if that i yi ai C and b y 0 has no feasible solution y It is necessary but not sufficient Let s write equations in a compact form Ax a1 x am x Rm and m AT y yi ai i 12 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Alternative Systems for General Cones Alternative System Pair I Ax b x C and AT y C bT y 1 Alternative System Pair II Ax 0 x C c x 1 0 and c AT y C 13 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 An SDP Cone Example when Alternative System failed C M2 a1 and 1 0 0 0 a2 b 0 2 0 1 1 0 14 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 When Farkas Lemma Holds for General Cones Let C be a closed convex cone in the rest of the course If there is y such that AT y int C then Alternative System Pair I is true Ax b x C and AT y C bT y 1 And if there is x such that AT x 0 x int C then Alternative System Pair II is true Ax 0 x C c x 1 0 and c AT y C 15 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 Conic Linear Programming in Compact Form CLP minimize c x subject to Ax b x C CLD maximize subject to bT y AT y s c s C Denote by Fp and Fd the primal and dual feasible sets respectively 16 Yinyu Ye MS E Stanford MS E310 Lecture Note 15 CLP Duality Theories The weak duality theorem shows that a feasible solution to either problem yields a bound on the value of the other problem We call c x b T y the duality gap Corollary 1 Let x Fp and y s Fd Then c x bT y implies that x is optimal for CLP and y s is optimal for CLD Is the reverse also true That is given x optimal for CLP then there is y s feasible for CLD and c x bT y This is called the Strong Duality Theorem and it is true for LP but it is False in general cases 17 Yinyu Ye MS E …