SDP Relaxation of the Max-Cut Problem

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSemidefinite relaxation of the max-cut problemA. Dudek, K. Ghobadi, E. Kim, S. Lin, R. Spjut, J. ZhuMSRI Workshop on Continuous Optimization and its ApplicationsJuly 17, 2007

3Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProject StatementProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblemFind the SDP relaxation of the Max-Cut problem:1 Solve this relaxation using e.g. SeDumi with MATLAB.2 Use randomly generated, weighted, undirected graphs,e.g. W=sprandsym(60,.5).3 Can you strengthen the relaxation by adding additionalconstraints? (Hint: consider constraints of the typex i x 2 k x j = x i x j .)Let G = (V , E) be a graph and let w : E → R be an edgeweight function on G and set n = |V |.

3Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProject StatementProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblemFind the SDP relaxation of the Max-Cut problem:1 Solve this relaxation using e.g. SeDumi with MATLAB.2 Use randomly generated, weighted, undirected graphs,e.g. W=sprandsym(60,.5).3 Can you strengthen the relaxation by adding additionalconstraints? (Hint: consider constraints of the typex i x 2 k x j = x i x j .)Let G = (V , E) be a graph and let w : E → R be an edgeweight function on G and set n = |V |.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsA Review of Max-cutProblem Statement and BackgroundReformulation and RelaxationSetting up an Instancew ij :={ w((i, j)) if (i, j) ∈ E0 otherwiseLet W = ( )w ij , the matrix whose ij-th entry is wij .A cut is a partition of V into two sets C ⊂ V and V \ C. Itssize iss(C) :=∑i∈C,j∈V \Cw ijA cut C is maximal if ∀ ˜C ∈ 2 V , s(C) ≥ s( ˜C)Max-Cut ProblemFor a weighted graph (V , E, w) find a maximal cut C.RemarkThe Max-Cut problem is NP-hard.4

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsA Review of Max-cutProblem Statement and BackgroundReformulation and RelaxationSetting up an Instancew ij :={ w((i, j)) if (i, j) ∈ E0 otherwiseLet W = ( )w ij , the matrix whose ij-th entry is wij .A cut is a partition of V into two sets C ⊂ V and V \ C. Itssize iss(C) :=∑i∈C,j∈V \Cw ijA cut C is maximal if ∀ ˜C ∈ 2 V , s(C) ≥ s( ˜C)Max-Cut ProblemFor a weighted graph (V , E, w) find a maximal cut C.RemarkThe Max-Cut problem is NP-hard.4

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsA Review of Max-cutProblem Statement and BackgroundReformulation and RelaxationSetting up an Instancew ij :={ w((i, j)) if (i, j) ∈ E0 otherwiseLet W = ( )w ij , the matrix whose ij-th entry is wij .A cut is a partition of V into two sets C ⊂ V and V \ C. Itssize iss(C) :=∑i∈C,j∈V \Cw ijA cut C is maximal if ∀ ˜C ∈ 2 V , s(C) ≥ s( ˜C)Max-Cut ProblemFor a weighted graph (V , E, w) find a maximal cut C.RemarkThe Max-Cut problem is NP-hard.4

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsA Review of Max-cutProblem Statement and BackgroundReformulation and RelaxationSetting up an Instancew ij :={ w((i, j)) if (i, j) ∈ E0 otherwiseLet W = ( )w ij , the matrix whose ij-th entry is wij .A cut is a partition of V into two sets C ⊂ V and V \ C. Itssize iss(C) :=∑i∈C,j∈V \Cw ijA cut C is maximal if ∀ ˜C ∈ 2 V , s(C) ≥ s( ˜C)Max-Cut ProblemFor a weighted graph (V , E, w) find a maximal cut C.RemarkThe Max-Cut problem is NP-hard.4

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsA Review of Max-cutProblem Statement and BackgroundReformulation and RelaxationSetting up an Instancew ij :={ w((i, j)) if (i, j) ∈ E0 otherwiseLet W = ( )w ij , the matrix whose ij-th entry is wij .A cut is a partition of V into two sets C ⊂ V and V \ C. Itssize iss(C) :=∑i∈C,j∈V \Cw ijA cut C is maximal if ∀ ˜C ∈ 2 V , s(C) ≥ s( ˜C)Max-Cut ProblemFor a weighted graph (V , E, w) find a maximal cut C.RemarkThe Max-Cut problem is NP-hard.4

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsA Review of Max-cutProblem Statement and BackgroundReformulation and RelaxationSetting up an Instancew ij :={ w((i, j)) if (i, j) ∈ E0 otherwiseLet W = ( )w ij , the matrix whose ij-th entry is wij .A cut is a partition of V into two sets C ⊂ V and V \ C. Itssize iss(C) :=∑i∈C,j∈V \Cw ijA cut C is maximal if ∀ ˜C ∈ 2 V , s(C) ≥ s( ˜C)Max-Cut ProblemFor a weighted graph (V , E, w) find a maximal cut C.RemarkThe Max-Cut problem is NP-hard.4

5Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemWe reforumlate the Max-Cut problem:Problem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblem: Reformulation of Max-Cut ProblemGiven any cut C, define{x := (x 1 , . . . , x n ) T 1 i ∈ C, x i :=−1 i ∉ CMaximize t = 1 ∑2 i

5Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemWe reforumlate the Max-Cut problem:Problem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblem: Reformulation of Max-Cut ProblemGiven any cut C, define{x := (x 1 , . . . , x n ) T 1 i ∈ C, x i :=−1 i ∉ CMaximize t = 1 ∑2 i

5Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemWe reforumlate the Max-Cut problem:Problem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblem: Reformulation of Max-Cut ProblemGiven any cut C, define{x := (x 1 , . . . , x n ) T 1 i ∈ C, x i :=−1 i ∉ CMaximize t = 1 ∑2 i

5Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemWe reforumlate the Max-Cut problem:Problem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblem: Reformulation of Max-Cut ProblemGiven any cut C, define{x := (x 1 , . . . , x n ) T 1 i ∈ C, x i :=−1 i ∉ CMaximize t = 1 ∑2 i

5Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemWe reforumlate the Max-Cut problem:Problem Statement and BackgroundReformulation and RelaxationSetting up an InstanceProblem: Reformulation of Max-Cut ProblemGiven any cut C, define{x := (x 1 , . . . , x n ) T 1 i ∈ C, x i :=−1 i ∉ CMaximize t = 1 ∑2 i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsReformulation of Max-Cut ProblemSome Equalities for tProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceSkip long string of equalities for t⎛ ⎛ ⎞ ⎛ ⎞⎞t = 1 ∑w ij (1 − x i x j ) = 1 ⎝2 ⎝ ∑ w ij⎠ − 2 ⎝ ∑ w ij x i x j⎠⎠24i

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: Equivalent Reformulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, rank(X) = 1, and X ≽ 0.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceReformulation and Relaxation of Max-Cut ProblemLemmat = x T Lx = trace(x T Lx) = trace(x T (Lx)) = trace((Lx)x T ) =trace(LX) = L • X, where X = xx T is a real symmetricmatrix, =⇒ X ≽ 0.diag(X) = e ⇐⇒ x 2i= 1 ⇐⇒ x i ∈ {−1, 1}.X = X T ≽ 0, rank(X) = 1 ⇐⇒ X = xx TW is the edge-weight matrix L = 1 4 (Diag(We) − W ) Problem: SDP Relaxation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e and X ≽ 0.

8Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSetting up an InstanceProblem Statement and BackgroundReformulation and RelaxationSetting up an InstanceWe setup an instance of the Max-Cut problem in MATLAB withthe following code:% Fix the dimension (number of vertices in graph)n=60;% Generate a weighted graphW=sprandsym(n,.5); W(1:n+1:n^2)=zeros(1,n);% Calculate the LaplacianL=1/4 * (diag(W*ones(n,1))-W);

9Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsInterfacing with SeDumiSeDumi Input FormatBasic Relaxation: Setting up the Input to SeDumiBasic Relaxation: Numerical ResultsTo be consistent with the notation of SeDumi’s documentationand interface, substitute “x” for “z”. The SeDumi primal form isMinimizesubject toc T zAz = b, for z ∈ Kwhere c, z, b ∈ R p , and A is a p × p matrix. In MATLAB,c=-L(:); % Format the Laplacian to vector form% Translate constraint diag(X)=e into vector format:A=sparse(1:n,1:n+1:n^2,ones(1,n),n,n^2);b=ones(n,1);% Tell SeDumi that X must be positive semidefiniteK.s=[n];[X,Y,INFO] = sedumi(A,b,c,K) % Run SeDumi

10% Tell SeDumi that X must be positive ProblemsemidefiniteSet-up SeDumi Input FormatK.s=[n];Solving Using SeDumi Basic Relaxation: Setting up the Input to SeDumiImproving the Implementation and Results Basic Relaxation: Numerical Results% Run SeDumi[X,Y,INFO] = sedumi(A,b,c,K)Interfacing with SeDumiA visual summary of Az = bIt may help to visualize the constraint:⎛⎜⎝1 0 . . . 0 0 0 0 . . . 0 0 0 0 . . . 00 0 . . . 0 0 1 0 . . . 0 0 0 0 . . . 00 0 . . . 0 0 0 0 . . . 0 0 0 1 . . . 0. .. .. . . . .. .. . . . .. .. .0 0 . . . 0 0 0 0 . . . 0 0 0 0 . . . 0. . .0 . . . 00 . . . 00 . . . 0.. .. .0 . . . 1⎞⎟⎠⎛n×n 2 ⎜⎜⎝x 11x 12.x 1nx 21x 22x 23.x 2nx 31x 32x 33.x 3n.x n1.x nn⎞⎟⎠n 2 ×1⎛=⎜⎝111.1⎞⎟⎠n×1

11Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSeDumi Input FormatBasic Relaxation: Setting up the Input to SeDumiBasic Relaxation: Numerical ResultsNumerical Results: Basic Relaxation G ( n, 1 2n Ratio50 6.292239e-0160 6.202113e-0170 6.067923e-0180 6.074731e-0190 5.980219e-01100 5.894269e-01110 5.849831e-01120 5.837704e-01130 5.793048e-01140 5.777026e-01150 5.749815e-01160 5.715093e-01170 5.702395e-01180 5.686022e-01190 5.675917e-01200 5.654331e-01250 5.591398e-01300 5.538571e-01350 5.506719e-01400 5.472742e-01450 5.441005e-01500 5.422019e-01600 5.390324e-01700 5.359528e-01800 5.337012e-01900 5.320470e-011000 5.303538e-01)

12Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTriangle InequalitiesStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAn idea proposed by Poljak and Rendl (1995), and furtherdeveloped by Helmberg, Rendl, Vanderbei and Wolkowicz(1996):For each triple, x i , x j , x k ∈ {−1, 1} we havex ij + x jk + x ki ≥ −1 (1)x ij − x jk − x ki ≥ −1 (2)−x ij + x jk − x ki ≥ −1 (3)−x ij − x jk + x ki ≥ −1. (4)With (1)-(4), we have the strengthened SDP relaxation:Problem: Strengthened Relaxation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, X ≽ 0, and (1) − (4).

12Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTriangle InequalitiesStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAn idea proposed by Poljak and Rendl (1995), and furtherdeveloped by Helmberg, Rendl, Vanderbei and Wolkowicz(1996):For each triple, x i , x j , x k ∈ {−1, 1} we havex ij + x jk + x ki ≥ −1 (1)x ij − x jk − x ki ≥ −1 (2)−x ij + x jk − x ki ≥ −1 (3)−x ij − x jk + x ki ≥ −1. (4)With (1)-(4), we have the strengthened SDP relaxation:Problem: Strengthened Relaxation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, X ≽ 0, and (1) − (4).

13Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesTriangle InequalitiesReplacing Inequalities (1)-(4) with Equalities for SeDumi’s Input FormatIntroduce slack variables s ijkl ≥ 0 to get equalities for SeDumi.x ij + x jk + x ki − s ijk1 = −1x ij − x jk − x ki − s ijk2 = −1−x ij + x jk − x ki − s ijk3 = −1−x ij − x jk + x ki − s ijk4 = −1Considering (1)-(4) when a pair of i, j, k are the same isredundant.So we are concerned with “only” N := 2n(n − 1)(n − 2)/3slack variables.

13Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesTriangle InequalitiesReplacing Inequalities (1)-(4) with Equalities for SeDumi’s Input FormatIntroduce slack variables s ijkl ≥ 0 to get equalities for SeDumi.x ij + x jk + x ki − s ijk1 = −1x ij − x jk − x ki − s ijk2 = −1−x ij + x jk − x ki − s ijk3 = −1−x ij − x jk + x ki − s ijk4 = −1Considering (1)-(4) when a pair of i, j, k are the same isredundant.So we are concerned with “only” N := 2n(n − 1)(n − 2)/3slack variables.

13Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesTriangle InequalitiesReplacing Inequalities (1)-(4) with Equalities for SeDumi’s Input FormatIntroduce slack variables s ijkl ≥ 0 to get equalities for SeDumi.x ij + x jk + x ki − s ijk1 = −1x ij − x jk − x ki − s ijk2 = −1−x ij + x jk − x ki − s ijk3 = −1−x ij − x jk + x ki − s ijk4 = −1Considering (1)-(4) when a pair of i, j, k are the same isredundant.So we are concerned with “only” N := 2n(n − 1)(n − 2)/3slack variables.

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTriangle InequalitiesA visual summary of Az = bStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple Equalities0B@It may help to visualize the constraint:0n×N1 0 . . . 0 0 0 0 . . . 0 0 0 0 . . . 00 0 . . . 0 0 1 0 . . . 0 0 0 0 . . . 00 0 . . . 0 0 0 0 . . . 0 0 0 1 . . . 0... ... ... ... ... .. . ... ... ... ... .. . ... ..0 0 . . . 0 0 0 0 . . . 0 0 0 0 . . . 0−IN×N tempA N×n 2. . .140 . . . 00 . . . 00 . . . 0.. ... ..0 . . . 11CA0B@s1,2,3,1s1,2,4,1.s1,3,4,1s1,3,5,1. . .s2,3,4,1s2,3,5,1. . .sn−2,n−1,n,1s1,2,3,2. . .sn−2,n−1,n,2. . ....s1,2,3,4. . .sn−2,n−1,n,4x11x12.x1nx21x22x23.x2nx31x32x33.x3n.xn1.xnn1CAN+n 2 ×1=„ In×1−IN×1«n+N×1

15Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesNumerical Results: Triangle Relaxation G ( n, 1 2)n Ratio 1 Ratio 25 7.142857e-01 7.142857e-016 7.500000e-01 7.500000e-017 7.177745e-01 6.875000e-018 7.417858e-01 7.368421e-019 7.570369e-01 7.272727e-0110 6.978259e-01 6.709667e-0111 7.433614e-01 7.241379e-0112 6.845083e-01 6.808511e-0113 6.986989e-01 6.862745e-0114 6.655176e-01 6.615384e-0115 6.613215e-01 6.433655e-01

16Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsQuadruple EqualitiesStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAn idea of Goemans developed by Anjos and Wolkowicz(2002):For each quadruple x i , x j , x j , x k ∈ {−1, 1} we have:x i x j x j x k = x i x 2j x k = x i x k . (5)With (5), we have the SDP formulation:Problem: An Equivalent Formulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, X ≽ 0, and x ij x jk = x ik .

16Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsQuadruple EqualitiesStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAn idea of Goemans developed by Anjos and Wolkowicz(2002):For each quadruple x i , x j , x j , x k ∈ {−1, 1} we have:x i x j x j x k = x i x 2j x k = x i x k . (5)With (5), we have the SDP formulation:Problem: An Equivalent Formulation of Max-Cut ProblemMaximize L • Xsubject to diag(X) = e, X ≽ 0, and x ij x jk = x ik .

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSecond lifting/relaxationStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAfter liftingY = [ y y T ] , where y = vec(X)We have the max-cut problem:Maximize[ ] L 0• Y = L • Y0 011subject to diag(Y ) = e (S1)Y ≽ 0(S2)Y 11 = Y ii ∀i = 2, . . . , n (S3)diag(Y ij ) = Y (i,j)iie (S4)Y ij e j = Y 11 e i ∀j = 2, . . . n, i < j (S5)rank(Y ) = 1.(S6)TheoremThe second lifting relaxation provides a tighter upper bound.17

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSecond lifting/relaxationStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAfter liftingY = [ y y T ] , where y = vec(X)We have the max-cut problem:Maximize[ ] L 0• Y = L • Y0 011subject to diag(Y ) = e (S1)Y ≽ 0(S2)Y 11 = Y ii ∀i = 2, . . . , n (S3)diag(Y ij ) = Y (i,j)iie (S4)Y ij e j = Y 11 e i ∀j = 2, . . . n, i < j (S5)rank(Y ) = 1.(S6)TheoremThe second lifting relaxation provides a tighter upper bound.17

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSecond lifting/relaxationStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAfter liftingY = [ y y T ] , where y = vec(X)We have the max-cut problem:Maximize[ ] L 0• Y = L • Y0 011subject to diag(Y ) = e (S1)Y ≽ 0(S2)Y 11 = Y ii ∀i = 2, . . . , n (S3)diag(Y ij ) = Y (i,j)iie (S4)Y ij e j = Y 11 e i ∀j = 2, . . . n, i < j (S5)rank(Y ) = 1.(S6)TheoremThe second lifting relaxation provides a tighter upper bound.17

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSecond lifting/relaxationStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAfter liftingY = [ y y T ] , where y = vec(X)We have the max-cut relaxation:Maximize[ ] L 0• Y = L • Y0 011subject to diag(Y ) = e (S1)Y ≽ 0(S2)Y 11 = Y ii ∀i = 2, . . . , n (S3)diag(Y ij ) = Y (i,j)iie (S4)Y ij e j = Y 11 e i ∀j = 2, . . . n, i < j (S5)TheoremThe second lifting relaxation provides a tighter upper bound.17

Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsSecond lifting/relaxationStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesAfter liftingY = [ y y T ] , where y = vec(X)We have the max-cut relaxation:Maximize[ ] L 0• Y = L • Y0 011subject to diag(Y ) = e (S1)Y ≽ 0(S2)Y 11 = Y ii ∀i = 2, . . . , n (S3)diag(Y ij ) = Y (i,j)iie (S4)Y ij e j = Y 11 e i ∀j = 2, . . . n, i < j (S5)TheoremThe second lifting relaxation provides a tighter upper bound.17

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

18Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsTighter Upper Bound TheoremStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesSuppose that X ∗ is an optimal solution to the first liftingrelaxation.If Rank(X ∗ ) = 1, then we have the optimal solution to theoriginal max-cut problem.If Rank(X ∗ ) > 1, we will show that after lifting X ∗ toY = (yy T ), where y = vec X ∗ , Y is not feasible to thesecond relaxation.By contradiction, we assume that Y is feasible to the secondrelaxation, then it satisfies (S4) and (S5), which implies thatXikX ∗kj ∗ = Xij ∗ for all i, j, k.Hence, X ∗ = xx T , where x = (X1k, ∗ X2k, ∗ · · · , Xnk), ∗ whichcontradicts Rank X ∗ > 1.Therefore, Y is not feasible to the second lifting relaxation.On the other hand, if Y is feasible to the second liftingrelaxation, by (S1) and (S2), diag(Y 11 ) = e and Y 11 ≽ 0.Then, Y 11 is feasible to the first lifting relaxation.□

19Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesNumerical Results: Comparison of AlgorithmsComparison of Algorithm Performancerelaxation n=4 n = 6 n = 8 n = 10SDP 2.00000000 6.43225787 9.52050490 17.0875592TRI 1.99999999 5.99999995 8.99999986 16.9999998QUAD 1.99999999 5.99999988 8.99999900 16.9999996

20Thanks!!Problem Set-upSolving Using SeDumiImproving the Implementation and ResultsStrengthening the Relaxation: Triangle InequalitiesStrengthening the Relaxation: Quadruple EqualitiesThanks for your attention!Semidefinite relaxation of the max-cut problemA. Dudek, K. Ghobadi, E. Kim, S. Lin, R. Spjut, J. ZhuMSRI Workshop on Continuous Optimization and its ApplicationsJuly 17, 2007