Stabilized Dynamic Constraint Aggregation (SDCA) for ... - gerad

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)1/27Stabilized Dynamic Constraint Aggregation(SDCA)for Solving Set Partitioning ProblemsPascal Benchimol (1) , Guy Desaulniers (1) , Jacques Desrosiers (2)———————————————(1) École Polytechnique de Montréal & GERAD, Canada(2) HEC Montréal & GERAD, CanadaColumn Generation 2012Bromont, Québec June 10-13, 2012

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)2/271 Dual Variable Stabilization (DVS)2 Dynamic Constraint Aggregation (DCA)3 Stabilized Dynamic Constraint Aggregation (SDCA)

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Primal-Dual pair of a linear problem3/27(P)minλs.t.∑c x λ x (1)x∈∆∑a x λ x = b [π ∈ R m ] (2)x∈∆λ x ≥ 0, ∀x ∈ ∆ (3)(D)maxππ ⊤ b (4)s.t. π ⊤ a x ≤ c x , ∀x ∈ ∆ (5)∆ is a finite set of combinatorial objects.Rows (or tasks) in (2) are indexed by t ∈ T := {1, . . . , m}.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Motivation4/27

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Motivation5/27

6/27Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)1.- Dual Variable Stabilization (DVS)du Merle et al. (1999), Oukil et al. (2007), Ben Amor et al. (2009)DVS penalizes a large portion of the dual solution space.A penalty function f t (π t ), ∀t ∈ T :continuous, concave, symmetric, and piecewise linear with 3 pieces.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Stabilization in the primal space7/27P εδ : A relaxation of the primal problem P ∆∑(P εδ ) min c x λ x − δ ⊤ −y − + δ ⊤ +y + (6)λ,yx∈∆∑s.t. a x λ x − y − + y + = b [π ∈ R m ] (7)x∈∆0 ≤ y − ≤ ε − [ω − ∈ R m ] (8)0 ≤ y + ≤ ε + [ω + ∈ R m ] (9)λ x ≥ 0, ∀x ∈ ∆ (10)

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)8/27To solve P, a sequence of linear approximations (relaxations), i.e.,stabilized problems P s (s ≥ 1) are solved with ε s and δ syielding objective values z 1 < z 2 < ... < z s < ... ≤ z ∗ .

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)8/27To solve P, a sequence of linear approximations (relaxations), i.e.,stabilized problems P s (s ≥ 1) are solved with ε s and δ syielding objective values z 1 < z 2 < ... < z s < ... ≤ z ∗ .

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)DVS algorithm9/271: Set s := 1 and choose stabilization parameters ε s and δ s .2: Solve stabilized problem P s : (λ s , y s ).3: if y s = 0 then4: Stop. λ s is optimal to P.5: else6: s := s + 1, update ε s and δ s , and return to Step 2.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)2.- Dynamic Constraint Aggregation (DCA)for set partitioning problems10/27Let Q := {T l } l∈L be a partition of the rows of T into clusters.∑(P Q ) minc x λ x (11)λx∈∆ Q∑s.t.a Qx λ x = 1 [π Q ∈ R |L| ] (12)x∈∆ Qλ x ≥ 0, ∀x ∈ ∆ Q (13)column a Qx ∈ {0, 1} |L| is a clustered version of a x ∈ {0, 1} m :component a Qxl = 1 if object x selects cluster l ∈ L, 0 otherwise.

11/27Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Each subset of identical rows defines a cluster in QFrom an integer solution

11/27Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Each subset of identical rows defines a cluster in QFrom an integer solutionFrom a fractional solution

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)12/27To solve P, a sequence of row-aggregated problems P Qh (h ≥ 1)are solved with various row-partitions Q h ,yielding objective values z 1 ≥ z 2 ≥ ... ≥ z h ≥ ... ≥ z ∗ .

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)12/27To solve P, a sequence of row-aggregated problems P Qh (h ≥ 1)are solved with various row-partitions Q h ,yielding objective values z 1 ≥ z 2 ≥ ... ≥ z h ≥ ... ≥ z ∗ .A column generation processP is solved using various clusters until an equivalence with theoriginal problem.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)12/27To solve P, a sequence of row-aggregated problems P Qh (h ≥ 1)are solved with various row-partitions Q h ,yielding objective values z 1 ≥ z 2 ≥ ... ≥ z h ≥ ... ≥ z ∗ .A column generation processP is solved using various clusters until an equivalence with theoriginal problem.λ Qhis optimal to P ifthere is no negative reduced cost variableswith respect to some disaggregated dual vector π ∈ R m .

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)DCA algorithm for set partitioning problems13/271: Select a row-partition Q.2: Solve row-aggregated problem P Q : (λ Q ; π Q ).3: Disaggregate the dual vector π Q to compute π ∈ R m .4: Check if there exist negative reduced cost variables.5: if no such variables exist then6: Stop. λ Q is optimal for P.7: else8: Update Q using a few negative reduced cost variables,and return to Step 2.(Each subset of identical rows defines a cluster.)

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Dual variable disaggregation for set partitioning problems14/27A sufficient condition:∑t∈T lπ t = π Ql , ∀l ∈ L. (14)

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Dual variable disaggregation for set partitioning problems14/27A sufficient condition:∑t∈T lπ t = π Ql , ∀l ∈ L. (14)Incompatible variables in ∆ ¯Q1 and ∆ ¯Q2 (Elhallaoui et al. 2005)Since the sum of the dual variables in cluster l is known by (14),the non-negativity constraints on the reduced costs of variables λ x ,x ∈ ∆ ¯Q 1∪ ∆ ¯Q 2, can be transformed into difference inequalities,i.e., the disaggregation is the solution of a shortest path problem.An incompatible column in ∆ ¯Q1 partially covers a single clusterwhereas a column in ∆ ¯Q 2is incompatible with two.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Results-1: DVS dominates standard CG and DCA15/27

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Results-1: DVS dominates standard CG and DCA15/27cpu DCAcpu DVScpu CGcpu DCAcpu DVScpu CGcpu DVSTasks Dayscpu DVSTasks Days5005.53 6.06 50061.28 48.67251000 5.84 8.26 1000 67.45 463.17Average 5.68 7.16 Average 64.37 255.92Table: CPU time ratios of standard CG and DCA over DVS

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)3.- Stabilized Dynamic Constraint Aggregation:integration of DCA within DVS to reduce MP cpu16/27

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)3.- Stabilized Dynamic Constraint Aggregation:integration of DCA within DVS to reduce MP cpu16/27DVS2500major iterations1 10 20 30 40420002cpu (sec.)150010000-2-4-6distance to optimal value500-800 50 100 150 200 250 300 350minor iterations-10cpu RMPcpu SPobjectiveFigure:DVS for a 2500-task, 5-day scenario

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)SDCA: Stabilized & row-aggregated problem17/27P s → PQsε s −, ε s +, δ−, s δ+, s y−, s y+ s → ε s Q− , εs Q+ , δs Q− , δs Q+ , y Q− s , y Q+s—————————————————————————PQ s → P s(λ s Q , ys Q , zs ) → (λ s , y s , z s )(π s Q , ωs Q , zs ) → (π s , ω s , z s )

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)SDCA: Stabilized & row-aggregated problem18/27Proposition 2Row-aggregated parameters: P s → P s QFor all l ∈ Lε s Q−,l := min t∈T l{ε s −,t},δ s Q−,l := ∑ t∈T lδ s −,t,ε s Q+,l := min t∈T l{ε s +,t};δ s Q+,l := ∑ t∈T lδ s +,tStabilized primal solution: P s Q→ P sλ s x := λ s Qx , ∀x ∈ ∆ Qλ s x := 0, ∀x ∈ ∆ \ ∆ Qy s −,t := y s Q−,l and y s +,t := y s Q+,l , ∀t ∈ T l, l ∈ L,

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)SDCA: Dual variable disaggregation19/27Define β s l,i :=i∑πl,j s ,j=1∀l ∈ L, i ∈ {1, . . . , |T l|}.The system of difference inequalities takes into account, for eachconstraint t ∈ T , the trust region carried out by the δ-parameters.β s l,|T l | = πs Q,l ,∀l ∈ Lβ s l,i − βs l,i−1 ≥ δs −,t, ∀t = (l, i) ∈ T such that y s −,t = 0βl,i s − βs l,i−1 = δs −,t, ∀t = (l, i) ∈ T such that 0 < y−,t s < ε s −,tβl,i s − βs l,i−1 ≤ δs −,t, ∀t = (l, i) ∈ T such that y−,t s = ε s −,tβl,i s − βs l,i−1 ≤ δs +,t, ∀t = (l, i) ∈ T such that y+,t s = 0β s l,i − βs l,i−1 = δs +,t,β s l,i − βs l,i−1 ≥ δs +,t,∀t = (l, i) ∈ T such that 0 < y+,t s < ε s +,t∀t = (l, i) ∈ T such that y+,t s = ε s +,t.Still solvable as a shortest path problem.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Results-3: DVS vs. SDCA20/27SDCA2500major iterations1 10 20 30 40420002cpu (sec.)150010000-2-4-6distance to optimal value500-800 50 100 150 200 250 300 350minor iterations-10cpu RMPcpu SPobjectiveFigure: SDCA for a 2500-task, 5-day scenario

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Computational Experiments21/27Stabilized Dynamic Constraint Aggregation (SDCA)within Column GenerationSet Partitioning model for the MDVSP.Random instances using the procedure of Carpaneto et al. (1989),modified by Oukil et al. (2007) to adjust the level of degeneracy.3 depots and between 500 and 3000 tasks spread over 2 or 5 days.5-day instances are highly degenerate.For each row-size, reported results are average over 6 instances.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Computational Experiments21/27Stabilized Dynamic Constraint Aggregation (SDCA)within Column GenerationSet Partitioning model for the MDVSP.Random instances using the procedure of Carpaneto et al. (1989),modified by Oukil et al. (2007) to adjust the level of degeneracy.3 depots and between 500 and 3000 tasks spread over 2 or 5 days.5-day instances are highly degenerate.For each row-size, reported results are average over 6 instances.** Initial primal & dual solutions: MDVSP relaxed as a SDVSP.** Barycenter initial dual estimate (Rousseau et al. 2007)

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Results-3: DVS vs. SDCA (MP CPU)22/272-day scenario5-day scenarioTrips DVS SDCA Rows (%) DVS SDCA Rows (%)500 7 4 84.2 7 3 71.91000 31 13 85.3 33 10 69.71500 109 44 85.6 109 20 68.82000 225 70 85.3 278 70 67.42500 471 136 85.2 575 89 69.73000 838 224 87.1 1160 159 68.9Total 1681 491 2162 350Table: Average Master Problem CPU time (sec)———————————Master Problem CPU times reduced by factors 3 and 7 vs. DVS.Row reduction to around 85% and 70%.SDCA robust with regard to increasing levels of degeneracy.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)References23/27DVS: Dual Variable StabilizationO. du Merle, D. Villeneuve, J. Desrosiers, and P. Hansen.Stabilized column generation.Discrete Mathematics, 194(1-3):229–237, 1999.doi:10.1016/S0012-365X(98)00213-1A. Oukil, H. Ben Amor, J. Desrosiers, and H. Elgueddari.Stabilized column generation for highly degenerate multiple-depotvehicle scheduling problems.Computers & Operations Research, 34(3):817–834, 2007.doi:10.1016/j.cor.2005.05.011H.M.T. Ben Amor, J. Desrosiers, and A. Frangioni.On the choice of explicit stabilizing terms in column generation.Discrete Applied Mathematics, 157(6):1167–1184, 2009.doi:10.1016/j.dam.2008.06.021L.-M. Rousseau, M. Gendreau, and D. Feillet.Interior point stabilization for column generation.Operations Research Letters, 35(5):660–668, 2007.doi:org/10.1016/j.orl.2006.11.004

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)References24/27DCA: Dynamic Constraint AggregationI. Elhallaoui, D. Villeneuve, F. Soumis, and G. Desaulniers.Dynamic aggregation of set partitioning constraints in columngeneration.Operations Research, 53(4):632-645, 2005.doi: 10.1287/opre.1050.0222.I. Elhallaoui, G. Desaulniers, A. Metrane, and F. Soumis.Bi-dynamic constraint aggregation and subproblem reduction.Computers & Operations Research, 35(5):1713-1724, 2008.doi: 10.1016/j.cor.2006.10.007.I. Elhallaoui, A. Metrane, F. Soumis, and G. Desaulniers.Multi-phase dynamic constraint aggregation for set partitioning typeproblems.Mathematical Programming, 123(2):345-370, 2010.doi: 10.1007/s10107-008-0254-5.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)References25/27SDCA: Stabilized Dynamic Constraint AggregationP. Benchimol, G. Desaulniers, and J. Desrosiers.Stabilized dynamic constraint aggregation for solving setpartitioning problems.Les Cahiers du GERAD, G-2011-56, 26 pages, 2011.IPS: Improved Primal SimplexV. Raymond, F. Soumis, and D. Orban.A new version of the improved primal simplex for degenerate linearprograms.Computers & Operations Research, 37(1):91–98, 2010.

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)26/27SDCA2500major iterations1 10 20 30 40420002cpu (sec.)150010000-2-4-6distance to optimal value500-800 50 100 150 200 250 300 350minor iterations-10cpu RMPcpu SPobjectiveFigure: SDCA for a 2500-task, 5-day scenario

Dual Variable Stabilization (DVS)Dynamic Constraint Aggregation (DCA)Stabilized Dynamic Constraint Aggregation (SDCA)Results-3: DVS vs. SDCA (Total CPU)27/27Tasks Days DVS SDCA Tasks Days DVS SDCA50020 21 50020 181000 126 126 1000 113 951500 380 342 1500 376 280252000 776 677 2000 859 6532500 1403 1138 2500 1683 11383000 2401 1942 3000 3035 1914Total 5106 4246 Total 6086 4098Table: Average Total CPU time (sec)