MPEC Problem Formulations in Chemical Engineering Applications

MPEC Problem Formulations in ChemicalEngineering ApplicationsB. T. Baumrucker ∗ , J. G. Renfro † , L.T. Biegler ‡June 16, 2007AbstractWith the development and widespread use of large-scale nonlinear programming(NLP) tools for process optimization, there has been an associatedapplication of NLP formulations with complementarity constraintsin order to represent discrete decisions. Also known as Mathematical Programswith Equilibrium Constraints (MPECs), these formulations can beused to model certain classes of discrete events and can be more efficientthan a mixed integer formulation. However, MPEC formulations andsolution strategies are not yet fully developed in process engineering. Inthis study, we discuss MPEC properties, including concepts of stationarityand linear independence that are essential for well-defined NLP formulations.Nonlinear programming based solution strategies for MPECs arethen reviewed and examples of complementarity drawn from chemical engineeringapplications are presented to illustrate the effectiveness of theseformulations.1 IntroductionMathematical programming methods have proven extremely valuable for thedesign and operation of chemical processes. This has been made possible bymore flexible modeling constructs, increased computing power, and a betterfundamental understanding of solution algorithms and problem formulations.Related optimization models have a number of options to represent discretedecisions, as in contact and friction problems in computational mechanics, equilibriumrelations in economics, and a wide variety of discrete events in processsystems. Many of these situations naturally lend themselves to complementarity∗ Chemical Engineering Department, Carnegie Mellon University, Pittsburgh, PA 15213USA† Honeywell International, 1250 West Sam Houston Parkway South, Houston, TX 77042,USA‡ Corresponding Author, Chemical Engineering Department, Carnegie Mellon University,Pittsburgh, PA 15213 USA, lb01@andrew.cmu.edu1

Assuming that x = 0 at the solution, then the active set c(x, y) = 0 consists ofg 1 and g 3 where:∇g 1 = [1 0] T (10)∇g 3 = [y 0] T (11)[ ] 1 0∇c T = ⇒ linearly dependent (12)y 0Failure of LICQ is noteworthy as uniqueness of the constraint multipliers holds ifand only if LICQ is satisfied. A weaker condition is the Mangasarian-FromovitzConstraint Qualification (MFCQ), which requires linear independence of theequality constraints and the existence of a search direction that lies both in thenull space of the equality constraint gradients and points into the interior ofthe region defined by the linearized active inequality constraints. MFCQ is anecessary and sufficient condition for boundedness of the multipliers. However,MPCCs violate MFCQ at all feasible points as well. Consequently, multipliersof the MPCC (3) will be non-unique and unbounded.Because these constraint qualifications do not hold, it is not surprising thatfinding an MPCC solution may be difficult. In order to classify this solution, wedefine a B-stationary point for an MPCC if it is feasible to the MPCC and d = 0is the solution to the following Linear Program with Equilibrium Constraints(LPEC):wheremind∇f(w ∗ ) T d (13a)s.t. g(w ∗ ) + ∇g(w ∗ ) T d ≥ 0 (13b)h(w ∗ ) + ∇h(w ∗ ) T d = 0(13c)0 ≤ x ∗ + d x ⊥ y ∗ + d y ≥ 0 (13d)w ∗ = (x ∗ , y ∗ , z ∗ )I X = {i : x ∗ i = 0}I Y = {i : y ∗ i = 0}Verification of this condition may require the solution of 2 m linear programswhere m is the cardinality of the biactive set I X ∩ I Y . A stronger, but moretractable form of stationarity for the MPCC problem is strong stationarity. Apoint is strongly stationary if it is feasible for the MPCC and d = 0 solves the5

following relaxed problem:mind∇f(w ∗ ) T d (14a)s.t. g(w ∗ ) + ∇g(w ∗ ) T d ≥ 0 (14b)h(w ∗ ) + ∇h(w ∗ ) T d = 0(14c)d x,i = 0, i ∈ I X \ I Y (14d)d y,i = 0, i ∈ I Y \ I X (14e)d x,i ≥ 0, i ∈ I X ∩ I Y (14f)d y,i ≥ 0, i ∈ I X ∩ I Y (14g)Strong stationarity implies no feasible descent direction for (14) and is equivalentto B-stationarity if the biactive set I X ∩ I Y is empty, or if the MPEC-LICQproperty holds [1]. MPEC-LICQ requires that the following set of vectors belinearly independent:{∇g i (w ∗ )|i ∈ I g } ∪ {∇h(w ∗ )} ∪ {∇x i |i ∈ I X } ∪ {∇y i |i ∈ I Y } (15)whereI g = {i : g i (w ∗ ) = 0}.MPEC-LICQ implies that the multipliers of (14) are bounded and unique. Ithas been shown that MPEC-LICQ is a generic property of MPCCs and of bilevelproblems formulated as MPCC [27] and that this property can be expected tohold “almost everywhere”. That is, if MPEC-LICQ is violated for a particularMPEC, it can always be satisfied by small perturbations of that problem.Satisfaction of MPEC-LICQ leads to the following result:Theorem 1 [1, 15, 26] If w ∗ is a solution to the MPCC (3) and MPEC-LICQholds at w ∗ , then w ∗ is strongly stationary.With this property, convergence to a strongly stationary point is much easier(and required for many NLP algorithms), since only one set of MPEC multipliersis needed to verify optimality and, consequently, this property allows thereformulation of (3) to a number of equivalent nonlinear programs. In the past,special MPEC solvers were required [4, 5, 17] to handle these difficulties. Withthis property, MPECs can now be addressed directly through the formulationand direct solution as NLPs. These NLP reformulations are covered in the nextsection.Finally, a related (but weaker and implied by MPEC-LICQ) constraint qualificationis MPEC-MFCQ, which requires that there exist a nonzero vector6

d ∈ R n such that:d x,i = 0, i ∈ I X \ I Y (16a)d y,i = 0, i ∈ I Y \ I X (16b)∇h(w ∗ ) T d = 0(16c)∇g i (w ∗ ) T d > 0 i ∈ I g (16d)d x,i > 0, d y,i > 0 i ∈ I X ∪ I Y (16e){∇h(w ∗ )} ∪ {∇x i |i ∈ I X } ∪ {∇y i |i ∈ I Y } are linearly independent (16f)MPEC-MFCQ implies that the multipliers of (14) will be bounded.3 Solution StrategiesIf no reformulation of an MPCC is made and the complementarity is representedby (5), (6) or (7), the constraint multipliers will be non-unique and unbounded,and dependent active constraints exist at every feasible point. However, someactive set NLP strategies may be able to overcome this difficulty as they weredeveloped to handle degenerate NLP formulations [2] and increase robustness ofactive set methods. In particular, encouraging results have been reported withthe Filter-SQP algorithm [6]. Moreover, elastic mode SQP algorithms havefinite termination and global convergence properties when solving MPCCs [1].On the other hand, an active set algorithm’s performance is still affected bycombinatorial complexity as the problem size increases.The following MPCC reformulations have been analyzed in [1, 11, 15, 22, 24]and allow standard NLP tools to be applied.Reg(ǫ): min f(w) (17a)s.t. h(w) = 0 (17b)g(w) ≥ 0(17c)x, y ≥ 0 (17d)x i y i ≤ ǫ ∀i (17e)RegComp(ǫ): min f(w) (18a)s.t. h(w) = 0 (18b)g(w) ≥ 0(18c)x, y ≥ 0 (18d)x T y ≤ ǫ(18e)7

RegEq(ǫ): min f(w) (19a)s.t. h(w) = 0 (19b)g(w) ≥ 0(19c)x, y ≥ 0 (19d)x i y i = ǫ ∀i (19e)PF(ρ): min f(w) + ρx T y (20a)s.t. h(w) = 0 (20b)g(w) ≥ 0(20c)x, y ≥ 0 (20d)Complementarity conditions may be relaxed and the problem reformulated asin the Reg(ǫ), RegComp(ǫ), or RegEq(ǫ) with a positive relaxation parameterǫ. The solution of the MPCC, w ∗ , can be obtained by solving a series of relaxedsolutions, w(ǫ), as ǫ approaches zero. These solution strategies are generally attractedto strongly stationary points, but with some exceptions (see [1]). Underassumptions of MPEC-MFCQ and sufficient second order MPEC conditions [24],Reg(ǫ) and RegComp(ǫ) exhibit local convergence of ‖w(ǫ) − w ∗ ‖ ≤ O ( ǫ 1/2) .(These properties can be strengthened to O (ǫ) if we can assume strict complementarityof the bound multipliers for i ∈ I X ∪ I Y .) Reg(ǫ) has been implementedand tested in a version of IPOPT called IPOPT-C [22]. On the otherhand, RegEq(ǫ) will exhibit slower convergence with ‖w(ǫ) − w ∗ ‖ ≤ O ( ǫ 1/4)under the same assumptions.In a related development, Nonlinear Complementarity Problem (NCP) functionsand smoothing functions have also been used extensively to solve MPCCs[3, 10, 14, 28]. NCP functions replace the complementarity condition in theproblem with an equivalent nonlinear equation. One widely studied NCP functionis the Fischer-Burmeister function:φ(x, y) = x + y − √ x 2 + y 2 (21)As this function is non-differentiable at x = y = 0, it is usually smoothed to theform:φ(x, y) = x + y − √ x 2 + y 2 + ǫ (22)for some small ǫ > 0. The solution to the original problem is then recovered bysolving a series of problems as ǫ approaches zero. An equivalence can be madebetween this method and RegEq(ǫ). Accordingly, the problem will converge atthe same slow convergence rate.In contrast to the regularized formulations, we also consider the exact l 1penalization shown in PF(ρ). Here the complementarity can be moved fromthe constraints to the objective function and the resulting problem is solvedfor a particular value of ρ. If ρ ≥ ρ c , where ρ c is the critical value of thepenalty parameter, then the complementarity constraints will be satisfied at8

the solution. Similarly, Anitescu et al. [1] consider a related “elastic mode”formulation, where artificial variables are introduced to relax the constraints inPF(ρ) and an additional l ∞ constraint penalty term is added. In both cases theresulting NLP formulation has the following properties:Theorem 2 [1] If w ∗ is a solution to PF(ρ) and w ∗ is feasible for the MPCC(3), then w ∗ is a strongly stationary solution to (3).However, the value of ρ c is not known beforehand. If the initial value is toosmall a series of problems with increasing ρ values may need to be solved, andhere we can apply the following result:Theorem 3 [15] If w ∗ is a limit point of a sequence of solutions {w k } toPF(ρ k ), MPEC-LICQ holds and ρ k x k i → 0 and ρ k yi k → 0 for i ∈ I X ∪ I Y ,then w ∗ is a strongly stationary solution to (3).Hence, the solution strategy is attracted to strongly stationary points, andwith a large enough ρ this l 1 penalization is exact in the sense that all stronglystationary points of the MPCC are local minimizers to PF(ρ). PF(ρ) allowsany general nonlinear programming solver to be used to solve a complementarityproblem. Provided that the penalty parameter is large enough, the MPCCmay then be solved as a single problem, instead of a series of problems. Finally,in order to provide greater numerical efficiency, it has been suggested todynamically change the penalty parameter during the course of the optimizationfor both active set and interior point optimization algorithms [1, 15]. Thismodification was recently introduced into the solver KNITRO [16].Finally, there is a strong interaction between the MPCC reformulation andthe applied NLP solver. In particular, as noted in [15], if a barrier NLP method(like KNITRO or IPOPT) is applied to PF(ρ), then there is a one-to-one correspondenceof intermediate solutions of PF(ρ), Reg(ǫ) and RegComp(ǫ). Thisequivalence can be seen by comparing the penalty parameter ρ to the multiplierassociated with the relaxed complementarity constraints in Reg(ǫ) andRegComp(ǫ). As a result, for a corresponding set of parameters ρ and ǫ wewould expect a barrier NLP method to exhibit similar performance with eitherPF(ρ), Reg(ǫ) or RegComp(ǫ).4 MPECLib ResultsThe formulations of the previous section provide a straightforward way to representand address any MPCC through nonlinear programming. An importantcomponent of this task is the efficient and reliability of the NLP solver. In thissection we compare and evaluate each of these formulations through a librarytest set provided with the GAMS modeling environment [8]. For this evaluationwe consider MPECLib, a large collection of test problems, and apply an automaticreformulation of MPECs provided by the NLPEC meta-solver in GAMS.9

4.1 Automatic Reformulation of ComplementarityMPCC models written in the GAMS modeling language may be automaticallyreformulated as an NLP using several of the previously described complementaritysolution techniques. This can be accomplished with the use of the NLPECmeta-solver [8]. When called, NLPEC reformulates the complementarity constraintsof a MPCC model with a user specified reformulation, including (17)-(20). NLPEC will then call a user-specified NLP solver to solve the reformulatedmodel. The final results from the NLP solver are then translated back into theoriginal MPCC model. NLPEC verifies that the complementarities are satisfiedin the resulting solution. This is particularly important when using the PF(ρ)formulation described above, as it may not converge to a stationary point of theMPCC if the penalty parameter is too small.4.2 Comparison of Solution Strategies on MPECLibMPECLib is a set of MPEC test problems maintained by Dirkse [19]. The testset currently consists of 92 problems, which include both small scale modelsfrom the literature and several large industrial models.The performance and robustness of several NLP reformulations includingPF(ρ) with ρ = 10, Reg(ǫ) with ǫ = 10 −8 , and NCP formulation x T y = 0,x, y ≥ 0 were compared on the MPECLib test set. CONOPT (version 3.14) andIPOPT (version 3.2.3) were used to the solve the resulting NLP problems. Allreformulations were performed automatically by the NLPEC solver describedearlier. Also included in the comparison is the IPOPT-C solver, which is capableof processing the complementarity constraints without reformulation. TheCONOPT and IPOPT runs were performed entirely within the GAMS environment.However, IPOPT-C is not currently linked to GAMS. In order toperform the tests with IPOPT-C, the GAMS models were instead convertedto the AMPL modeling language format. The problems were then run usingAMPL and IPOPT-C.The results are presented in a Dolan-Morè plot. All results were obtained ona Intel Pentium 4, 1.8 GHz computer with 992 MB of RAM. This plot portraysboth robustness and relative performance of a set of solvers on a given problemset. Here we define a performance ratio calculated as:η p,s =t p,smin {t p,s : 1 ≤ s ≤ n s }(23)where n s is the number of solvers, which are run over the n p problems in set P,and where t p,s is the solution time for solver s on problem p. For all solvers sthat fail to solve problem p, we set the performance ratio to an arbitrarily largenumber. The profiles are then defined by:p s (τ) = 1 n psize {p ∈ P : η p,s ≤ τ} (24)10

and p s (τ) is the percentage of problems solved within τ ≥ η p,s times the bestperformance. This analysis can be performed automatically using the PAVERserver [18].Figure 1 shows the performance plot of different reformulations and solverstested on all 92 problems of the test set. Since the problems are inherentlynonconvex, multiple local solutions can be expected for each problem. Becauseof this, not all of the different reformulation and solver combinations convergedto the same local solutions. This makes a direct comparison of performancedifficult as the solvers often converge to different solutions. On the other hand,this figure is useful to demonstrate the robustness of the methods, as seen bylim τ →∞ p s (τ). For all but one solution method, the reformulation and solvercombination was able to solve at least 84% of all of the test problems. Themost reliable solvers were IPOPT-Reg(ǫ) (99%), CONOPT-Penalty (98%) andIPOPT-Penalty (94 %). In contrast, IPOPT with the NCP formulation (listedas IPOPT-Mult on the figure) was only able to solve 57% of all of the problems.This is not entirely unexpected as IPOPT is an interior point algorithm and theNCP formulation provides no strict interior to the NLP problem.Figure 2 shows the same reformulations and solvers tested as before excepton only 22 of the 92 test problems. These 22 problems all converged to the samesolution for all reformulations and solvers when they converged. This allows fora fair comparison of the solvers’ performance. Table 1 displays these 22 testproblems and their respective objective function values.A few observations can be made on these results. CONOPT-Reg(ǫ) performedthe fastest on 73 % of the problems but it turns out to solve only 76%of them. Instead, CONOPT-Penalty and IPOPT-Penalty (PF(ρ) are the nextbest in performance and they also prove to the most robust, solving over 90%of the problems. These turn out to be the best all-around methods.For the results in Figure 2, the CONOPT formulations take advantage ofthe active set strategy and particularly the detection and removal of degenerateconstraints. As a result, the CONOPT-Mult (with the NCP reformulation)performs well; this method is fastest on 50% of the problems but no morerobust than CONOPT-Reg. Because the test problems are not large, CONOPTprovides the best performance on this test set.On the other hand, the IPOPT formulations (IPOPT-Penalty, IPOPT-Reg,IPOPT-C) follow similar trends within Figure 2, with the penalty formulation asthe most robust. This can be explained by the similarities among these methods,as analyzed in [15]. Finally, IPOPT-Mult (with the NCP reformulation) is theworst performer as it cannot remove dependent constraints and therefore suffersmost from the inherent degeneracies in MPCCs.5 Process Engineering Examples of MPCCsComplementarity applications in chemical engineering are widespread for modelingdiscrete decisions. Their use for modeling and optimizing split range controllersand tiered splitters has been known for some time [7]. However, with a11

100Performance Profile9080Percent Of Models Solved7060504030CONOPT-PenaltyIPOPT-Penalty20CONOPT-MultIPOPT-MultCONOPT-Reg10IPOPT-RegIPOPT-CCAN_SOLVE01 10 100 1000Time FactorFigure 1: Performance Profiles of several reformulations on all of the 92MPECLib Test Problems100Performance Profile9080Percent Of Models Solved70605040CONOPT-Penalty30IPOPT-PenaltyCONOPT-MultIPOPT-MultCONOPT-Reg20IPOPT-RegIPOPT-CCAN_SOLVE101 10 100 1000Time FactorFigure 2: Performance Profiles of several reformulations on 22 Select Test Problems12

Problem Name Objective Function Value Constraints Variables Complementaritiesbard2 -6600 10 13 8bard3 -12.67872 6 7 4bartruss3 0 3.54545E-07 29 36 26bartruss3 1 3.54545E-07 29 36 11bartruss3 2 10166.57 29 36 6bartruss3 3 3E-07 27 34 26bartruss3 4 3E-07 27 34 11bartruss3 5 10166.57 27 34 6desilva -1 5 7 4ex9 1 4m -61 5 6 4findb10s 2.02139E-07 203 198 176fjq1 3.207699 7 8 6gauvin 20 3 4 2kehoe1 3.6345595 11 11 5outrata31 2.882722 5 6 4outrata33 2.888119 5 6 4outrata34 5.7892185 5 6 4qvi 7.67061E-19 3 5 3three 2.58284E-20 4 3 1tinque dhs2 N/A 4834 4805 3552tinque sws3 12467.56 5699 5671 4480tollmpec -20.82589 2377 2380 2376Table 1: Objective Function Values of the 22 MPECLib Test Problems ThatConverged to Same Solutions13

etter understanding of NLP-based solution strategies for MPCCs, it is essentialto consider the formulation of well-posed complementarity models. In thissection, we consider a systematic MPCC modeling strategy and apply it to anumber of widely applied process examples.To develop these concepts, we again caution that complementarity formulationsare nonconvex problems and may lead to multiple local solutions. Consequently,with the application of standard NLP solvers, the user will need tobe satisfied with local solutions, unless additional problem-specific informationcan be applied. Despite the nonconvexity in all MPCC formulations, it is clearthat poorly posed MPCC formulations also need to be avoided. For instance,the feasible region for:0 ≤ y ⊥ (1 − y) ≥ 0 (25)consists of only two isolated points, and the disjoint feasible region associatedwith this example often leads to convergence failures, unless the model is initializedclose to a solution with either y = 0 or y = 1. Moreover, without carefulconsideration, it is not difficult to create MPCCs with similar difficulties.Because the MPCC can be derived from an associated MPEC, we considerthe formulation in (2). To avoid disjoint regions, we require that both θ(x, y)and C(x) be convex in y for all (x, y) ∈ Z. This leads to a well-defined solutionfor the inner problem. Moreover, the resulting complementarities are then“strongly monotone”, which have been shown to be very useful in practice. Thisobservation leads to the following guidelines for the examples in this study:• Start with the inner level problem:minyϕ(x)y s.t. y a ≤ y ≤ y b (26)where ϕ(x) is a switching function for the discrete decision.• For (26) it is especially important that the upper level constraints in (2),i.e., (x, y) ∈ Z, not limit the selection of any value of y ∈ C(x).• The resulting MPEC is then converted to an MPCC. For instance, applyingthe KKT conditions to (26) we obtain the complementaritiesϕ(x) − s a + s b = 0 (27a)0 ≤ (y − y a ) ⊥ s a ≥ 0 (27b)0 ≤ (y b − y) ⊥ s b ≥ 0 (27c)• The relations in (27) can be further simplified through variable eliminationand application of the complementarity conditions.• Finally, the resulting complementarity conditions are incorporated withinNLPs using the formulations described in section 3.14

Note that in the absence of upper level constraints, the solution of (27)allows y to take any value in [y a , y b ] when ϕ(x) = 0. This is necessary to avoiddisjoint feasible regions for the resulting MPCC. As a result, limitations on yfrom upper level constraints should be avoided, and this is frequently a problemspecific task. For instance, complementarity formulations should not be used tomodel EXCLUSIVE OR (as in (25)) because they lead to disjoint regions for y.In the remainder of this section, we apply these guidelines to develop complementaritymodels that arise in process applications. A number of these canalso be found in [21].5.1 Commonly Used FunctionsThe following nonsmooth functions can be represented by complementarity formulations.• The signum or sign operation y = sgn(x) can be represented by a complementarityformulation:min −y · x s.t. − 1 ≤ y ≤ 1 (28)and the corresponding complementarity relations can be written as:x = s b − s a , 0 ≤ s a ⊥ (y + 1) ≥ 0, 0 ≤ s b ⊥ (1 − y) ≥ 0 (29)• The absolute value operator z = |f(x)| can be rewritten as:z = f(x)y,y = arg{min −f(x)ŷ s.t. − 1 ≤ ŷ ≤ 1} (30)ŷThe corresponding KKT conditions are:z = f(x)y, f(x) = s b − s a0 ≤ s b ⊥ (1 − y) ≥ 0, 0 ≤ s a ⊥ (y + 1) ≥ 0These relations can be simplified by applying the complementarity conditionsto eliminate the variable y in the first two equations, leading to:z = s b + s a , f(x) = s b − s a (31)0 ≤ s b ⊥ s a ≥ 0• The max operator z = max{f(x), z a } can be rewritten as:z = f(x)+(z a −f(x))yy = arg{minŷ(f(x) −z a )ŷ s.t. 0 ≤ ŷ ≤ 1} (32)The corresponding KKT conditions are:z = f(x) + (z a − f(x))y f(x) − z a = s a − s b0 ≤ s b ⊥ (1 − y) ≥ 0, 0 ≤ s a ⊥ y ≥ 015

Simplifying the complementarity conditions to eliminate the variable y inthe first two equations leads to:z = f(x) + s b , f(x) − z a = s a − s b (33)0 ≤ s b ⊥ s a ≥ 0• The min operator y = min (f(x), y b ) can be treated in a similar way bydefining the problem:z = f(x) + (f(x) − z b )yy = arg{minŷ(f(x) − z b )ŷ s.t. 0 ≤ ŷ ≤ 1} (34)Applying the KKT conditions and simplifying leads to the following complementaritysystem:5.2 Flow Reversalz = f(x) + s a , z b − f(x) = s b − s a (35)0 ≤ s b ⊥ s a ≥ 0Flow reversal can occur in many places in chemical processes including fuelheaders and pipeline distribution networks. This is problematic for modelingsince physical properties and other equations implicitly assume the sign of theflowrate to be positive. These situations may be modeled with the absolute valueoperator, using complementarities as in (31). The magnitude of the flowrate canthen be used in the sign sensitive equations.On the other hand, it may not be possible to implement these changes insome modeling environments, such as commercial process optimization softwarewith directional streams and integrated thermo-physical models. In this case,each stream with the potential for flow reversal can be modeled as two parallelcounter-current streams. The stream flowrates are then complemented, suchthat either one stream or the other has zero flow. Using mixers and splitters asappropriate, the two streams can then be integrated into any process flowsheet[9]. While this method may obscure the original representation of the system,it remains equivalent.5.3 Relief Valves, Check Valves, Compressor Kick-BackRelief valves, check valves and compressor kick-back operation are all relatedphenomenon that can be modeled using complementarities in a manner similarto the functions used in Section 5.1.• Check valves prevent flows in reverse directions that may arise from changesin differential pressure. Assuming that the directional flow is a monotonicfunction of the differential pressure, we model flow through the check valveas:flow = max{0, f(∆p)}and rewrite the max operator as the complementarity system given in (33).16

Figure 3: Flowsheet Example of Compressor Kick-Back• As shown in Figure 3, compressor kickback, where recirculation is requiredwhen the compressor feed drops below a critical value, can be modeled ina similar manner. Following the stream notation in Figure 3 we can write:f S2 = max(f S1 , f crit ), f S5 = f crit − f S1and rewrite the max operator using the complementarities in (33).• Relief valves open and allow flow only when a pressure, p, or pressuredifferential, ∆p, is larger than a predetermined value. Once open the flowis some function of the pressure, f(p). This can be modeled as:flow = f(p)y (36)minŷ(p max − p)ŷ (37a)s.t. 0 ≤ ŷ ≤ 1 (37b)The inner minimization problem sets y = 1 if p > p max and sets y = 0if p < p max . This zero-one behavior determines if the flowrate should bezero, when valve is closed, or determined by the expression f(p), when thevalve is open. The related complementarity conditions are:flow = f(p)y(p max − p) = s 0 − s 10 ≤ y ⊥ s 0 ≥ 00 ≤ (1 − y) ⊥ s 1 ≥ 05.4 Piecewise FunctionsPiecewise smooth functions are often encountered in physical property models,tiered pricing and table lookups. The composite function can be represented by17

the following inner minimization problem and associated equation [21]:minys.t.N ∑i=1(x − a i )(x − a i−1 )y i (38a)N∑y i = 1i=1y i ≥ 0(38b)(38c)z =N∑f i (x)y i (39)i=1where N is the number of piecewise segments, f i (x) is the function over theinterval x ∈ [a i−1 , a i ], and z represents the value of the piecewise function. ThisLP will set y i = 1 and y j≠i = 0 when x ∈ [a i−1 , a i ]. The associated equationwill then set z = f i (x), which is the function value on the interval. The NLP(38) can be rewritten as the following complementarity system:N∑y i = 1 (40a)i=1(x − a i )(x − a i−1 ) − γ − s i = 0 (40b)0 ≤ y i ⊥ s i ≥ 0 (40c)In some cases, the function of interest may only be piecewise continuousand y i can “cheat” by taking fractional values. For instance, cost per unit mayincrease stepwise over different ranges, and jumps in the costs may occur at a i .A way around this problem would be to define f i (x) as the total cost (i.e., unitcost times quantity), which is continuous everywhere, but not differentiable ata i .5.5 PI Controller SaturationPI controller saturation has been studied using complementarity formulations[30]. The PI control law takes the following form:(u(t) = K c e(t) + 1 ∫ t)e(t ′ )dt ′ (41)τ Iwhere u(t) is the control law output, e(t) is the error in the measured variable,K c is the controller gain and τ I is the integral time constant. The controlleroutput v(t) is typically subject to upper and lower bounds, v up and v lo , i.e.,v = max(v lo , min(v up , u(t))). The following inner minimization relaxes thecontroller output to take into account the saturation effects:0miny up,y lo(v up − u(t))y up + (u(t) − v lo )y lo (42)s.t. 0 ≤ y lo , y up ≤ 118

with v = u(t) + (v up − u(t))y up + (v lo − u(t))y lo . Applying the KKT conditionsto (42) leads to:v = u(t) + (v up − u(t))y up + (v lo − u(t))y lo (43)(v up − u(t)) − s 0,up + s 1,up = 0(u − v lo ) − s 0,lo + s 1,lo = 00 ≤ s 0,up ⊥ y up ≥ 0, 0 ≤ s 1,up ⊥ (1 − y up ) ≥ 00 ≤ s 0,lo ⊥ y lo ≥ 0, 0 ≤ s 1,lo ⊥ (1 − y lo ) ≥ 0Eliminating the variables y up and y lo and applying the complementarity conditionsleads to the following simplification:v = u(t) + s 1,lo − s 1,up (44)(v up − u(t)) − s 0,up + s 1,up = 0(u − v lo ) − s 0,lo + s 1,lo = 00 ≤ s 1,up ⊥ s 0,up ≥ 0, 0 ≤ s 0,lo ⊥ s 1,lo ≥ 05.6 Disappearance of PhasesThe disappearance of phases in vapor liquid equilibrium systems was consideredin [10]. Here the equilibrium between two phases is relaxed by an additionalvariable β and defined by y j = βK j x j . For a flash vessel with feed F, thecomplementarity system can be derived from the following inner minimimizationproblem:minL,Vleading to the following complementarity system:(1 − β)(L − V ) (45a)s.t. L + V = F (45b)L, V ≥ 0 (45c)y j = βK j x jβ = 1 − s l + s v(46a)(46b)0 ≤ L ⊥ s l ≥ 0 (46c)0 ≤ V ⊥ s v ≥ 0 (46d)In this manner, phase existence can be determined within the context of anoptimization problem. If either liquid or vapor is absent, the correspondingslack variable may become positive. This in turn relaxes the calculation ofthe phase equilibrium, which is appropriate since only one phase is present.As seen in [10, 14, 20, 23], this formulation can also be extended to modeldistillation columns, and, as shown in [10], the conditions are also equivalent tominimization of Gibbs free energy.19

6 Case StudiesFrom the results of Sections 3 and 4, we find that the penalty formulationperforms well, particularly with an active set solver such as CONOPT. With thisexamination of NLP reformulations for MPCC as well as a systematic strategyfor developing complementarity formulations, we now consider two large-scalecase studies that illustrate both of these concepts. The first considers discretedecisions in dynamic systems with the reformulation of the signum function,while the second considers the disappearance of phases in optimization modelsfor distillation.6.1 Dynamic Optimization with Direct TranscriptionThe goal of this case study is to demonstrate the NLP formulations in Section 3on an example with arbitrarily many complementarity conditions. The exampleproblem is derived from discretization of a differential inclusion (ẋ ∈ sgn(x)+2),where the derivative can be defined by a complementarity system [29]. While wedo not consider limiting properties of the discretization, our numerical resultswill focus on how the solution time varies as a function of the problem size(controlled by the discretization strategy).The differential inclusion can be reformulated as the following optimal controlproblem:∫ tendmin (x end − 5/3) 2 + x 2 · dt (47a)t 0s.t. ẋ = u + 2 (47b)x = s + − s −(47c)0 ≤ 1 − u ⊥ s + ≥ 0 (47d)0 ≤ u + 1 ⊥ s − ≥ 0 (47e)Applying the implicit Euler’s method with a fixed step size, the discretizedproblem becomes:NFE∑min (x end − 5/3) 2 + h · x 2 ii=1(48a)s.t. ẋ i = u i + 2 i = 1, . . .,NFE (48b)x i = x i−1 + h · ẋ i i = 1, . . .,NFE (48c)x i = s + i− s − ii = 1, . . .,NFE (48d)0 ≤ 1 − u i ⊥ s + i ≥ 0 i = 1, . . .,NFE (48e)0 ≤ u i + 1 ⊥ s − i ≥ 0 i = 1, . . .,NFE (48f)The resulting discretized problem can be scaled up to be arbitrarily large byincreasing the number of finite elements NFE, with a discrete decision in each20

element. This system was modeled in GAMS and solved via automatic reformulationusing the NLPEC package. Additionally, the GAMS model was convertedto an AMPL model and the problems were solved using IPOPT-C.This dynamic optimization problem could also be modeled easily as anMIQP. However, finding a solution is NP-Hard and with an increasing discretization,the problem can become expensive. Solving this system as an MPCCgenerally leads to only polynomial complexity. On the other hand, the MPCCmodel is nonconvex and global optimality cannot be ensured, although on thisexample a comparison with MIQP solvers showed that global solutions werealways obtained by the MPCC solver.The discretized MPCC was solved using the penalty reformulation (PF(ρ)with CONOPT using ρ = 1000). The results are shown in Table 2. It can be seenthat the number of iterations increases almost linearly with the number of finiteelements and this behavior is typical for active set strategies. In addition, thecomputational time per iteration increases linearly with the number of finiteelements, as the majority of CPU time is expected to be spent in the sparselinear solver. In this case, sparse linear solvers increase approximately linearlyin computation time with problem size. Consequently, the total computationaltime for CONOPT-PF grows approximately quadratically with problem size.IPOPT-C was also used to solve this problem and the results are shown inTable 2. It can be seen that the number of iterations increases only weakly as afunction of problem size, with no clear trend. In fact, the iteration count seemsto saturate with problem size. In addition, the computational time per iterationalso increases linearly with the number of finite elements. This is expected sincethe largest computational cost of IPOPT is due to the linear solver. Here, thissolver cost increases linearly with problem size. As a result, the total computationaltime increase is only slightly superlinear with problem size. Moreover, itis also notable that while CONOPT is faster for the smaller problems, IPOPT-C becomes faster for the larger problems. This is consistent with expectations.CONOPT must also deal with the combinatorial complexity of identifying theactive set at every iteration, whereas the barrier method IPOPT-C identifiesthe active set only at the solution.Finally, the SBB solver, a simple branch and bound solver in GAMS, wasapplied to an MIQP formulation of this problem. Here, binary variables replaceu i in every finite element (48) and corresponding relations are added to reflectthe sign of x i . From the results in Table 2, we can observe the NP-hard aspectof solving the MIQP with this approach. Solving the problem with 1000 finiteelements requires over 400 CPUs and the problem with 2000 finite elementscannot be solved within 6000 CPUs.6.2 Distillation OptimizationAs seen in Section 5.6, phase behavior of a vapor liquid system is determinedby the minimization of the Gibbs free energy and can be modeled throughcomplementarity constraints. These conditions allow phase disappearance tobe modeled in distillation systems for optimization of both steady state and21

Objective SBB CONOPT-PF IPOPT-CNFE Function CPU s. CPU s. Iterations CPU s. Iterations10 1.4738 0.242 0.047 15 0.110 17100 1.7536 50.266 0.234 78 1.250 411000 1.7864 410.195 9.453 680 28.406 782000 1.7888 >6000 35.359 1340 14.062 253000 1.7894 — 112.094 2020 106.188 844000 1.7892 — 211.969 2679 84.875 565000 1.7895 — 340.922 3342 199.391 876000 1.7898 — 468.891 3998 320.140 1157000 1.7896 — 646.953 4655 457.984 1418000 1.7898 — 836.891 5310 364.937 98Table 2: Solution times (Pentium 4, 1.8 GHz, 992 MB RAM) for differentsolution strategiesdynamic tray columns [14, 20, 23]. In these studies smoothing methods andregularization methods were applied for distillation optimization, with the refluxratio, feedtray location and tray number as decision variables. In this section, werevisit these steady state distillation optimizations for the sake of a performancecomparison, particularly with the penalty formulation, PF(ρ).Here we define distillation column models that consist of MESH (Mass balance,Equilibrium, Summation, and Heat balance) equations that are modifiedto optimize the feed tray location and total tray count. As shown in Figure4, streams for the feed and the reflux are fed to all trays as dictated by twodiscretized Gaussian distribution functions. As developed and described in [14],both distribution functions are defined by a mean and standard deviation. Forthe feed and reflux distributions, their means represent the feedtray locationand the total number of trays, respectively, and these are used as continuousdecision variables in the distillation optimization. We also choose a standarddeviation of 0.5, so that distribution flow to trays away from the mean tapersoff smoothly to negligible values. Moreover, the grayed area in Figure 4 consistsonly of vapor traffic, and consequently, each tray model must include complementaritiesthat allow for disappearance of the liquid phase. As described inSection 5.6, phase equilibrium is relaxed by the complementarity constraints:y ij = β i K ij x ijβ i = 1 − s l i + sv i(49a)(49b)0 ≤ L i ⊥ s l i ≥ 0 (49c)0 ≤ V i ⊥ s v i ≥ 0 (49d)where i = {1, 2, . . .,N max } and N max is the maximum number of equilibriumstages, j is the component index, L i and V i are the liquid and vapor flow rateon tray i, and x and y are the respective mole fractions.The resulting complementarity distillation model is an alternative nonlin-22

Figure 4: Diagam of distillation column showing that the feed and reflux flowsare distributed across multiple trays according to the DDF. The grayed sectionof the column is above the primary reflux location and has negligible liquidflows.ear program for distillation design and optimization. As shown in [14, 23], itdetermines the optimal number of trays, reflux ratio and feedtray location fordetailed column models. As case studies we consider two systems drawn from[14]. The first system is a benzene/toluene separation and the second systemis an argon column from an air separation unit. Both models use ideal thermodynamics,are modeled in GAMS and solved with CONOPT. Three MPCCformulations were considered for these cases:• Penalty formulation, PF(ρ) with ρ = 1000 chosen as the penalty weightfor the benzene/toluene separation and ρ = 10 6 for the argon column.• Relaxed formulation, Reg(ǫ). This strategy was solved in two NLP stageswith ǫ = 10 −6 followed by ǫ = 10 −12 . This formulation was also appliedto distillation optimization in [23].• NCP formulation using the Fischer-Burmeister function (22). This approachis equivalent to RegEq(ǫ) and was solved in three NLP stages withǫ = 10 −4 followed by ǫ = 10 −8 and ǫ = 10 −12 . This strategy was appliedto distillation optimization in [14].Benzene-Toluene Column OptimizationThis binary column has a maximum of 25 trays, its feed is 100 mol/s of a70%/30% mixture of benzene/toluene and distillate flow is specified to be 50%23

of the feed. The reflux ratio is allowed to vary between one and 20, the feedtraylocation varies between 2 and 20 and the total tray number varies between 3and 25. The objective function for the benzene-toluene separation minimizes:objective = wt · D · x D,Toluene + wr · r + wn · Nt (50)where Nt is the number of trays, r is the reflux ratio, D is distillate flow,x D,Toluene is the toluene mole fraction and wt, wr, and wn are the weighingparameters for each term; these weights allow the optimization to trade offproduct purity, energy cost and capital cost. The benzene-toluene optimizationswere initialized with 21 trays, a feedtray location at the seventh tray and areflux ratio at 2.2. Temperature and mole fraction profiles were initialized withlinear interpolations based on the top and bottom product properties. Theresulting GAMS models consisted of 353 equations and 359 variables for theReg(ǫ) and NCP formulations, and 305 equations and 361 variables for thePF(ρ) formulation. The following cases were considered:• Case 1 (wt = 1, wr = 1, wn = 1): This represents the base case with equalweights on toluene in distillate, reflux ratio and tray count. As seen in theresults in Table 3, the optimal solution has an objective function value of9.4723 with intermediate values of r and Nt. This is found quickly by thePF(ρ) formulation. On the other hand, the Reg(ǫ) formulation terminatesclose to this solution, while the NCP formulation terminates early withpoor progress.• Case 2 (wt = 1, wr = 0.1, wn = 1): In this case, less emphasis is given toenergy cost. As seen in the results in Table 3, the optimal solution nowhas a lower objective function value of 6.8103 along with a higher value ofr and lower value of Nt. This is found quickly by both PF(ρ) and Reg(ǫ),although only the former satisfies the convergence tolerance. On the otherhand, the NCP formulation again terminates early with poor progress.• Case 3 (wt = 1, wr = 1, wn = 0.1): In contrast to Case 2, less emphasisis now given to capital cost. As seen in the results in Table 3, the optimalsolution now has an objective function value of 2.9048 with lowervalues of r and higher values of Nt. This is found quickly by the Reg(ǫ)formulation. Although Reg(ǫ) does not satisfy the convergence tolerance,the optimum could also be verified by PF(ρ). On the other hand, PF(ρ)quickly converges to a slightly different solution, which it identifies as a localoptimum, while the NCP formulation requires more time to terminatewith poor progress.Argon Column OptimizationThis argon separation column has a maximum of 63 trays, its feed is 6546.54lbmol/h of a 0.005%/9.753%/90.24% mixture of nitrogen/argon/oxygen anddistillate flow is specified to be 202.4576 lbmol/h with less than 1 mol % oxygen.24

Cases 1 2 3 Argoniter-PF(ρ) 356 395 354 1182iter-Reg(ǫ) 538 348 370 10269iter-NCP 179 119 254 328CPUs-PF(ρ) 7.188 7.969 4.875 69.281CPUs-Reg(ǫ) 14.594 7.969 7.125 706.406CPUs-NCP 7.906 3.922 9.188 20.781obj-PF(ρ) 9.4723 6.8103 3.0053 77.248obj-Reg(ǫ) 9.5202* 6.8103* 2.9048* 82.491*obj-NCP 11.5904* 9.7288* 2.9332* 88.406*reflux-PF(ρ) 1.619 4.969 1.485 32.248reflux-Reg(ǫ) 1.811 4.969 1.431 33.491reflux-NCP 1.683 2.881 1.359 38.406Nt-PF(ρ) 6.524 5.528 8.844 45Nt-Reg(ǫ) 6.645 5.528 9.794 49Nt-NCP 9.437 9.302 9.721 50Table 3: Distillation Optimization: Comparision of MPCC Formulations. Allcomputations were performed on a Pentium 4 with 1.8 GHz and 992 MB RAM.*CONOPT terminated at feasible point with negligible improvement of theobjective function, but without satisfying KKT tolerance.The reflux ratio is allowed to vary between 20 and 100, the feedtray locationvaries between 1 and 10 and the total tray number varies between 31 and 63.The objective function for the argon problem minimizes: objective = r + Nt.The argon column optimizations were initialized with 50 trays, a feedtray locationat the first tray and a reflux ratio at 32.8. Temperature and mole fractionprofiles were initialized with linear interpolations based on the top and bottomproduct properties. The resulting GAMS models consist of 1260 equations and1267 variables for the Reg(ǫ) and NCP formulations, and 1136 equations and1269 variables for the PF(ρ) formulation.As seen in the results in Table 3, the optimal solution has an objective functionvalue of 77.248. This was found relatively quickly by the PF(ρ) formulation.On the other hand, the Reg(ǫ) formulation terminates close to this solution butrequires an order of magnitude more effort. Again, the NCP formulation terminatesearly with poor progress.Along with the numerical study in Section 4, these four cases demonstratethat optimization of detailed distillation column models can be performed efficientlywith MPCC formulations. In particular, it can be seen that the penaltyformulation (PF(ρ)) represents a significant improvement over the NCP formulationboth in terms of iterations and CPU seconds; PF(ρ) offers advantagesover the Reg(ǫ) formulation as well.25

7 ConclusionsWith the development and widespread use of large-scale nonlinear programmingtools for process optimization, there has been an associated applicationof complementarity formulations to represent discrete decisions. This study investigatesboth the formulation and solution strategies for the MathematicalPrograms with Compementarity Constraints (MPCCs). Because MPCCs havedependent constaints at all feasible points along with unbounded multipliers,special care is needed in the formulation of associated optimization problemsalong with a reliable solution algorithm. In this study, we investigate and summarizea number of regularization and penalty MPCC formulations and findthat the penalty formulation is particularly advantageous, especially when coupledwith an active set NLP solver. This conclusion is observed in a numericalcomparison with a library of MPCC test problems.Because all MPCCs can be formulated as underlying MPEC problems thatresult from nested optimization formulations, a necessary condition for wellbehavedMPCCs is that the inner optimization problem be convex with solutionsthat are not restricted by upper level constraints. This condition is used to motivatea systematic strategy for the formulation of complementarity constraints.This strategy is applied to a number of widely-used examples in process engineering,including flow reversals, relief valves, compressor kickbacks, controllersaturation and disappearance of phases in equilibrium stages.This combination of well-posed formulations and MPCC solutions strategiesis demonstrated on two large-scale case studies with as many as 8000 discretedecisions. First, in the discretized dynamic system with switches, both thepenalty and relaxed formulations are very effective with active set and barriersolvers, respectively. Second, in dealing with disappearing phases in distillationoptimization models, the penalty formulation shows significant performance improvementsover the NCP formulation in a previous study. From these results,we find that well-posed complementarities coupled with NLP problems basedon penalty formulations are efficient and effective strategies for the solution ofMPCCs in process engineering.AcknowledgementsFunding from the National Science Foundation under grant #CTS-0438279 isgratefully appreciated. The authors also wish to thank Dr. Steven Dirkse forfruitful discussions and help with the NLPEC package.References[1] Anitescu, M., P. Tseng, and S.J. Wright, “Elastic-mode algorithms formathematical programs with equilibrium constraints: global convergenceand stationarity properties,” Math. Program., 110, pp. 337-371 (2007).26

[2] Burke, J.V. and S-P. Han, A Robust Sequential Quadratic ProgrammingMethod, Mathematical Programing, 43, pp. 277-303, (1989).[3] Chen, X. and M. Fukushima, “A Smoothing Method for a MathematicalProgram with P-Matrix Linear Complementarity Constraints,” ComputationOptimization and Applications, 27, pp. 223-246 (2004).[4] M. C. Ferris and J. S. Pang, (eds.) Complementarity and Variational Problems:State of the Art, Philadelphia, Pennsylvania, 1997. SIAM Publications.[5] M. C. Ferris, “Complementarity Problems and Applications,” presented atSIAM Conference on Optimization, Stockholm (2005)[6] Fletcher, R. and S. Leyffer, “Numerical experience with solving MPECs asNLPs”, Numerical Analysis Report NA/210, Department of Mathematics,University of Dundee (2002).[7] Gallun, S.E., R.H. Luecke, D.E. Scott, A.M. Morshedi, “Use open equationsfor better models”, Hydrocarbon Processing, pp. 78-90 (1992).[8] GAMS: The Solver Manuals, GAMS Development Corporation, Washington,DC (2004)[9] Getting Started with ROMeo, SimSci-Esscor, 2004.[10] Gopal, V. and L.T. Biegler. “Smoothing methods for ComplementarityProblems in Process Engineering” J. AIChE, 45, 7, pp. 1535-1547, (1999).[11] Hu, X. M. and D. Ralph, “Convergence of a Penalty Method of MathematicalProgramming with Complementarity Constraints,” Journal of OptimizationTheory and Applications, 123, 2, pp. 365-390 (2004).[12] Heemels, M. PhD Thesis, Technische Universiteit Eindoven (1999).[13] Kameswaran, S., G. Staus and L. T. Biegler, “Parameter Estimation ofCore Flood and Reservoir Models,” Computers and Chemical Engineering,29, 8, pp. 1787-1800 (2005).[14] Lang, Y-D and L.T. Biegler, “Distributed Stream Method for Tray Optimization”,AIChE Journal, 48, 3, pp.582-595 (2002).[15] Leyffer, S., G. Lopez-Calva and J. Nocedal, “Interior Methods for MathematicalPrograms with Complementarity Constraints,” SIAM Journal ofOptimization, 17, 1, pp. 52-77 (2006).[16] Lopez-Calva, G. “Regularization via Exact Penalty Methods” 4th InternationalConference on Complementarity Problems, August 10th, 2005.[17] Luo, Z.Q., J.S. Pang, and D. Ralph, Mathematical Programs with EquilibriumConstraints, Cambridge University Press, Cambridge, 1996.27

[18] Mittelmann, H.D. and A. Pruessner, “A Server for Automated PerformanceAnalysis of Benchmarking Data”, Optimization Methods and Software, 21,1, pp. 105-120 (2006).[19] MPEC Library, http://www.gamsworld.eu/mpec/mpeclib.htm[20] Raghunathan, A. and L. T. Biegler, “MPEC Formulations and Algorithmsin Process Engineering,” Computers and Chemical Engineering, 27, pp.1381-1392 (2003)[21] Raghunathan, A., PhD Thesis, Department of Chemical Engineering,Carnegie Mellon University (2004)[22] Raghunathan, A.U. and L.T. Biegler, “Interior point methods for MathematicalPrograms with Complementarity Constraints (MPCCs),” SIAM J.Optimization, 15, 3, pp. 720-750 (2005)[23] Raghunathan, A., M.S. Diaz, L.T. Biegler “An MPEC Formulation for DynamicOptimization of Distillation Operation,” Computers and ChemicalEngineering, 28/10 pp. 2037-2052 (2004)[24] Ralph, D. and S. J. Wright, “Some Properties of Regularization and PenalizationSchemes for MPECs,” Optimization Methods and Software, 19,5, pp. 527-556 (2004)[25] Poku, M., L. Biegler, and D. Ternet, Real-Time Optimization (RTO) OnlineTool - ROMeo & IPOPT, PREPRINT, 2005.[26] Scheel, H. and S. Scholtes, “Mathematical Programs with ComplementarityConstraints: Stationarity, Optimality, and Sensitivity,” Mathematics ofOperations Research, 25, 1, (2000)[27] Scholtes, S. and M. Stöhr, “How Stringent is the Linear Independence Assumptionfor Mathematical Programs with Complementarity Constraints?”Mathematics of Operations Research, 26, 4, pp. 851-863 (2001).[28] Stein, O., J. Oldenburg, W. Marquardt, “Continuous Reformulations ofDiscrete-Continuous Optimization Problems,” Computers & Chemical Engineering,Vol. 28, No. 10, 2004, 1951 - 1966[29] Stewart, D. and M. Anitescu “Optimal Control of Systems with DiscontinuousDifferential Equations,” Preprint number: ANL/MCS-P1258-0605,Mathematics and Computer Science Division, Argonne National Laboratory,June 2005.[30] Young, J.C.C., R. Baker, and C.L.E. Swartz, “Input saturation effects inoptimizing control: inclusion within a simultaneous optimization framework,”Computers and Chemical Engineering, 28, pp. 1347-1360 (2004)28