On the approximation of real rational functions via mixed-integer ...

Applied Mathematics and Computation 112 (2000) 113±124www.elsevier.nl/locate/amcOn the approximation of real rationalfunctions via mixed-integerlinear programmingNikos Papamarkos *Department of Electrical and Computer Engineering, Electric Circuits Analysis Laboratory,Democritus University of Thrace, 67100 Xanthi, GreeceAbstractThis paper is introducing a new method for the approximation of real rationalfunctions via mixed-integer linear programming. The formulation of the linear approximationproblem is based on the minimization of a suitable minimax criterion, incombination with a branch and bound linear integer technique. The proposed algorithmcan be used in many rational approximation problems, where some coecients of therational function are required to take only integer values. The formulation of theproblem ensures always the global solution. The proposed algorithm was extensivelytested on a variety of problems. An analytical example is presented to illustrate the useand e€ectiveness of the algorithm. Ó 2000 Elsevier Science Inc. All rights reserved.Keywords: Integer programming; Linear programming; Optimization; Rational approximation;Branch and bound tree search1. IntroductionThis paper is concerned with the problem of approximation of real rationalfunctions via integer linear programming. Many algorithms have been proposedfor the continuous linear programming case, where there is no conditionfor the coecients to obtain integer values. The most important of them are the* Tel.: +30 541 79566; fax: +30 541 26947; e-mail: papamark@voreas.ee.duth.gr0096-3003/00/$ - see front matter Ó 2000 Elsevier Science Inc. All rights reserved.PII: S0096-3003(99)00052-1

orG…x k † A…x k†B…x k † 6 d 8x k 2 S and B…x k † > 0; …3†where d, a suciently small positive value.To formulate the approximation problem, at each sampling point x k 2 S, thevariables n 0 kare de®ned through the following relationjG…x k †B…x k † A…x k †j ˆ n 0 k : …4†Since we accept that B…x k † > 0 for every x k 2 S, the relation (4) may also bewritten in the formG…x k † A…x k†B…x k † ˆn0 kB…x k † :…5†From Eq. (5), it is clear that for a satisfactory approximation the quantitiesd 0 k ˆ B…x k†n 0 …6†kmust achieve large positive values. If, therefore,n 0 ˆ maximumfn 0 k g…7†x k 2Sand( )1d ˆ maximum n 0…8†x k 2SB kthen, the approximation problem may be formulated as follows:maximize dsubject to the constraintsjG…x k †B…x k † A…x k †j 6 n 0 ;n 0B…x k † 6 1 dfor k ˆ 1; 2; ...; K;where n 0 P 0 and d > 0.From the above formulation of the approximation problem, we can concludethat· For every x k 2 S the following relations always hold:G…x k † A…x k†B…x k † 6N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124 115n0B…x k † 6 1 d :…9†…10†· The second set of constraints guarantees the positiveness of B…x k † for everyx k 2 S.

116 N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124· The approximation problem (9) does not have a linear form.By using (1), the approximation problem (9) is rewritten in the followingform:maximize dsubject to! G…x k † 1 ‡ XMbj~B j …x k † XNa j~A j …x k †!6 n 0 ;…11†jˆ1jˆ1! 1 ‡ XMb j~B j …x k † ‡ dn 0 i 6 0 for k ˆ 1; 2; ...; K;jˆ1where n 0 P 0 and d > 0.The approximation problem, as formulated in (11), is not linear, but it maybe converted to a linear one via the transformations:n ˆ 1n 0 ;b 0 j ˆ bjn 0 ; for j ˆ 1; 2; ...; M;…12†a 0 j ˆ ajn 0 ; for j ˆ 1; 2; ...; N:Now, using (12), the approximation problem is reformulated asmaximize dsubject to!n ‡ XMb 0 ~ jB j …x k † XNa 0 ~ jA j …x k † 6 1;G k G kjˆ1n ‡ XMn ‡ XMjˆ1jˆ1b 0 j ~ B j …x k †!jˆ1‡ XNjˆ1a 0 j ~ A j …x k † 6 1;!b 0 ~ jB j …x k † ‡ d 6 0 for k ˆ 1; 2; ...; K;…13†where n 0 P 0 and d > 0.The above problem (13) shows that the approximation problem (11) can beeasily linearized by using relations (12). If some or all of the function coecientsmust take only integer values then an integer approximation problemmust be formulated.

N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124 117Fig. 1. A typical structure of a binary tree.2.1. Formulation of the mixed integer linear programming algorithmThe Mixed Integer Linear Programming Algorithm (MILPA) used is basedon a tree search technique and speci®cally on the branch and bound binary treealgorithm. Fig. 1 shows the structure of a binary tree, where from each nodetwo branches are departed. Each branch of the tree ends into a ``child'' node.The initial node is the root node and is the practical continuous solution of theproblem. At each node of the tree, a linear programming problem is solved inwhich all the variables are considered to be continuous and where additionallinear constraints are embedded in the constraints of the original (initial)problem. These constraints correspond to the speci®c path of the current node.The entire tree technique is based on a systematic search of the tree nodes andcan be followed by di€erent tasks. In this implementation, the so-called preordertechnique is used, which for the tree of Fig. 1 corresponds to the (AB-CDEFGHIJKLMNO) path. The pre-order technique speeds up the entireprocess by cutting branches of the tree. In addition, if a good estimation of the®nal integer solution is known from the beginning, then the pre-order techniqueachieves the optimal solution faster.The tree search technique is implemented by using a three-dimensionalmatrix. This matrix de®nes for each node the additional linear constraintsthat must be added to the initial problem. The ®rst column of this matrixgives the consecutive number of the variables. The second column contains

118 N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124the constant values of the additional constraints. Finally, the third columncontains only the values 1 or 1, which de®ne if the additional constraintsare of type 6 or P , respectively. To de®ne the end of the path, all theelements of the last row of the matrix take the value 5. For example thefollowing matrix231 5 12 8 167…14†4 3 14 1 55 5 5de®nes that we are in node E and that the constraintsx 1 6 5x 2 P 8x 3 6 14…15†must be added to the original linear problem.In each node, every integer variable produces two linear programmingproblems. More speci®cally, if in a node ~x j is the value of the variable x j thenthe solution set Q of the integer problem is divided into two sets Q 1 and Q 2such aswithQ ˆ Q 1 [ Q 2 and Q 1 \ Q 2 ˆ 0 …16†Q 1 ˆ Q \fx j : x j 6 ‰~x j Šg;Q 2 ˆ Q \fx j : x j P ‰~x j Š‡1g;where [x] denotes the integer part of x. Obviously, every solution of Q belongsto Q 1 or to Q 2 .As it is analyzed, each node decomposes the problem to two new linearprogramming problems, which correspond to the two ``child'' nodes. This topdownprocess is ®nished when it is found a solution, which is(a) an integer solution,(b) a non-feasible solution,(c) a non-limited solution,(d) a non-integer solution, which is not better from a previous solution.The integer algorithm described above cannot be applied directly to theproblem (13), because the variables of this problem are not the coecients ofthe rational function but the coecients divided by the auxiliary variable n 0 .Inother words, we want integer values for the variables a j and b j and not(or > for instead of) for the variables a 0 j ˆ a j=n 0 and b 0 j ˆ b j=n 0 . However, if the

N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124 119tree search algorithm refers to the variables a j and b j , then, at each tree node,additional constraints of the form:a j 6 p;a j P p;b j 6 p;…17†b j P p:can be transformed into the following form:qn ‡ a 0 j 6 0;qn a 0 j 6 0;rn ‡ b 0 j 6 0;…18†rn b 0 j 6 0:This form is suitable for the linear programming problem (13) and thereforecan be added to it. Doing this, at each tree node a linear programming problemis solved in the following form:maximize dsubject to!n ‡ XMb 0 ~ jB j …x k † XNa 0 ~ jA j …x k † 6 1;G kjˆ1!jˆ1…19†0; G k n ‡ XMb 0 ~ jB j …x k † ‡ XNa 0 ~ jA j …x k † 6 1; r m1 n ‡ b 0 j 6 0; r m 2n b 0 j 6 j 2‰1 ...MŠ;jˆ1jˆ1! n ‡ XMb 0 ~ jB j …x k † ‡ d 6 0;jˆ1 q n1n ‡ a 0 j 6 0; q n 2n a 0 j6 0; j 2‰1 ...NŠ;where q n1; q n2and r m1 ; r m2 integers, n 0 P 0, d > 0 and k ˆ 1; 2; ...; K.It is noted that in the linear problem (19), the variables a 0 j and b0 jare unrestrictedin sign. On the other hand, linear programming algorithms, such asthe RSA, require only non-negative variables. A simple way to overcome thisdiculty is to shift the variables by a suitable shift constant V > 0. Therefore,if we de®ne

120 N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124p j ˆ V ‡ a 0 jfor j ˆ 1; 2; ...; N;q j ˆ V ‡ b 0 jfor j ˆ 1; 2; ...; M;…20†then, the linear problem (19) is reformulated asmaximize dsubject toG kn ‡ XMjˆ1q j~B j …x k †jˆ1! XN! G k n ‡ XMq j~B j …x k †jˆ1jˆ1‡ XNp j~ Aj …x k †‡d k y 6 1;jˆ1p j~A j …x k †d k y 6 1;! n ‡ XMq j~B j …x k † ‡ d ‡ h k y 6 0 y ˆ 1; q n1n ‡ p j 6 V ; q n2n ‡ p j P V ; j 2‰1 ...NŠ; r m1 n ‡ q j 6 V ; r m2 n ‡ q j P V ; j 2‰1 ...MŠ;…21†where q n1; q n2and r m1 ; r m2 known integers, n 0 P 0, d > 0 and k ˆ 1; 2; ...; K.All the variables are positive, andh k ˆ V XMjˆ1~B j …x k †; …22†d k ˆG k V XM~B j …x k †‡V XNjˆ1jˆ1~A j …x k †ˆG k h k ‡ v XNjˆ1~A j …x k †: …23†Problem (21) is the linear programming problem that must be solved in eachnode. It corresponds to a total number of M ‡ N ‡ 3 variables and 3K ‡ D n ‡ 1linear constraints, where D n is the number of the additional linear constraints inthe node n. Usually, M ‡ N ‡ 3 3K ‡ D n ‡ 1, which means that the linearprogramming problem must be solved by its dual, using a method such as theRSA.According to the above analysis, the mixed integer linear rational approximationalgorithm consists of the following steps:Step 1. The integer and continuous coecients are de®ned and the LRAapproximation problem (13) is formulated. If it is known from the beginning,we can give an estimation of the global-integer solution.Step 2. The LRA problem is solved as a continuous variable problem. Let Xthe vector of variables that must take integer values in the ®nal mixed-integersolution.

N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124 121Step 3. If all the elements of X are integers then the optimal integer solutionis found.Step 4. A new linear programming sub-problem of the form (21) is formulatedand solved by the RSA.Step 5. If the sub-problem does not have any solution or its solution is worsethan a previous solution, then go to Step 6. If the sub-problem has an integersolution better than a previous integer solution then put X in ˆ X and go to Step6. In any other case, go to Step 7.Step 6. Move to the next node of the tree and go to Step 7. If all the nodes ofthe tree are examined then go to Step 7.Step 7. De®ne the additional linear constraints and go to Step 4.Step 8. The optimal solution is found.3. ExampleThe proposed method was successfully tested on a variety of approximationproblems. In this paper we use the MIRFA algorithm for the approximation ofthe rational function with n ˆ m ˆ 3 coecients. The sampling data are producedfrom the rational function4 ‡ 0:2x ‡ 3x 2f …x† ˆ…24†1 ‡ 2x ‡ 0:8x 2 ‡ 4x 3in the rage [0,2]. Speci®cally, x k ; k ˆ 1; ...; K, with K ˆ 30, equal spaced valuesare used for the independent variable x, which from (24) produce 30 samplingdata for the function f(x). The values of f …x k †; k ˆ 1; ...; K are truncated tothe ®fth decimal point.The A ~ i …x† and ~B i …x† functions have the following form~A i …x† ˆx i1 ; ~B i …x† ˆx i : …25†The application of the RFA algorithm, after 22 loops, gives the followingcontinuous optimal solutiond ˆ 336215:824n ˆ 2:97428 10 6a 1 ˆ 4:0 b 1 ˆ 1:99778a 2 ˆ 0:200093 b 2 ˆ 0:799946a 3 ˆ 3:00022 b 3 ˆ 4:00034Obviously, the values of the optimal coecients are not exactly the initial of(24) due to the truncating procedure.The application of the MIRFA algorithm, with V ˆ 100, gives the followingoptimal integer solution:

122 N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124Fig. 2. Approximation results.Fig. 3. Final approximation error.

N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124 123d ˆ 5:545848 n ˆ 0:18031509a 1 ˆ 4 b 1 ˆ 3a 2 ˆ 6 b 2 ˆ 5a 3 ˆ 2 b 3 ˆ 4Fig. 2 depicts the comparison between f(x) and the solutions of RFA andMIRFA. Fig. 3 shows the ®nal approximation error in each sampling point.4. ConclusionsThis paper has introduced a method for approximation of real rationalfunctions via mixed integer linear programming. The proposed algorithm extendsthe RFA algorithm to the case where some of the function coecientstake integer values. The conversion of the RFA linear programming problemto a mixed integer one, is not a simple task. Our approach is based on thebranch and bound tree search technique and on a suitable transformation ofthe additional constraints produced on tree nodes. Thus, on each tree node alinear programming problem is formulated and solved by using the dual formand the RSA.The formulation of the problem and the use of linear programming guaranteesthat the approximation problem has always a solution and that thissolution is the global optimal solution.References[1] H. Press, B.P. Flannery, S.A. Teukolsky, W.T. Vetterling, Numerical Recipes: The ArtScienti®c Computing, Cambridge University Press, Cambridge, 1986.[2] I. Barrodale, M.J. Powell, F.D.K. Roberts, The di€erential correction algorithm for rationall 1 approximation, SIAM Journal of Numerical Analysis 9 (1972) 493±504.[3] N. Papamarkos, G. Vachtsevanos, B. Mertzios, On the optimum approximation of realrational functions via linear programming, Applied Mathematics and Computation 26 (1988)267±287.[4] N. Papamarkos, A program for the optimum approximation of real rational functions vialinear programming, Advances in Engineering Software 11 (1) (1989) 37±48.[5] N. Papamarkos, G. Vachtsevanos, A fast linear programming approach to the design of 1-DIIR digital ®lters, International Journal of Circuits Theory Applications 18 (1990) 325±333.[6] G. Vachtsevanos, N. Paramarkos, B.G. Mertzios, On the approximation of the magnituderesponse of 2-D IIR digital ®lters using linear programming, Circuits, Systems and SignalProcessing 6 (1987) 45±60.[7] N. Papamarkos, I. Spiliotis, A. Zoumadakis, Character recognition by signature approximation,International Journal of Pattern Recognition and Arti®cial Intelligence 8 (5) (1994) 1171±1187.

124 N. Papamarkos / Appl. Math. Comput. 112 (2000) 113±124[8] N. Papamarkos, B. Gatos, A new approach for multithreshold selection, Computer Vision,Graphics, and Image Processing-Graphical Models and Image Processing 56 (5) (1994) 357±370.[9] N. Papamarkos, Multirational function approximation via linear programming, AppliedMathematics and Computation, Applied Mathematics and Computation 75 (1996) 75±89.[10] S. Walukiewicz, Integer Programming (Mathematics and Its Applications), Kluwer AcademicPublishers, Dordrecht, 1990.[11] S.I. Gass, Linear Programming: Methods and Applications, 4th ed., McGraw-Hill, New York,1975.[12] D.J. Pannell, Introduction to Practical Linear Programming, Wiley, New York, 1996.