24.01.2014 Views

Vilfredo Pareto and Multi-objective Optimization - Department of ...

Vilfredo Pareto and Multi-objective Optimization - Department of ...

Vilfredo Pareto and Multi-objective Optimization - Department of ...

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

Documenta Math. 447<br />

<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong><br />

Matthias Ehrgott<br />

2010 Mathematics Subject Classification: 90C29<br />

Keywords<strong>and</strong>Phrases: <strong>Multi</strong>-<strong>objective</strong>optimization, <strong>Pareto</strong>optimality<br />

A multi-<strong>objective</strong> optimization problem consists in the simultaneous optimization<br />

<strong>of</strong> p <strong>objective</strong> functions f 1 ,...,f p subject to some constraints, which I will<br />

just write as x ∈ X, where X is a subset <strong>of</strong> R n . It is usually assumed that there<br />

does not exist any x ∈ X such that all functions f k attain their minimima at x.<br />

Hence, due to the absence <strong>of</strong> a total order on R p , it is necessary to define the<br />

minimization with respect to partial orders. So let Y := {f(x) : x ∈ X} be the<br />

set <strong>of</strong> outcome vectors. To compare elements <strong>of</strong> Y, I will follow the definition<br />

<strong>of</strong> Koopmans (1951). Let y 1 ,y 2 ∈ Y. Then y 1 ≦ y 2 if <strong>and</strong> only if yk 1 ≦ y2 k for<br />

all k = 1,...p; y 1 ≤ y 2 if <strong>and</strong> only if y 1 ≦ y 2 , but y 1 ≠ y 2 <strong>and</strong> y 1 < y 2 if <strong>and</strong><br />

only if yk 1 < y2 k<br />

for all k = 1,...p.<br />

It is here that <strong>Pareto</strong> makes his appearance. In countless books <strong>and</strong> articles<br />

on multi-<strong>objective</strong> optimization, one can find a definition like this:<br />

Definition 1. Let X ⊂ R n be a non-empty set <strong>of</strong> feasible solutions <strong>and</strong> f =<br />

(f 1 ,...f p ) : R n → R p be a function. Feasible solution ˆx ∈ X is called a <strong>Pareto</strong><br />

optimal solution <strong>of</strong> the multi-<strong>objective</strong> optimization problem<br />

min{f(x) : x ∈ X} (1)<br />

if <strong>and</strong> only if there does not exist any x ∈ X such that f(x) ≤ f(ˆx).<br />

Sometimes <strong>Pareto</strong> optimality is defined with respect to outcome vectors.<br />

Definition 2. Let Y ∈ R p be a non-empty set <strong>of</strong> outcome vectors. Outcome<br />

vector ŷ ∈ Y is called <strong>Pareto</strong> optimal if <strong>and</strong> only if there does not exist any<br />

y ∈ Y such that y ≤ ŷ.<br />

Where does the name <strong>Pareto</strong> optimal come from? <strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> Francis<br />

Ysidro Edgeworth are <strong>of</strong>ten called as the fathers <strong>of</strong> multi-<strong>objective</strong> optimization.<br />

Sentences like the “introduction <strong>of</strong> the <strong>Pareto</strong> optimal solution in<br />

1896” (Chen et al., 2005, p. VII); “The concept <strong>of</strong> noninferior solution was introduced<br />

at the turn <strong>of</strong> the century [1896] by <strong>Pareto</strong>, a prominent economist”<br />

(Chankong <strong>and</strong> Haimes, 1983, p. 113); “Edgeworth <strong>and</strong> <strong>Pareto</strong> were probably<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


448 Matthias Ehrgott<br />

the first who introduced an optimality concept for such problems” (Jahn, 2004,<br />

p. 113); “wurden besonders von F.Y. Edgeworth (1845–1926) <strong>and</strong> V. <strong>Pareto</strong><br />

(1848–1929 [sic!]) hinreichende Bedingungen für <strong>Pareto</strong>maximalität bzw. Gleichgewichtsbedingungen<br />

angegeben.” (Göpfert <strong>and</strong> Nehse, 1990, p. 9) or “The<br />

foundations are connected with the names <strong>of</strong> <strong>Vilfredo</strong> <strong>Pareto</strong> (1848–1923) <strong>and</strong><br />

Francis Ysidro Edgeworth (1845–1926)” (Löhne, 2011, p. 1) abound in textbooks.<br />

TheInternationalSocietyon<strong>Multi</strong>pleCriteriaDecisionMakingbestows<br />

the Edgeworth–<strong>Pareto</strong> award “upon a researcher who, over his/her career, has<br />

established a record <strong>of</strong> creativity to the extent that the field <strong>of</strong> MCDM would<br />

notexistinitscurrentformwithoutthefar-reachingcontributionsfromthisdistinguished<br />

scholar”, see http://www.mcdmsociety.org/intro.html#Awards.<br />

Edgeworth was an influential Pr<strong>of</strong>essor <strong>of</strong> Economics at King’s College London<br />

<strong>and</strong> from 1891 Pr<strong>of</strong>essor <strong>of</strong> Political Economy at Oxford University. In his<br />

best known book Mathematical Psychics (Edgeworth, 1881) he applied formal<br />

mathematics to decision making in economics. He developed utility theory,<br />

introducing the concept <strong>of</strong> indifference curve <strong>and</strong> is best known for the Edgeworth<br />

box. But because multi-<strong>objective</strong> optimization is concerned with <strong>Pareto</strong><br />

optimality rather than Edgeworth optimality, this story focuses on his contemporary.<br />

Fritz Wilfried <strong>Pareto</strong><br />

According to Yu (1985, p. 49) <strong>Pareto</strong> “was a famous Italian engineer” but he<br />

is certainly much better known as an economist. The following information<br />

is taken from Stadler (1979) <strong>and</strong> the wikipedia entry (http://en.wikipedia.<br />

org/wiki/<strong>Vilfredo</strong>_<strong>Pareto</strong>) on <strong>Pareto</strong>.<br />

<strong>Vilfredo</strong> Federico Damaso <strong>Pareto</strong> was born on 15 July 1848 in Paris as Fritz<br />

Wilfried <strong>Pareto</strong>, son <strong>of</strong> a French woman <strong>and</strong> an Italian civil engineer, who was<br />

a supporter <strong>of</strong> the German revolution <strong>of</strong> 1848. His name was changed to the<br />

Italian version when his family moved back to Italy in 1855 (or 1858). In 1870<br />

he graduated from Polytechnic Institute <strong>of</strong> Turin with a dissertation entitled<br />

“The Fundamental Principles <strong>of</strong> Equilibrium in Solid Bodies”. He then worked<br />

as an engineer <strong>and</strong> manager for an Italian railway company. He was very<br />

politically active, an ardent supporter <strong>of</strong> free market economy. He obtained a<br />

lecturer position in economics <strong>and</strong> management at the University <strong>of</strong> Florence<br />

in 1886 (according to wikipedia). Eventually he resigned from his positions in<br />

1889. During the 1880s he became acquainted with leading economists <strong>of</strong> the<br />

time <strong>and</strong> he published many articles by 1893 (not all academic, though). In<br />

1893 he moved to Lausanne where he lectured at the University <strong>of</strong> Lausanne<br />

<strong>and</strong> became the successor <strong>of</strong> Léon Walras as Pr<strong>of</strong>essor <strong>of</strong> Political Economy. In<br />

his later years he mainly worked in Sociology. <strong>Vilfredo</strong> <strong>Pareto</strong> died at Célégny,<br />

Switzerl<strong>and</strong>, on 19 August 1923. The University <strong>of</strong> Lausanne still has a Centre<br />

d’études interdisciplinaires Walras <strong>Pareto</strong> (http://www.unil.ch/cwp). Apart<br />

from <strong>Pareto</strong> optimality, <strong>Pareto</strong>’s name is attached to the <strong>Pareto</strong> principle (or<br />

80–20 rule), observing in 1906 that 80% <strong>of</strong> the property in Italy was owned by<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong> 449<br />

Figure 1: <strong>Vilfredo</strong> <strong>Pareto</strong> 1848–1923 (Picture scanned from the second French<br />

edition <strong>of</strong> <strong>Pareto</strong> (1906) published in 1927.)<br />

20% <strong>of</strong> the population <strong>and</strong> the <strong>Pareto</strong> distribution, a power law probability<br />

distribution.<br />

<strong>Pareto</strong> Optimality<br />

The origin <strong>of</strong> the term <strong>Pareto</strong> optimality goes back to the following text from<br />

<strong>Pareto</strong> (1906, Chapter VI, Section 33).<br />

Principeremo col definire un termine di cui è comodo fare uso per<br />

scansare lungaggini. Diremo che i componenti di una collettività<br />

godono, in una certa posizione, del massimo di <strong>of</strong>elimità, qu<strong>and</strong>o<br />

è impossibile allontanarsi pochissimo da quella posizione giov<strong>and</strong>o,<br />

o nuocendo, a tutti i componenti la collettività; ogni piccotissimo<br />

spostamento da quella posizione avendo necessariamente per effetto<br />

digiovareapartedeicomponentitacollettivitàedinuocereadaltri.<br />

Or in the English translation (<strong>Pareto</strong>, 1971, p. 261):<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


450 Matthias Ehrgott<br />

Wewillbeginbydefiningatermwhichisdesirabletouseinorderto<br />

avoid prolixity. We will say that the members <strong>of</strong> a collectivity enjoy<br />

maximum ophelimity in a certain position when it is impossible to<br />

find a way <strong>of</strong> moving from that position very slightly in such a<br />

manner that the ophelimity enjoyed by each <strong>of</strong> the individuals <strong>of</strong><br />

that collectivity increases or decreases. That is to say, any small<br />

displacement in departing from that position necessarily has the<br />

effect <strong>of</strong> increasing the ophelimity which certain individuals enjoy,<br />

<strong>and</strong> decreasing that which others enjoy, <strong>of</strong> being agreeable to some<br />

<strong>and</strong> disagreeable to others.<br />

Of course, <strong>Pareto</strong> here refers to the distribution <strong>of</strong> utility (ophelimity) among<br />

individuals in an economy rather than solutions <strong>of</strong> an optimization problem.<br />

<strong>Multi</strong>-<strong>objective</strong> optimization or mathematical optimization in general as we<br />

know it today, did not exist during <strong>Pareto</strong>’s lifetime, it only developed in<br />

the 1940s. And it is some <strong>of</strong> the founding works <strong>of</strong> Operations Research <strong>and</strong><br />

optimization that need to be cited here. Nobel Laureate in Economics T.C.<br />

Koopmans (1951) formally studied production as a resource allocation problem<br />

<strong>and</strong> the combination <strong>of</strong> activities to represent the output <strong>of</strong> commodities as a<br />

function <strong>of</strong> various factors. In this work he introduced the following definition<br />

<strong>of</strong> efficient vector (p. 60). “A point y in the commodity space is called efficient<br />

if it is possible [i.e., if y ∈ (A)], <strong>and</strong> if there exists no possible point ȳ ∈ (A) such<br />

that ȳ −y ≥ 0.” Note that (A) is what I called Y in Definition 2, i.e., possible<br />

means that there is some x such that y = Ax. Koopmans does hence only talk<br />

about efficient vectors in terms <strong>of</strong> the outcome set. He proves necessary <strong>and</strong><br />

sufficient conditions for efficiency, but he does not refer to <strong>Pareto</strong>, nor does he<br />

talk about <strong>Pareto</strong> optimal solutions as in Definition 1 – instead he refers to “an<br />

activity vector x (that) shall lead to an efficient point y = Ax”.<br />

Another classic reference in optimization is the seminal paper by<br />

Kuhn <strong>and</strong> Tucker (1951). They refer to the “vector maximum <strong>of</strong> Koopmans’<br />

efficient point type for several concave functions g 1 (x),...,g p (x)”. This<br />

seems to be the earliest reference to the optimization <strong>of</strong> several functions in<br />

mathematics. Kuhn <strong>and</strong> Tucker cite Koopmans (1951) when they talk about<br />

vector maximum. They also apply the term efficient to the solutions <strong>of</strong> vector<br />

optimization problems (i.e., in decision space) <strong>and</strong> introduce the notion <strong>of</strong><br />

proper efficiency. But, again, no mention <strong>of</strong> <strong>Pareto</strong>. Kuhn <strong>and</strong> Tucker (1951)<br />

cite another Nobel Laureate in Economics who contributed to the foundations<br />

<strong>of</strong> multi-<strong>objective</strong> optimization, Kenneth J. Arrow.<br />

Arrow discusses <strong>Pareto</strong> extensively in his economical work <strong>and</strong> statements <strong>of</strong><br />

the impossibility theorem today usually refer to <strong>Pareto</strong> optimality as one <strong>of</strong> the<br />

axioms that cannot be jointly satisfied by a social choice function, but this term<br />

does not appear in Arrow’s original formulation (Arrow, 1951). Arrow’s important<br />

contribution to multi-<strong>objective</strong> optimization (Arrow et al., 1953) starts as<br />

follows “A point s <strong>of</strong> a closed convex subset S <strong>of</strong> k-space is admissible if there is<br />

no t ∈ S with t i ≤ s i for all i = 1,...,k, t ≠ s.” This is, <strong>of</strong> course, the same as<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong> 451<br />

Koopmans’ definition <strong>of</strong> efficient point (whose paper Arrow et al. (1953) cite),<br />

<strong>and</strong> again is relevant in the outcome set <strong>of</strong> a multi-<strong>objective</strong> problem rather<br />

than the set <strong>of</strong> feasible solutions – no trace <strong>of</strong> <strong>Pareto</strong> here, either.<br />

There are a number <strong>of</strong> other definitions <strong>of</strong> <strong>Pareto</strong> optimal, efficient, or admissible<br />

points. Zadeh (1963) defines “A system S 0 ∈ C is noninferior in C if the<br />

intersection <strong>of</strong> C <strong>and</strong> Σ > (S 0 ) is empty.” Σ > (S 0 ) is the set <strong>of</strong> all systems which<br />

are better than S 0 with respect to a partial order ≥. Chankong <strong>and</strong> Haimes<br />

(1983) later use the same definition. While Zadeh cites Koopmans <strong>and</strong> Kuhn<br />

<strong>and</strong> Tucker, <strong>Pareto</strong> remains notably absent. The final term that is common<br />

today is that <strong>of</strong> a nondominated point.<br />

<strong>Multi</strong><strong>objective</strong> <strong>Optimization</strong> <strong>and</strong> Economics<br />

When did the term <strong>Pareto</strong> optimal first appear in the literature? As we have<br />

seen, it was not used in early mathematical works on multi-<strong>objective</strong> optimization.<br />

The answer is once again in economics. Little (1950, p. 87) in a discussion<br />

<strong>of</strong> the distribution <strong>of</strong> income (i.e., in the same context as <strong>Pareto</strong> himself) uses<br />

the term <strong>Pareto</strong> ‘optimum’ (with the quotation marks). The origin <strong>of</strong> the term<br />

is, therefore, clearly found in economics. It has then apparently mostly been<br />

used in economics, appearing in journals such as Public Choice <strong>and</strong> Journal<br />

<strong>of</strong> Economic Theory. As shown above, it was not used by the economists<br />

who are credited with having contributed to the origins <strong>of</strong> the mathematical<br />

theory <strong>of</strong> multi-<strong>objective</strong> optimization, but migrated to mathematics later on.<br />

The first journal articles that I could find are Basile <strong>and</strong> Vincent (1970) <strong>and</strong><br />

Vincent <strong>and</strong> Leitmann (1970). These articles also used the term undominated<br />

as an alternative. This then turned to nondominated in Yu <strong>and</strong> Leitmann<br />

(1974).<br />

Economics had a strong influence on the early history <strong>of</strong> multi-<strong>objective</strong> optimization,<br />

especially <strong>Pareto</strong>’s original definition <strong>of</strong> the term maximum ophelimity<br />

<strong>and</strong> the origin <strong>of</strong> the term <strong>Pareto</strong> optimum in Little (1950). The move<br />

into mathematics <strong>and</strong> optimization coincides with the mathematization <strong>of</strong> economics<br />

by scholars such as Koopmans <strong>and</strong> Arrow <strong>and</strong> finally the introduction<br />

<strong>of</strong> the topic into mathematical optimization by Kuhn <strong>and</strong> Tucker. It seems<br />

to have taken quite a while for <strong>Pareto</strong>’s name to appear in the mathematical<br />

optimization literature.<br />

The consequence <strong>of</strong> the history <strong>of</strong> <strong>Pareto</strong> optimality outlined above, is that<br />

at present there are quite a few terms (efficient, noninferior, nondominated,<br />

admissible, <strong>Pareto</strong> optimal) that express the same idea. Since multi-<strong>objective</strong><br />

optimization <strong>of</strong>ten distinguishes between decision vectors x ∈ X <strong>and</strong> outcome<br />

vectors y ∈ Y, one can find a large number <strong>of</strong> combinations <strong>of</strong> these terms<br />

in the literature used in parallel today, such as <strong>Pareto</strong> optimal decisions <strong>and</strong><br />

efficient outcomes.<br />

Itturnsoutthatthehistory<strong>of</strong>multi-<strong>objective</strong>optimization(vectoroptimization)<br />

is quite an interesting read, <strong>and</strong> I would like to refer interested readers<br />

to Stadler (1979) as a starting point. The history <strong>of</strong> multiple criteria deci-<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


452 Matthias Ehrgott<br />

sion making in general is the topic <strong>of</strong> the book Köksalan et al. (2011). These<br />

works also consider roots <strong>of</strong> multi-<strong>objective</strong> optimization in game theory <strong>and</strong><br />

the theory <strong>of</strong> ordered spaces <strong>and</strong> vector norms.<br />

References<br />

K. J. Arrow. Social Choice <strong>and</strong> Individual Values. Cowles Commission for<br />

Research in Economics Monograph No. 12. John Wiley & Sons, New York,<br />

1951.<br />

K. J. Arrow, E. W. Barankin, <strong>and</strong> D. Blackwell. Admissible points <strong>of</strong> convex<br />

sets. In H.W. Kuhn <strong>and</strong> A.W. Tucker, editors, Contributions to the Theory<br />

<strong>of</strong> Games, volume 2, pages 87–91. Princeton University Press, Princeton,<br />

1953.<br />

G. Basile <strong>and</strong> T. L. Vincent. Absolutely cooperative solution for a linear, multiplayer<br />

differential game. Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications,<br />

6:41–46, 1970.<br />

V. Chankong <strong>and</strong> Y. Y. Haimes. <strong>Multi</strong><strong>objective</strong> Decision Making – Theory <strong>and</strong><br />

Methodology. Elsevier Science, New York, 1983.<br />

G. Chen, X. Huang, <strong>and</strong> X. Yang. Vector <strong>Optimization</strong> – Set-Valued <strong>and</strong> Variational<br />

Analysis, volume 541 <strong>of</strong> Lecture Notes in Economics <strong>and</strong> Mathematical<br />

Systems. Springer Verlag, Berlin, 2005.<br />

F. Y. Edgeworth. Mathematical Psychics. C. Kegan Paul & Co., London, 1881.<br />

A. Göpfert <strong>and</strong> R. Nehse. Vektoroptimierung, volume 74 <strong>of</strong> Mathematisch-<br />

Naturwissenschaftliche Bibliothek. BSB B.G. Teubner Verlagsgesellschaft,<br />

Leipzig, 1990.<br />

J. Jahn. Vector <strong>Optimization</strong> – Theory, Applications, <strong>and</strong> Extensions. Springer<br />

Verlag, Berlin, 2004.<br />

M. Köksalan, J. Wallenius, <strong>and</strong> S. Zionts. <strong>Multi</strong>ple Criteria Decision Making<br />

– From Early History to the 21st Century. World Scientific Publishing,<br />

Singapore, 2011.<br />

T. C. Koopmans. Analysis <strong>of</strong> production as an efficient combination <strong>of</strong> activities.<br />

In T.C. Koopmans, editor, Activity Analysis <strong>of</strong> Production <strong>and</strong> Allocation,<br />

Cowles Commission for Research in Economics Monograph No. 13,<br />

pages 33–97. John Wiley & Sons, New York, 1951.<br />

H. W. Kuhn <strong>and</strong> A. W. Tucker. Nonlinear programming. In J. Neyman, editor,<br />

Proceedings <strong>of</strong> the Second Berkeley Symposium on Mathematical Statistics<br />

<strong>and</strong> Probability, pages 481–492. University <strong>of</strong> California Press, Berkeley,<br />

1951.<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong> 453<br />

I.M.D.Little. A Critique <strong>of</strong> Welfare Economics. TheClarendonPress, Oxford,<br />

1950.<br />

A. Löhne. Vector <strong>Optimization</strong> with Infimum <strong>and</strong> Supremum. Springer Verlag,<br />

Berlin, 2011.<br />

V. <strong>Pareto</strong>. Manuale di Economia Politica. Società Editrice Libraria, Milan,<br />

1906.<br />

V. <strong>Pareto</strong>. Manual <strong>of</strong> Political Economy. Augustus M. Kelley Publishers, New<br />

York, 1971.<br />

W. Stadler. A survey <strong>of</strong> multicriteria optimization or the vector maximum<br />

problem, Part I: 1776-1960. Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications,<br />

29:1–52, 1979.<br />

T. L. Vincent <strong>and</strong> G. Leitmann. Control-space properties <strong>of</strong> cooperative games.<br />

Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications, 6:91–113, 1970.<br />

P. L. Yu. <strong>Multi</strong>ple Criteria Decision Making: Concepts, Techniques <strong>and</strong> Extensions.<br />

Plenum Press, New York, 1985.<br />

P. L. Yu <strong>and</strong> G. Leitmann. Compromise solutions, domination structures, <strong>and</strong><br />

Salukvadze’s solution. Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications, 13:<br />

362–378, 1974.<br />

L. A. Zadeh. Optimality <strong>and</strong> non-scalar-valued performance criteria. IEEE<br />

Transactions on Automatic Control, 8:59–60, 1963.<br />

Matthias Ehrgott<br />

<strong>Department</strong><strong>of</strong>EngineeringScience<br />

The University <strong>of</strong> Auckl<strong>and</strong><br />

New Zeal<strong>and</strong><br />

m.ehrgott@auckl<strong>and</strong>.ac.nz<br />

Documenta Mathematica · Extra Volume ISMP (2012) 447–453


454<br />

Documenta Mathematica · Extra Volume ISMP (2012)

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!