Vilfredo Pareto and Multi-objective Optimization - Department of ...
Vilfredo Pareto and Multi-objective Optimization - Department of ...
Vilfredo Pareto and Multi-objective Optimization - Department of ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Documenta Math. 447<br />
<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong><br />
Matthias Ehrgott<br />
2010 Mathematics Subject Classification: 90C29<br />
Keywords<strong>and</strong>Phrases: <strong>Multi</strong>-<strong>objective</strong>optimization, <strong>Pareto</strong>optimality<br />
A multi-<strong>objective</strong> optimization problem consists in the simultaneous optimization<br />
<strong>of</strong> p <strong>objective</strong> functions f 1 ,...,f p subject to some constraints, which I will<br />
just write as x ∈ X, where X is a subset <strong>of</strong> R n . It is usually assumed that there<br />
does not exist any x ∈ X such that all functions f k attain their minimima at x.<br />
Hence, due to the absence <strong>of</strong> a total order on R p , it is necessary to define the<br />
minimization with respect to partial orders. So let Y := {f(x) : x ∈ X} be the<br />
set <strong>of</strong> outcome vectors. To compare elements <strong>of</strong> Y, I will follow the definition<br />
<strong>of</strong> Koopmans (1951). Let y 1 ,y 2 ∈ Y. Then y 1 ≦ y 2 if <strong>and</strong> only if yk 1 ≦ y2 k for<br />
all k = 1,...p; y 1 ≤ y 2 if <strong>and</strong> only if y 1 ≦ y 2 , but y 1 ≠ y 2 <strong>and</strong> y 1 < y 2 if <strong>and</strong><br />
only if yk 1 < y2 k<br />
for all k = 1,...p.<br />
It is here that <strong>Pareto</strong> makes his appearance. In countless books <strong>and</strong> articles<br />
on multi-<strong>objective</strong> optimization, one can find a definition like this:<br />
Definition 1. Let X ⊂ R n be a non-empty set <strong>of</strong> feasible solutions <strong>and</strong> f =<br />
(f 1 ,...f p ) : R n → R p be a function. Feasible solution ˆx ∈ X is called a <strong>Pareto</strong><br />
optimal solution <strong>of</strong> the multi-<strong>objective</strong> optimization problem<br />
min{f(x) : x ∈ X} (1)<br />
if <strong>and</strong> only if there does not exist any x ∈ X such that f(x) ≤ f(ˆx).<br />
Sometimes <strong>Pareto</strong> optimality is defined with respect to outcome vectors.<br />
Definition 2. Let Y ∈ R p be a non-empty set <strong>of</strong> outcome vectors. Outcome<br />
vector ŷ ∈ Y is called <strong>Pareto</strong> optimal if <strong>and</strong> only if there does not exist any<br />
y ∈ Y such that y ≤ ŷ.<br />
Where does the name <strong>Pareto</strong> optimal come from? <strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> Francis<br />
Ysidro Edgeworth are <strong>of</strong>ten called as the fathers <strong>of</strong> multi-<strong>objective</strong> optimization.<br />
Sentences like the “introduction <strong>of</strong> the <strong>Pareto</strong> optimal solution in<br />
1896” (Chen et al., 2005, p. VII); “The concept <strong>of</strong> noninferior solution was introduced<br />
at the turn <strong>of</strong> the century [1896] by <strong>Pareto</strong>, a prominent economist”<br />
(Chankong <strong>and</strong> Haimes, 1983, p. 113); “Edgeworth <strong>and</strong> <strong>Pareto</strong> were probably<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
448 Matthias Ehrgott<br />
the first who introduced an optimality concept for such problems” (Jahn, 2004,<br />
p. 113); “wurden besonders von F.Y. Edgeworth (1845–1926) <strong>and</strong> V. <strong>Pareto</strong><br />
(1848–1929 [sic!]) hinreichende Bedingungen für <strong>Pareto</strong>maximalität bzw. Gleichgewichtsbedingungen<br />
angegeben.” (Göpfert <strong>and</strong> Nehse, 1990, p. 9) or “The<br />
foundations are connected with the names <strong>of</strong> <strong>Vilfredo</strong> <strong>Pareto</strong> (1848–1923) <strong>and</strong><br />
Francis Ysidro Edgeworth (1845–1926)” (Löhne, 2011, p. 1) abound in textbooks.<br />
TheInternationalSocietyon<strong>Multi</strong>pleCriteriaDecisionMakingbestows<br />
the Edgeworth–<strong>Pareto</strong> award “upon a researcher who, over his/her career, has<br />
established a record <strong>of</strong> creativity to the extent that the field <strong>of</strong> MCDM would<br />
notexistinitscurrentformwithoutthefar-reachingcontributionsfromthisdistinguished<br />
scholar”, see http://www.mcdmsociety.org/intro.html#Awards.<br />
Edgeworth was an influential Pr<strong>of</strong>essor <strong>of</strong> Economics at King’s College London<br />
<strong>and</strong> from 1891 Pr<strong>of</strong>essor <strong>of</strong> Political Economy at Oxford University. In his<br />
best known book Mathematical Psychics (Edgeworth, 1881) he applied formal<br />
mathematics to decision making in economics. He developed utility theory,<br />
introducing the concept <strong>of</strong> indifference curve <strong>and</strong> is best known for the Edgeworth<br />
box. But because multi-<strong>objective</strong> optimization is concerned with <strong>Pareto</strong><br />
optimality rather than Edgeworth optimality, this story focuses on his contemporary.<br />
Fritz Wilfried <strong>Pareto</strong><br />
According to Yu (1985, p. 49) <strong>Pareto</strong> “was a famous Italian engineer” but he<br />
is certainly much better known as an economist. The following information<br />
is taken from Stadler (1979) <strong>and</strong> the wikipedia entry (http://en.wikipedia.<br />
org/wiki/<strong>Vilfredo</strong>_<strong>Pareto</strong>) on <strong>Pareto</strong>.<br />
<strong>Vilfredo</strong> Federico Damaso <strong>Pareto</strong> was born on 15 July 1848 in Paris as Fritz<br />
Wilfried <strong>Pareto</strong>, son <strong>of</strong> a French woman <strong>and</strong> an Italian civil engineer, who was<br />
a supporter <strong>of</strong> the German revolution <strong>of</strong> 1848. His name was changed to the<br />
Italian version when his family moved back to Italy in 1855 (or 1858). In 1870<br />
he graduated from Polytechnic Institute <strong>of</strong> Turin with a dissertation entitled<br />
“The Fundamental Principles <strong>of</strong> Equilibrium in Solid Bodies”. He then worked<br />
as an engineer <strong>and</strong> manager for an Italian railway company. He was very<br />
politically active, an ardent supporter <strong>of</strong> free market economy. He obtained a<br />
lecturer position in economics <strong>and</strong> management at the University <strong>of</strong> Florence<br />
in 1886 (according to wikipedia). Eventually he resigned from his positions in<br />
1889. During the 1880s he became acquainted with leading economists <strong>of</strong> the<br />
time <strong>and</strong> he published many articles by 1893 (not all academic, though). In<br />
1893 he moved to Lausanne where he lectured at the University <strong>of</strong> Lausanne<br />
<strong>and</strong> became the successor <strong>of</strong> Léon Walras as Pr<strong>of</strong>essor <strong>of</strong> Political Economy. In<br />
his later years he mainly worked in Sociology. <strong>Vilfredo</strong> <strong>Pareto</strong> died at Célégny,<br />
Switzerl<strong>and</strong>, on 19 August 1923. The University <strong>of</strong> Lausanne still has a Centre<br />
d’études interdisciplinaires Walras <strong>Pareto</strong> (http://www.unil.ch/cwp). Apart<br />
from <strong>Pareto</strong> optimality, <strong>Pareto</strong>’s name is attached to the <strong>Pareto</strong> principle (or<br />
80–20 rule), observing in 1906 that 80% <strong>of</strong> the property in Italy was owned by<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong> 449<br />
Figure 1: <strong>Vilfredo</strong> <strong>Pareto</strong> 1848–1923 (Picture scanned from the second French<br />
edition <strong>of</strong> <strong>Pareto</strong> (1906) published in 1927.)<br />
20% <strong>of</strong> the population <strong>and</strong> the <strong>Pareto</strong> distribution, a power law probability<br />
distribution.<br />
<strong>Pareto</strong> Optimality<br />
The origin <strong>of</strong> the term <strong>Pareto</strong> optimality goes back to the following text from<br />
<strong>Pareto</strong> (1906, Chapter VI, Section 33).<br />
Principeremo col definire un termine di cui è comodo fare uso per<br />
scansare lungaggini. Diremo che i componenti di una collettività<br />
godono, in una certa posizione, del massimo di <strong>of</strong>elimità, qu<strong>and</strong>o<br />
è impossibile allontanarsi pochissimo da quella posizione giov<strong>and</strong>o,<br />
o nuocendo, a tutti i componenti la collettività; ogni piccotissimo<br />
spostamento da quella posizione avendo necessariamente per effetto<br />
digiovareapartedeicomponentitacollettivitàedinuocereadaltri.<br />
Or in the English translation (<strong>Pareto</strong>, 1971, p. 261):<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
450 Matthias Ehrgott<br />
Wewillbeginbydefiningatermwhichisdesirabletouseinorderto<br />
avoid prolixity. We will say that the members <strong>of</strong> a collectivity enjoy<br />
maximum ophelimity in a certain position when it is impossible to<br />
find a way <strong>of</strong> moving from that position very slightly in such a<br />
manner that the ophelimity enjoyed by each <strong>of</strong> the individuals <strong>of</strong><br />
that collectivity increases or decreases. That is to say, any small<br />
displacement in departing from that position necessarily has the<br />
effect <strong>of</strong> increasing the ophelimity which certain individuals enjoy,<br />
<strong>and</strong> decreasing that which others enjoy, <strong>of</strong> being agreeable to some<br />
<strong>and</strong> disagreeable to others.<br />
Of course, <strong>Pareto</strong> here refers to the distribution <strong>of</strong> utility (ophelimity) among<br />
individuals in an economy rather than solutions <strong>of</strong> an optimization problem.<br />
<strong>Multi</strong>-<strong>objective</strong> optimization or mathematical optimization in general as we<br />
know it today, did not exist during <strong>Pareto</strong>’s lifetime, it only developed in<br />
the 1940s. And it is some <strong>of</strong> the founding works <strong>of</strong> Operations Research <strong>and</strong><br />
optimization that need to be cited here. Nobel Laureate in Economics T.C.<br />
Koopmans (1951) formally studied production as a resource allocation problem<br />
<strong>and</strong> the combination <strong>of</strong> activities to represent the output <strong>of</strong> commodities as a<br />
function <strong>of</strong> various factors. In this work he introduced the following definition<br />
<strong>of</strong> efficient vector (p. 60). “A point y in the commodity space is called efficient<br />
if it is possible [i.e., if y ∈ (A)], <strong>and</strong> if there exists no possible point ȳ ∈ (A) such<br />
that ȳ −y ≥ 0.” Note that (A) is what I called Y in Definition 2, i.e., possible<br />
means that there is some x such that y = Ax. Koopmans does hence only talk<br />
about efficient vectors in terms <strong>of</strong> the outcome set. He proves necessary <strong>and</strong><br />
sufficient conditions for efficiency, but he does not refer to <strong>Pareto</strong>, nor does he<br />
talk about <strong>Pareto</strong> optimal solutions as in Definition 1 – instead he refers to “an<br />
activity vector x (that) shall lead to an efficient point y = Ax”.<br />
Another classic reference in optimization is the seminal paper by<br />
Kuhn <strong>and</strong> Tucker (1951). They refer to the “vector maximum <strong>of</strong> Koopmans’<br />
efficient point type for several concave functions g 1 (x),...,g p (x)”. This<br />
seems to be the earliest reference to the optimization <strong>of</strong> several functions in<br />
mathematics. Kuhn <strong>and</strong> Tucker cite Koopmans (1951) when they talk about<br />
vector maximum. They also apply the term efficient to the solutions <strong>of</strong> vector<br />
optimization problems (i.e., in decision space) <strong>and</strong> introduce the notion <strong>of</strong><br />
proper efficiency. But, again, no mention <strong>of</strong> <strong>Pareto</strong>. Kuhn <strong>and</strong> Tucker (1951)<br />
cite another Nobel Laureate in Economics who contributed to the foundations<br />
<strong>of</strong> multi-<strong>objective</strong> optimization, Kenneth J. Arrow.<br />
Arrow discusses <strong>Pareto</strong> extensively in his economical work <strong>and</strong> statements <strong>of</strong><br />
the impossibility theorem today usually refer to <strong>Pareto</strong> optimality as one <strong>of</strong> the<br />
axioms that cannot be jointly satisfied by a social choice function, but this term<br />
does not appear in Arrow’s original formulation (Arrow, 1951). Arrow’s important<br />
contribution to multi-<strong>objective</strong> optimization (Arrow et al., 1953) starts as<br />
follows “A point s <strong>of</strong> a closed convex subset S <strong>of</strong> k-space is admissible if there is<br />
no t ∈ S with t i ≤ s i for all i = 1,...,k, t ≠ s.” This is, <strong>of</strong> course, the same as<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong> 451<br />
Koopmans’ definition <strong>of</strong> efficient point (whose paper Arrow et al. (1953) cite),<br />
<strong>and</strong> again is relevant in the outcome set <strong>of</strong> a multi-<strong>objective</strong> problem rather<br />
than the set <strong>of</strong> feasible solutions – no trace <strong>of</strong> <strong>Pareto</strong> here, either.<br />
There are a number <strong>of</strong> other definitions <strong>of</strong> <strong>Pareto</strong> optimal, efficient, or admissible<br />
points. Zadeh (1963) defines “A system S 0 ∈ C is noninferior in C if the<br />
intersection <strong>of</strong> C <strong>and</strong> Σ > (S 0 ) is empty.” Σ > (S 0 ) is the set <strong>of</strong> all systems which<br />
are better than S 0 with respect to a partial order ≥. Chankong <strong>and</strong> Haimes<br />
(1983) later use the same definition. While Zadeh cites Koopmans <strong>and</strong> Kuhn<br />
<strong>and</strong> Tucker, <strong>Pareto</strong> remains notably absent. The final term that is common<br />
today is that <strong>of</strong> a nondominated point.<br />
<strong>Multi</strong><strong>objective</strong> <strong>Optimization</strong> <strong>and</strong> Economics<br />
When did the term <strong>Pareto</strong> optimal first appear in the literature? As we have<br />
seen, it was not used in early mathematical works on multi-<strong>objective</strong> optimization.<br />
The answer is once again in economics. Little (1950, p. 87) in a discussion<br />
<strong>of</strong> the distribution <strong>of</strong> income (i.e., in the same context as <strong>Pareto</strong> himself) uses<br />
the term <strong>Pareto</strong> ‘optimum’ (with the quotation marks). The origin <strong>of</strong> the term<br />
is, therefore, clearly found in economics. It has then apparently mostly been<br />
used in economics, appearing in journals such as Public Choice <strong>and</strong> Journal<br />
<strong>of</strong> Economic Theory. As shown above, it was not used by the economists<br />
who are credited with having contributed to the origins <strong>of</strong> the mathematical<br />
theory <strong>of</strong> multi-<strong>objective</strong> optimization, but migrated to mathematics later on.<br />
The first journal articles that I could find are Basile <strong>and</strong> Vincent (1970) <strong>and</strong><br />
Vincent <strong>and</strong> Leitmann (1970). These articles also used the term undominated<br />
as an alternative. This then turned to nondominated in Yu <strong>and</strong> Leitmann<br />
(1974).<br />
Economics had a strong influence on the early history <strong>of</strong> multi-<strong>objective</strong> optimization,<br />
especially <strong>Pareto</strong>’s original definition <strong>of</strong> the term maximum ophelimity<br />
<strong>and</strong> the origin <strong>of</strong> the term <strong>Pareto</strong> optimum in Little (1950). The move<br />
into mathematics <strong>and</strong> optimization coincides with the mathematization <strong>of</strong> economics<br />
by scholars such as Koopmans <strong>and</strong> Arrow <strong>and</strong> finally the introduction<br />
<strong>of</strong> the topic into mathematical optimization by Kuhn <strong>and</strong> Tucker. It seems<br />
to have taken quite a while for <strong>Pareto</strong>’s name to appear in the mathematical<br />
optimization literature.<br />
The consequence <strong>of</strong> the history <strong>of</strong> <strong>Pareto</strong> optimality outlined above, is that<br />
at present there are quite a few terms (efficient, noninferior, nondominated,<br />
admissible, <strong>Pareto</strong> optimal) that express the same idea. Since multi-<strong>objective</strong><br />
optimization <strong>of</strong>ten distinguishes between decision vectors x ∈ X <strong>and</strong> outcome<br />
vectors y ∈ Y, one can find a large number <strong>of</strong> combinations <strong>of</strong> these terms<br />
in the literature used in parallel today, such as <strong>Pareto</strong> optimal decisions <strong>and</strong><br />
efficient outcomes.<br />
Itturnsoutthatthehistory<strong>of</strong>multi-<strong>objective</strong>optimization(vectoroptimization)<br />
is quite an interesting read, <strong>and</strong> I would like to refer interested readers<br />
to Stadler (1979) as a starting point. The history <strong>of</strong> multiple criteria deci-<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
452 Matthias Ehrgott<br />
sion making in general is the topic <strong>of</strong> the book Köksalan et al. (2011). These<br />
works also consider roots <strong>of</strong> multi-<strong>objective</strong> optimization in game theory <strong>and</strong><br />
the theory <strong>of</strong> ordered spaces <strong>and</strong> vector norms.<br />
References<br />
K. J. Arrow. Social Choice <strong>and</strong> Individual Values. Cowles Commission for<br />
Research in Economics Monograph No. 12. John Wiley & Sons, New York,<br />
1951.<br />
K. J. Arrow, E. W. Barankin, <strong>and</strong> D. Blackwell. Admissible points <strong>of</strong> convex<br />
sets. In H.W. Kuhn <strong>and</strong> A.W. Tucker, editors, Contributions to the Theory<br />
<strong>of</strong> Games, volume 2, pages 87–91. Princeton University Press, Princeton,<br />
1953.<br />
G. Basile <strong>and</strong> T. L. Vincent. Absolutely cooperative solution for a linear, multiplayer<br />
differential game. Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications,<br />
6:41–46, 1970.<br />
V. Chankong <strong>and</strong> Y. Y. Haimes. <strong>Multi</strong><strong>objective</strong> Decision Making – Theory <strong>and</strong><br />
Methodology. Elsevier Science, New York, 1983.<br />
G. Chen, X. Huang, <strong>and</strong> X. Yang. Vector <strong>Optimization</strong> – Set-Valued <strong>and</strong> Variational<br />
Analysis, volume 541 <strong>of</strong> Lecture Notes in Economics <strong>and</strong> Mathematical<br />
Systems. Springer Verlag, Berlin, 2005.<br />
F. Y. Edgeworth. Mathematical Psychics. C. Kegan Paul & Co., London, 1881.<br />
A. Göpfert <strong>and</strong> R. Nehse. Vektoroptimierung, volume 74 <strong>of</strong> Mathematisch-<br />
Naturwissenschaftliche Bibliothek. BSB B.G. Teubner Verlagsgesellschaft,<br />
Leipzig, 1990.<br />
J. Jahn. Vector <strong>Optimization</strong> – Theory, Applications, <strong>and</strong> Extensions. Springer<br />
Verlag, Berlin, 2004.<br />
M. Köksalan, J. Wallenius, <strong>and</strong> S. Zionts. <strong>Multi</strong>ple Criteria Decision Making<br />
– From Early History to the 21st Century. World Scientific Publishing,<br />
Singapore, 2011.<br />
T. C. Koopmans. Analysis <strong>of</strong> production as an efficient combination <strong>of</strong> activities.<br />
In T.C. Koopmans, editor, Activity Analysis <strong>of</strong> Production <strong>and</strong> Allocation,<br />
Cowles Commission for Research in Economics Monograph No. 13,<br />
pages 33–97. John Wiley & Sons, New York, 1951.<br />
H. W. Kuhn <strong>and</strong> A. W. Tucker. Nonlinear programming. In J. Neyman, editor,<br />
Proceedings <strong>of</strong> the Second Berkeley Symposium on Mathematical Statistics<br />
<strong>and</strong> Probability, pages 481–492. University <strong>of</strong> California Press, Berkeley,<br />
1951.<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
<strong>Vilfredo</strong> <strong>Pareto</strong> <strong>and</strong> <strong>Multi</strong>-<strong>objective</strong> <strong>Optimization</strong> 453<br />
I.M.D.Little. A Critique <strong>of</strong> Welfare Economics. TheClarendonPress, Oxford,<br />
1950.<br />
A. Löhne. Vector <strong>Optimization</strong> with Infimum <strong>and</strong> Supremum. Springer Verlag,<br />
Berlin, 2011.<br />
V. <strong>Pareto</strong>. Manuale di Economia Politica. Società Editrice Libraria, Milan,<br />
1906.<br />
V. <strong>Pareto</strong>. Manual <strong>of</strong> Political Economy. Augustus M. Kelley Publishers, New<br />
York, 1971.<br />
W. Stadler. A survey <strong>of</strong> multicriteria optimization or the vector maximum<br />
problem, Part I: 1776-1960. Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications,<br />
29:1–52, 1979.<br />
T. L. Vincent <strong>and</strong> G. Leitmann. Control-space properties <strong>of</strong> cooperative games.<br />
Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications, 6:91–113, 1970.<br />
P. L. Yu. <strong>Multi</strong>ple Criteria Decision Making: Concepts, Techniques <strong>and</strong> Extensions.<br />
Plenum Press, New York, 1985.<br />
P. L. Yu <strong>and</strong> G. Leitmann. Compromise solutions, domination structures, <strong>and</strong><br />
Salukvadze’s solution. Journal <strong>of</strong> <strong>Optimization</strong> Theory <strong>and</strong> Applications, 13:<br />
362–378, 1974.<br />
L. A. Zadeh. Optimality <strong>and</strong> non-scalar-valued performance criteria. IEEE<br />
Transactions on Automatic Control, 8:59–60, 1963.<br />
Matthias Ehrgott<br />
<strong>Department</strong><strong>of</strong>EngineeringScience<br />
The University <strong>of</strong> Auckl<strong>and</strong><br />
New Zeal<strong>and</strong><br />
m.ehrgott@auckl<strong>and</strong>.ac.nz<br />
Documenta Mathematica · Extra Volume ISMP (2012) 447–453
454<br />
Documenta Mathematica · Extra Volume ISMP (2012)