An Introduction to Approximation Algorithms - School of Technology ...

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
An Introduction to Approximation Algorithms
Daya Gaur
Department of Computer Science and Engineering
Indian Institute of Technology Ropar
IGGA Workshop at DAIICT
14 March 2012

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
approximation algorithms
Example
trade o accuracy for time.
for every instance we compute an α approximate solution in
polynomial time.
Travelling salesperson: an approximation algorithm returns a tour
no more than twice the length of the shortest tour for every
instance.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Types of approximation algorithms
Good news: there are problems in NP that admit FPTAS.
Bad news: there are problems in NP than do not admit any
approximation algorithm (unless..)
Inapproximability can be thought of as more rened study of class
NP-C.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Vertex cover
IP : minimize ∑ w v x v
v∈V
x u + x v >= 1 ∀(u,v) ∈ E
x v ∈ {0,1} ∀v ∈ V
LP : minimize ∑ w v x v
v∈V
x u + x v >= 1 ∀(u,v) ∈ E
x v ≥ 0 ∀v ∈ V

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Matching
D − LP : minimize ∑ y uv
(u,v)∈e
∑ y uv ≤ w u for all u ∈ V
v∈n(u)
y uv ≥ 0 ∀v ∈ V
M : minimize ∑ y uv
(u,v)∈e
∑ y uv ≤ w u for all u ∈ V
v∈n(u)
y uv ∈ {0,1} ∀v ∈ V

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Set cover
IP : minimize ∑ w s x s
s∈S
∑ x s >= 1 ∀v ∈ U
s:v∈s
x s ∈ {0,1} ∀s ∈ S
LP : minimize ∑ w s x s
v∈v
∑ x s >= 1 ∀v ∈ U
s:v∈s
x s ≥ 0 ∀s ∈ S

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Set cover: dual
LP : maximize ∑ y u
u∈V
∑ y v ≤ w s ∀s ∈ S
v∈s
y v ≥ 0 ∀v ∈ V
ILP : maximize ∑ y u
u∈V
∑ y v ≤ w s ∀s ∈ S
v∈s
y v ∈ {0,1} ∀v ∈ V

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Shortest paths in digraphs
⎧
⎨
∑ x e =
⎩
e∈n(v)
ip : minimize ∑ w e x e
e∈e
1 if v = s
−1 if v = t
0 otherwise
x e ∈ {0,1} ∀e ∈ E
vertex, edge incidence matrix. (u,e) is 1 is edge goes out from u,
−1 otherwise.
LP relaxation polytope is integral (integrality gap is 1).

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Constrained shortest paths
⎧
⎨
∑ x e =
⎩
e∈n(v)
ip : minimize ∑ w e x e
e∈e
1 if v = s
−1 if v = t
0 otherwise
∑ d e x e ≤ d
e∈e
x e ∈ {0,1}∀e ∈ e

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Constrained shortest paths
IP 1 : min ∑
e∈e
w e x e + λ(∑
∑ x e =
e∈n(v)
⎧
⎨
⎩
e∈e
d e x e − d)
1 if v = s
−1 if v = t
0 otherwise
∑ d e x e ≤ d
e∈e
x e ∈ {0, 1} ∀e ∈ e
Let x∗ be the optimal integral solution to the constrainted shortest
path problem. v(ip, x∗) ≥ v(IP 1 , x∗) for λ ≥ 0.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Constrained shortest paths
IP L : min ∑
e∈e
∑ x e =
e∈n(v)
w e x e + λ(∑
⎧
⎨
⎩
e∈e
d e x e − d)
1 if v = s
−1 if v = t
0 otherwise
x e ∈ {0,1} ∀e ∈ e
Let x ′ be the optimal integral solution to IP L .
v(IP 1 , x∗) ≥ v(IP L , x ′ ) ∀λ ≥ 0.
Theorem
The value of optimal solution to IP L is a lower bound on the value of x∗.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Lagrangian relaxation
IP L is the lagrangian relaxation.
we want the best possible lower bound, therefore nd λ such
that optimal to IP L maximized.
largarangian can be solved using subgradient methods (note
that the function might not be dierentiable).
or using column generation (dantzig-wolfe decomposition).
lagrangian bound is atleast as good as the linear programming
bound, for integral polytopes the two bounds coincide

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Greedy algorithm for set cover
r = {}
sol = {}
while r != u
s : min { w(s)/|s \ r| }
sol

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Analysis
Proof.
∑ p(e) = w(sol)
e∈u
consider a set s = (s 1 ,...,s k )
for all
∑
s i
∈s
i, p(s i ) ≤ w(s)
k − i
p(s i ) ≤ w(s)h(k)
p(e)/h(n) is feasible in the dual
by weak duality the performance ratio is h(n).
s ′ is are ordered in the order they are covered by the greedy.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Vertex cover
ip : minimize ∑
v∈v
w v x v
x u + x v >= 1 ∀(u, v) ∈ e
x v ∈ {0, 1} ∀v ∈ v
lp : minimize ∑ w v x v
v∈v
x u + x v >= 1 ∀(u, v) ∈ e
x v ≥ 0 ∀v ∈ v
let x ∗ be the optimal lp solution.
x v =
{ 1 if x ∗
v >= 1 2
0 otherwise.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Vertex cover
each constraint contains atleast one variable with value ≥ 1/2
in x ∗ .
rounding gives a feasible integral solution.
value of the integral solution is at most double the value of the
optimal lp solution.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Half integrality of vertex cover
Denition
A solution to an LP is an extreme point if it cannot be expressed as
a convex combination of two other feasible solutions.
Lemma
Every extreme point solution is half integral i.e., x v ∈ {0,1/2,1}.
Proof.
a v =
V p = {v | x ∗ v > 1/2} V n = {v | xv ∗ < 1/2}
⎧
⎨ x ∗ ⎧
v + ɛ if v ∈ V p ⎨ x
xv ∗ v ∗ − ɛ
− ɛ if v ∈ V n b
⎩
v = xv ∗ + ɛ
⎩
x ∗ v
otherwise.
x ∗ v
if v ∈ V p
if v ∈ V n
otherwise.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Pricing method for vertex cover
Denition
p e ≥ 0 is the price associated with each edge e.
w v is the cost associated with each vertex v.
price p is fair if for every vertex ∑ e on v p e ≤ w v .
Theorem
A fair price is a lower bound on the cost of any vertex cover.
∑
∑ p e ≤ w v
e on v
p e ≤ w(s)
∑
v∈s e on v

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Algorithm
Denition
A vertex is saturated if ∑ e on v p e = w v
an edge is uncovered if neither of its endpoints are in the cover.
price of e = 0
while there exists an
uncovered edge e
raise price on e
without violating
fairness
s = { v | v is saturated}

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Analysis
Theorem
Cost of vertex cover produced is at most twice the fair price.
Proof.
Every vertex in the cover is saturated.
∑
∑ p e = w v
e on v
p e = w(s)
∑
v∈s e on v
2∑
e
p e ≥ w(s)

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Complementary slackness
n
∑
i=1
P : min
a ji x i ≥ b j ,
∑ n i=1 c ix i
j = 1..m
x i ≥ 0
m
∑
j=1
D : max
a ji y j ≤ c i ,
∑ m j=1 b jy j
i = 1..n
y j ≥ 0

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Complementary slackness
Theorem
Let x,y be primal and dual feasible solutions. x,y are optimal if
and only if following conditions are satised:
1 x i (∑ m j=1 a jiy j − c i ) = 0 for all 1 ≤ i ≤ n
2 y j (∑ n i=1 a jix i − b j ) = 0 for all 1 ≤ j ≤ m.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Relaxed complementary slackness
Denition
Let x,y be primal and dual feasible solutions. x,y are said to satisfy
relaxed complementary slackness condition if:
1 x i > 0 =⇒ c i
α ≤ ∑m j=1 a jiy j ≤ c i = 0 for all 1 ≤ i ≤ n
2 y j > 0 =⇒ (b j ≤ ∑ n i=1 a jix i ≤ βb j ) = 0 for all 1 ≤ j ≤ m.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
rcsc
Theorem
If x,y are feasible and satisfy rcsc then ∑ n i=1 c ix i ≤ αβ ∑ m j=1 b jy j .
Proof.
n
∑
i=1
n
∑
m∑
c i x i ≤ α
i=1(
j=1
m
∑
j=1( n∑
i=1
a ij x i
)
y j ≤ β
a ij y j
)
x i
m
∑ b j y j
j=1

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Pricing method for vertex cover revisited
α = 1, β = 2
x v > 0 =⇒ ∑ e on v p e = w v
p e > 0 =⇒ x u + x v ≤ 2
primal conditions: pick only saturated vertices in the cover.
dual conditions: from every edge pick atmost two vertices in
the cover.
primal conditions satised by pricing algorithm, dual conditions
satised automatically

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Point Cover
all the horizontal lines (12) comprise the optimal solution, and the greedy
algorithm will pick all the vertical lines (22). the example can be
generalized.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
2D
The problem is equivalent to vertex cover in bipartite graphs in 2-D.
Theorem ( KönigEgerváry)
The size of the minimum vertex cover is the same as the size of the
maximum matching in a bipartite graph.
2-D can be solved optimally.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
d dimensions
Let L be the set of all the axis parallel lines, associated with each
line l ∈ L is a binary variable y l whose value is 1 if the line is picked
in the solution, 0 otherwise. L(x i ) is the set of axis parallel lines
through point x i .
IP: min ∑ y l (1)
l∈L
∑
l : l∈L(x i )
y l ≥ 1 ∀ x i (2)
y l ∈ {0,1} (3)

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
The linear programming relaxation LP to the integer program IP, is
obtained by replacing constraints of type (3) with non-negativity
constraints y l ≥ 0. The linear programming dual of LP is:
LP-dual: max
n
∑
i=1
z i (4)
∑ z i ≤ 1 ∀ l ∈ L (5)
i : l∈L(x i )
z i ≥ 0 (6)

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
Primal-dual d approximation
while there exists an uncovered point, the algorithm picks all
the d axis parallel lines that go through the point.
set y l = 1 if the line is picked by the algorithm else y l = 0. let
x i be the uncovered point picked in iteration j, then set z i = 1.
solutions constructed above are feasible, and the value of the
primal solution is at most d times the value of the dual
solution, i.e. the performance ratio is d.

Introduction Lower bounds Dual tting Rounding Primal-dual Point cover
R. Bar-Yehuda and S. Even, A linear-time approximation algorithm for the weighted cover
problem. Journal of Algorithms, 2:198203, 1981.
G. B. Dantzig, L. R. Ford, D. R. Fulkerson, A primal-dual algorithm for linear programs, In Linear
Inequalities and Related Systems, Editors Kuhn and Tucker, pages 171181, Princeton University
Press, 1956.
T. Gonzalez, editor, Handbook of Approximation Algorithms and Meta Heuristics, CRC Press,
2007.
D. Hochbaum, editor, Approximation Algorithms for NP-hard problems, PWS Publishing, Boston,
1997.
V. Vazirani, Approximation Algorithms, Springer, 2001.
Figures are from Wikipedia.