EE364a Homework 1 solutions

EE364a, Winter 2007-08Prof. S. BoydEE364a Homework 1 solutions2.1 Let C ⊆ R n be a convex set, with x 1 ,...,x k ∈ C, and let θ 1 ,...,θ k ∈ R satisfy θ i ≥ 0,θ 1 + · · ·+θ k = 1. Show that θ 1 x 1 + · · ·+θ k x k ∈ C. (The definition of convexity is thatthis holds for k = 2; you must show it for arbitrary k.) Hint. Use induction on k.Solution. This is readily shown by induction from the definition of convex set. Weillustrate the idea for k = 3, leaving the general case to the reader. Suppose thatx 1 ,x 2 ,x 3 ∈ C, and θ 1 + θ 2 + θ 3 = 1 with θ 1 ,θ 2 ,θ 3 ≥ 0. We will show that y =θ 1 x 1 + θ 2 x 2 + θ 3 x 3 ∈ C. At least one of the θ i is not equal to one; without loss ofgenerality we can assume that θ 1 ≠ 1. Then we can writey = θ 1 x 1 + (1 − θ 1 )(µ 2 x 2 + µ 3 x 3 )where µ 2 = θ 2 /(1 − θ 1 ) and µ 2 = θ 3 /(1 − θ 1 ). Note that µ 2 ,µ 3 ≥ 0 andµ 1 + µ 2 = θ 2 + θ 31 − θ 1= 1 − θ 11 − θ 1= 1.Since C is convex and x 2 ,x 3 ∈ C, we conclude that µ 2 x 2 + µ 3 x 3 ∈ C. Since this pointand x 1 are in C, y ∈ C.2.2 Show that a set is convex if and only if its intersection with any line is convex. Showthat a set is affine if and only if its intersection with any line is affine.Solution. We prove the first part. The intersection of two convex sets is convex.Therefore if S is a convex set, the intersection of S with a line is convex.Conversely, suppose the intersection of S with any line is convex. Take any two distinctpoints x 1 and x 2 ∈ S. The intersection of S with the line through x 1 and x 2 is convex.Therefore convex combinations of x 1 and x 2 belong to the intersection, hence also toS.2.5 What is the distance between two parallel hyperplanes {x ∈ R n | a T x = b 1 } and{x ∈ R n | a T x = b 2 }?Solution. The distance between the two hyperplanes is |b 1 − b 2 |/‖a‖ 2 . To see this,consider the construction in the figure below.1

(c) S = {x ∈ R n | x ≽ 0, x T y ≤ 1 for all y with ‖y‖ 2 = 1}.(d) S = {x ∈ R n | x ≽ 0, x T y ≤ 1 for all y with ∑ ni=1 |y i | = 1}.Solution.(a) S is a polyhedron. It is the parallelogram with corners a 1 + a 2 , a 1 − a 2 , −a 1 + a 2 ,−a 1 − a 2 , as shown below for an example in R 2 .a 2a 1c2c 1For simplicity we assume that a 1 and a 2 are independent. We can express S asthe intersection of three sets:• S 1 : the plane defined by a 1 and a 2• S 2 = {z + y 1 a 1 + y 2 a 2 | a T 1 z = a T 2 z = 0, −1 ≤ y 1 ≤ 1}. This is a slab parallelto a 2 and orthogonal to S 1• S 3 = {z + y 1 a 1 + y 2 a 2 | a T 1 z = a T 2 z = 0, −1 ≤ y 2 ≤ 1}. This is a slab parallelto a 1 and orthogonal to S 1Each of these sets can be described with linear inequalities.• S 1 can be described asv T k x = 0, k = 1,...,n − 2where v k are n − 2 independent vectors that are orthogonal to a 1 and a 2(which form a basis for the nullspace of the matrix [a 1 a 2 ] T ).• Let c 1 be a vector in the plane defined by a 1 and a 2 , and orthogonal to a 2 .For example, we can takeThen x ∈ S 2 if and only ifc 1 = a 1 − aT 1 a 2a‖a 2 ‖ 2 2 .2−|c T 1 a 1 | ≤ c T 1 x ≤ |c T 1 a 1 |.3

• Similarly, let c 2 be a vector in the plane defined by a 1 and a 2 , and orthogonalto a 1 , e.g.,Then x ∈ S 3 if and only ifc 2 = a 2 − aT 2 a 1a‖a 1 ‖ 2 1 .2−|c T 2 a 2 | ≤ c T 2 x ≤ |c T 2 a 2 |.Putting it all together, we can describe S as the solution set of 2n linear inequalitiesvk T x ≤ 0, k = 1,...,n − 2−vk T x ≤ 0, k = 1,...,n − 2c T 1 x ≤ |c T 1 a 1 |−c T 1 x ≤ |c T 1 a 1 |c T 2 x ≤ |c T 2 a 2 |−c T 2 x ≤ |c T 2 a 2 |.(b) S is a polyhedron, defined by linear inequalities x k ≥ 0 and three equality constraints.(c) S is not a polyhedron. It is the intersection of the unit ball {x | ‖x‖ 2 ≤ 1} andthe nonnegative orthant R n +. This follows from the following fact, which followsfrom the Cauchy-Schwarz inequality:x T y ≤ 1 for all y with ‖y‖ 2 = 1 ⇐⇒ ‖x‖ 2 ≤ 1.Although in this example we define S as an intersection of halfspaces, it is not apolyhedron, because the definition requires infinitely many halfspaces.(d) S is a polyhedron. S is the intersection of the set {x | |x k | ≤ 1, k = 1,...,n}and the nonnegative orthant R n +. This follows from the following fact:x T y ≤ 1 for all y withn∑|y i | = 1 ⇐⇒ |x i | ≤ 1, i = 1,...,n.i=1We can prove this as follows. First suppose that |x i | ≤ 1 for all i. Thenx T y = ∑ ix i y i ≤ ∑ i|x i ||y i | ≤ ∑ i|y i | = 1if ∑ i |y i | = 1.Conversely, suppose that x is a nonzero vector that satisfies x T y ≤ 1 for all ywith ∑ i |y i | = 1. In particular we can make the following choice for y: let k bean index for which |x k | = max i |x i |, and take y k = 1 if x k > 0, y k = −1 if x k < 0,and y i = 0 for i ≠ k. With this choice of y we havex T y = ∑ ix i y i = y k x k = |x k | = maxi|x i |.4

Therefore we must have max i |x i | ≤ 1.All this implies that we can describe S by a finite number of linear inequalities:it is the intersection of the nonnegative orthant with the set {x | − 1 ≼ x ≼ 1},i.e., the solution of 2n linear inequalities−x i ≤ 0, i = 1,...,nx i ≤ 1, i = 1,...,n.Note that as in part (c) the set S was given as an intersection of an infinitenumber of halfspaces. The difference is that here most of the linear inequalitiesare redundant, and only a finite number are needed to characterize S.None of these sets are affine sets or subspaces, except in some trivial cases. For example,the set defined in part (a) is a subspace (hence an affine set), if a 1 = a 2 = 0; the setdefined in part (b) is an affine set if n = 1 and S = {1}; etc.2.11 Hyperbolic sets. Show that the hyperbolic set {x ∈ R 2 + | x 1 x 2 ≥ 1} is convex. As ageneralization, show that {x ∈ R n ∏+ | ni=1x i ≥ 1} is convex. Hint. If a,b ≥ 0 and0 ≤ θ ≤ 1, then a θ b 1−θ ≤ θa + (1 − θ)b; see §3.1.9.Solution.(a) We prove the first part without using the hint. Consider a convex combination zof two points (x 1 ,x 2 ) and (y 1 ,y 2 ) in the set. If x ≽ y, then z = θx + (1 − θ)y ≽ yand obviously z 1 z 2 ≥ y 1 y 2 ≥ 1. Similar proof if y ≽ x.Suppose y ⋡ x and x ⋡ y, i.e., (y 1 − x 1 )(y 2 − x 2 ) < 0. Then(θx 1 + (1 − θ)y 1 )(θx 2 + (1 − θ)y 2 )= θ 2 x 1 x 2 + (1 − θ) 2 y 1 y 2 + θ(1 − θ)x 1 y 2 + θ(1 − θ)x 2 y 1= θx 1 x 2 + (1 − θ)y 1 y 2 − θ(1 − θ)(y 1 − x 1 )(y 2 − x 2 )≥ 1.(b) Assume that ∏ i x i ≥ 1 and ∏ i y i ≥ 1. Using the inequality in the hint, we have∏(θx i + (1 − θ)y i ) ≥ ∏ x θ iyi1−θ = ( ∏ii2.12 Which of the following sets are convex?x i ) θ ( ∏ i(a) A slab, i.e., a set of the form {x ∈ R n | α ≤ a T x ≤ β}.y i ) 1−θ ≥ 1.(b) A rectangle, i.e., a set of the form {x ∈ R n | α i ≤ x i ≤ β i , i = 1,...,n}. Arectangle is sometimes called a hyperrectangle when n > 2.(c) A wedge, i.e., {x ∈ R n | a T 1 x ≤ b 1 , a T 2 x ≤ b 2 }.5

(d) The set of points closer to a given point than a given set, i.e.,where S ⊆ R n .{x | ‖x − x 0 ‖ 2 ≤ ‖x − y‖ 2 for all y ∈ S}(e) The set of points closer to one set than another, i.e.,where S,T ⊆ R n , and{x | dist(x,S) ≤ dist(x,T)},dist(x,S) = inf{‖x − z‖ 2 | z ∈ S}.(f) The set {x | x + S 2 ⊆ S 1 }, where S 1 ,S 2 ⊆ R n with S 1 convex.(g) The set of points whose distance to a does not exceed a fixed fraction θ of thedistance to b, i.e., the set {x | ‖x − a‖ 2 ≤ θ‖x − b‖ 2 }. You can assume a ≠ b and0 ≤ θ ≤ 1.Solution.(a) A slab is an intersection of two halfspaces, hence it is a convex set and a polyhedron.(b) As in part (a), a rectangle is a convex set and a polyhedron because it is a finiteintersection of halfspaces.(c) A wedge is an intersection of two halfspaces, so it is convex and a polyhedron. Itis a cone if b 1 = 0 and b 2 = 0.(d) This set is convex because it can be expressed as⋂{x | ‖x − x 0 ‖ 2 ≤ ‖x − y‖ 2 },y∈Si.e., an intersection of halfspaces. (Recall from exercise 2.7 that, for fixed y, theset{x | ‖x − x 0 ‖ 2 ≤ ‖x − y‖ 2 }is a halfspace.)(e) In general this set is not convex, as the following example in R shows. WithS = {−1, 1} and T = {0}, we have{x | dist(x,S) ≤ dist(x,T)} = {x ∈ R | x ≤ −1/2 or x ≥ 1/2}which clearly is not convex.6

(f) This set is convex. x + S 2 ⊆ S 1 if x + y ∈ S 1 for all y ∈ S 2 . Therefore{x | x + S 2 ⊆ S 1 } = ⋂the intersection of convex sets S 1 − y.(g) The set is convex, in fact a ball.{x | ‖x − a‖ 2 ≤ θ‖x − b‖ 2 }= {x | ‖x − a‖ 2 2 ≤ θ 2 ‖x − b‖ 2 2}y∈S 2{x | x + y ∈ S 1 } = ⋂y∈S 2(S 1 − y),= {x | (1 − θ 2 )x T x − 2(a − θ 2 b) T x + (a T a − θ 2 b T b) ≤ 0}If θ = 1, this is a halfspace. If θ < 1, it is a ballwith center x 0 and radius R given by{x | (x − x 0 ) T (x − x 0 ) ≤ R 2 },x 0 = a − ( θ2 b θ 21 − θ , R = ‖b‖ 2 2 − ‖a‖ 2 ) 1/22+ ‖x 2 1 − θ 2 0 ‖ 2 2 .2.15 Some sets of probability distributions. Let x be a real-valued random variable withprob(x = a i ) = p i , i = 1,...,n, where a 1 < a 2 < · · · < a n . Of course p ∈ R n lies inthe standard probability simplex P = {p | 1 T p = 1, p ≽ 0}. Which of the followingconditions are convex in p? (That is, for which of the following conditions is the set ofp ∈ P that satisfy the condition convex?)(a) α ≤ E f(x) ≤ β, where E f(x) is the expected value of f(x), i.e., E f(x) =∑ ni=1p i f(a i ). (The function f : R → R is given.)(b) prob(x > α) ≤ β.(c) E |x 3 | ≤ αE|x|.(d) E x 2 ≤ α.(e) E x 2 ≥ α.(f) var(x) ≤ α, where var(x) = E(x − E x) 2 is the variance of x.(g) var(x) ≥ α.(h) quartile(x) ≥ α, where quartile(x) = inf{β | prob(x ≤ β) ≥ 0.25}.(i) quartile(x) ≤ α.Solution. We first note that the constraints p i ≥ 0, i = 1,...,n, define halfspaces,and ∑ ni=1 p i = 1 defines a hyperplane, so P is a polyhedron.The first five constraints are, in fact, linear inequalities in the probabilities p i .7

(a) E f(x) = ∑ ni=1 p i f(a i ), so the constraint is equivalent to two linear inequalitiesn∑α ≤ p i f(a i ) ≤ β.i=1(b) prob(x ≥ α) = ∑ i: a i ≥α p i , so the constraint is equivalent to a linear inequality∑i: a i ≥αp i ≤ β.(c) The constraint is equivalent to a linear inequalityn∑p i (|a 3 i | − α|a i |) ≤ 0.i=1(d) The constraint is equivalent to a linear inequalityn∑p i a 2 i ≤ α.i=1(e) The constraint is equivalent to a linear inequalityn∑p i a 2 i ≥ α.i=1The first five constraints therefore define convex sets.(f) The constraintn∑ n∑var(x) = E x 2 − (E x) 2 = p i a 2 i − ( p i a i ) 2 ≤ αi=1 i=1is not convex in general. As a counterexample, we can take n = 2, a 1 = 0, a 2 = 1,and α = 1/5. p = (1, 0) and p = (0, 1) are two points that satisfy var(x) ≤ α,but the convex combination p = (1/2, 1/2) does not.(g) This constraint is equivalent ton∑ n∑a 2 ip i + ( a i p i ) 2 = b T p + p T Ap ≤ αi=1 i=1where b i = a 2 i and A = aa T . This defines a convex set, since the matrix aa T ispositive semidefinite.Let us denote quartile(x) = f(p) to emphasize it is a function of p. The figureillustrates the definition. It shows the cumulative distribution for a distribution p withf(p) = a 2 .8

p 1 + p 2 + · · · + p n−1prob(x ≤ β)1p 1 + p 20.25p 1a 1 a 2 a nβ(h) The constraint f(p) ≥ α is equivalent toprob(x ≤ β) < 0.25 for all β < α.If α ≤ a 1 , this is always true. Otherwise, define k = max{i | a i < α}. This is afixed integer, independent of p. The constraint f(p) ≥ α holds if and only ifk∑prob(x ≤ a k ) = p i < 0.25.i=1This is a strict linear inequality in p, which defines an open halfspace.(i) The constraint f(p) ≤ α is equivalent toprob(x ≤ β) ≥ 0.25 for all β ≥ α.Here, let us define k = max{i | a i ≤ α}. Again, this is a fixed integer, independentof p. The constraint f(p) ≤ α holds if and only ifk∑prob(x ≤ a k ) = p i ≥ 0.25.i=1If α ≤ a 1 , then no p satisfies f(p) ≤ α, which means that the set is empty.9