Boyd Convex Optimization book - SFU Wiki

338 6 Approximation and fitting
at given points u i ∈ R k ? (Here we do not restrict f to lie in any finite-dimensional
subspace of functions.) The answer is: if and only if there exist g 1 , . . . , g m such
that
y j ≥ y i + g T i (u j − u i ), i, j = 1, . . . , m. (6.19)
To see this, first suppose that f is convex, dom f = R k , and f(u i ) = y i ,
i = 1, . . . , m. At each u i we can find a vector g i such that
f(z) ≥ f(u i ) + g T i (z − u i ) (6.20)
for all z. If f is differentiable, we can take g i = ∇f(u i ); in the more general case,
we can construct g i by finding a supporting hyperplane to epi f at (u i , y i ). (The
vectors g i are called subgradients.) By applying (6.20) to z = u j , we obtain (6.19).
Conversely, suppose g 1 , . . . , g m satisfy (6.19). Define f as
f(z) =
max (y i + gi T (z − u i ))
i=1,...,m
for all z ∈ R k . Clearly, f is a (piecewise-linear) convex function. The inequalities
(6.19) imply that f(u i ) = y i , for i = 1, . . . , m.
We can use this result to solve several problems involving interpolation, approximation,
or bounding, with convex functions.
Fitting a convex function to given data
Perhaps the simplest application is to compute the least-squares fit of a convex
function to given data (u i , y i ), i = 1, . . . , m:
minimize
∑ m
i=1 (y i − f(u i )) 2
subject to f : R k → R is convex, dom f = R k .
This is an infinite-dimensional problem, since the variable is f, which is in the
space of continuous real-valued functions on R k . Using the result above, we can
formulate this problem as
minimize
∑ m
i=1 (y i − ŷ i ) 2
subject to ŷ j ≥ ŷ i + gi T (u j − u i ), i, j = 1, . . . , m,
which is a QP with variables ŷ ∈ R m and g 1 , . . . , g m ∈ R k . The optimal value of
this problem is zero if and only if the given data can be interpolated by a convex
function, i.e., if there is a convex function that satisfies f(u i ) = y i . An example is
shown in figure 6.24.
Bounding values of an interpolating convex function
As another simple example, suppose that we are given data (u i , y i ), i = 1, . . . , m,
which can be interpolated by a convex function. We would like to determine the
range of possible values of f(u 0 ), where u 0 is another point in R k , and f is any
convex function that interpolates the given data. To find the smallest possible
value of f(u 0 ) we solve the LP
minimize y 0
subject to y j ≥ y i + gi T (u j − u i ), i, j = 0, . . . , m,