Foundations of Data Science

was included, we would just multiply by the index of summation.
Convert the equation to matrix form by defining z kj = 1 √ n
exp(− 2πi
n ki j) and write
f = Zx. Actually instead of x, write the vector consisting of the nonzero elements of x.
By its definition, x ≠ 0. To prove the lemma we need to show that f is nonzero. This will
be true provided Z is nonsingular. If we rescale Z by dividing each column by its leading
entry we get the Vandermonde determinant which is nonsingular.
Theorem 10.7 Let n x be the number of nonzero elements in x and let n f be the number
of nonzero elements in the Fourier transform of x. Let n x divide n. Then n x n f ≥ n.
Proof: If x has n x nonzero elements, f cannot have a consecutive block of n x zeros. Since
n x divides n there are n n x
blocks each containing at least one nonzero element. Thus, the
product of nonzero elements in x and f is at least n.
Fourier transform of spikes prove that above bound is tight
To show that the bound in Theorem 10.7 is tight we show that the Fourier transform
of the sequence of length n consisting of √ n ones, each one separated by √ n − 1 zeros,
is the sequence itself. For example, the Fourier transform of the sequence 100100100 is
100100100. Thus, for this class of sequences, n x n f = n.
Theorem 10.8 Let S ( √ n, √ n) be the sequence of 1’s and 0’s with √ n 1’s spaced √ n
apart. The Fourier transform of S ( √ n, √ n) is itself.
Proof: Consider the columns 0, √ n, 2 √ n, . . . , ( √ n − 1) √ n. These are the columns for
which S ( √ n, √ n) has value 1. The element of the matrix Z in the row j √ n of column
k √ n, 0 ≤ k < √ n is z nkj = 1. Thus, for these rows Z times the vector S ( √ n, √ n) = √ n
and the 1/ √ n normalization yields f j
√ n = 1.
For rows whose index is not of the form j √ n, the row b, b ≠ j √ n, j ∈ {0, √ n, . . . , √ n − 1},
the elements in row b in the columns 0, √ n, 2 √ n, . . . , ( √ n − 1) √ n are 1, z b , z 2b , . . . , z (√ n−1)b
)
and thus f b = √ 1
n
(1 + z b + z 2b · · · + z (√ n−1)b
= √ 1 z √ nb −1
n
= 0 since z b√n = 1 and z ≠ 1
z−1 .
Uniqueness of l 1 optimization
Consider a redundant representation for a sequence. One such representation would be
representing a sequence as the concatenation of two sequences, one specified by its coordinates
and the other by its Fourier transform. Suppose some sequence could be represented
as a sequence of coordinates and Fourier coefficients sparsely in two different ways. Then
by subtraction, the zero sequence could be represented by a sparse sequence. The representation
of the zero sequence cannot be solely coordinates or Fourier coefficients. If
y is the coordinate sequence in the representation of the zero sequence, then the Fourier
portion of the representation must represent −y. Thus y and its Fourier transform would
have sparse representations contradicting n x n f ≥ n. Notice that a factor of two comes in
341