Foundations of Data Science

In the discrete case, x = [x 0 , x 1 , . . . , x n−1 ] and f = [f 0 , f 1 , . . . , f n−1 ]. The Fourier transform
and its inverse are f = Ax with a ij = ω ij where ω is the principle n th root of unity.
There are many other transforms such as the Laplace, wavelets, chirplets, etc. In fact,
any nonsingular n × n matrix can be used as a transform.
If one has a discrete time sequence x of length n, the Nyquist theorem states that n
coefficients in the frequency domain are needed to represent the signal x. However, if the
signal x has only s nonzero elements, even though one does not know which elements they
are, one can recover the signal by randomly selecting a small subset of the coefficients in
the frequency domain. It turns out that one can reconstruct sparse signals with far fewer
samples than one might suspect and an area called compressed sampling has emerged
with important applications.
Motivation
Let A be an n × d matrix with n much smaller than d whose elements are generated
by independent, zero mean, unit variance, Gaussian processes. Let x be a sparse
d-dimensional vector with at most s nonzero coordinates, s