Foundations of Data Science

1 1 1 1 1 1 1 1 1
1 z z 2 z 3 z 4 z 5 z 6 z 7 z 8
1 z 2 z 4 z 6 z 8 z z 3 z 5 z 7
1 z 3 z 6 1 z 3 z 6 1 z 3 z 6
1 z 4 z 8 z 3 z 7 z 2 z 6 z z 5
1 z 5 z z 6 z 2 z 7 z 3 z 8 z 4
1 z 6 z 3 1 z 6 z 3 1 z 6 z 3
1 z 7 z 5 z 3 z z 8 z 6 z 4 z 2
1 z 8 z 7 z 6 z 5 z 4 z 3 z 2 z
Figure 10.5: The matrix Z for n=9.
when we subtract the two representations.
Suppose two sparse signals had Fourier transforms that agreed in almost all of their
coordinates. Then the difference would be a sparse signal with a sparse transform. This is
not possible. Thus, if one selects log n elements of their transform these elements should
distinguish between these two signals.
10.4.3 Biological
There are many areas where linear systems arise in which a sparse solution is unique.
One is in plant breading. Consider a breeder who has a number of apple trees and for
each tree observes the strength of some desirable feature. He wishes to determine which
genes are responsible for the feature so he can cross bread to obtain a tree that better
expresses the desirable feature. This gives rise to a set of equations Ax = b where each
row of the matrix A corresponds to a tree and each column to a position on the genone.
See Figure 10.6. The vector b corresponds to the strength of the desired feature in each
tree. The solution x tells us the position on the genone corresponding to the genes that
account for the feature. It would be surprising if there were two small independent sets
of genes that accounted for the desired feature. Thus, the matrix must have a property
that allows only one sparse solution.
10.4.4 Finding Overlapping Cliques or Communities
Consider a graph that consists of several cliques. Suppose we can observe only low
level information such as edges and we wish to identify the cliques. An instance of this
problem is the task of identifying which of ten players belongs to which of two teams
of five players each when one can only observe interactions between pairs of individuals.
There is an interaction between two players if and only if they are on the same team.
In this situation we have a matrix A with ( ) (
10
5 columns and 10
)
2 rows. The columns
represent possible teams and the rows represent pairs of individuals. Let b be the ( )
10
2
dimensional vector of observed interactions. Let x be a solution to Ax = b. There is a
342