Information Theory, Inference, and Learning ... - MAELabs UCSD

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981
You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.
13.10: Dual codes 217
One way of thinking of this equation is that each row of H specifies a vector
to which t must be orthogonal if it is a codeword.
The generator matrix specifies K vectors from which all codewords
can be built, and the parity-check matrix specifies a set of M vectors
to which all codewords are orthogonal.
The dual of a code is obtained by exchanging the generator matrix
and the parity-check matrix.
Definition. The set of all vectors of length N that are orthogonal to all codewords
in a code, C, is called the dual of the code, C ⊥ .
If t is orthogonal to h1 and h2, then it is also orthogonal to h3 ≡ h1 + h2;
so all codewords are orthogonal to any linear combination of the M rows of
H. So the set of all linear combinations of the rows of the parity-check matrix
is the dual code.
For our Hamming (7, 4) code, the parity-check matrix is (from p.12):
H = P I3
⎡
= ⎣
1 1 1 0 1 0
⎤
0
0 1 1 1 0 1 0 ⎦ . (13.28)
1 0 1 1 0 0 1
The dual of the (7, 4) Hamming code H (7,4) is the code shown in table 13.16.
0000000
0010111
0101101
0111010
1001110
1011001
1100011
1110100
A possibly unexpected property of this pair of codes is that the dual,
H ⊥ (7,4) , is contained within the code H (7,4) itself: every word in the dual code
is a codeword of the original (7, 4) Hamming code. This relationship can be
written using set notation:
H ⊥ (7,4) ⊂ H (7,4). (13.29)
The possibility that the set of dual vectors can overlap the set of codeword
vectors is counterintuitive if we think of the vectors as real vectors – how can
a vector be orthogonal to itself? But when we work in modulo-two arithmetic,
many non-zero vectors are indeed orthogonal to themselves!
⊲ Exercise 13.7. [1, p.223] Give a simple rule that distinguishes whether a binary
vector is orthogonal to itself, as is each of the three vectors [1 1 1 0 1 0 0],
[0 1 1 1 0 1 0], and [1 0 1 1 0 0 1].
Some more duals
In general, if a code has a systematic generator matrix,
G = [IK|P T ] , (13.30)
where P is a K × M matrix, then its parity-check matrix is
H = [P|IM ] . (13.31)
Table 13.16. The eight codewords
of the dual of the (7, 4) Hamming
code. [Compare with table 1.14,
p.9.]