Information Theory, Inference, and Learning ... - MAELabs UCSD

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981
You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.
578 48 — Convolutional Codes and Turbo Codes
1111
1110
1101
1100
1011
1010
1001
1000
0111
0110
0101
0100
0011
0010
0001
0000
received
z4 〈d
z3 〈d
z2 〈d
z1
〈d
✲
⊕
✻
z0
✲ t (b)
✻
✲
t (a)
⊕ ⊕ ✛ ❵ s
❄ ⊕ ✲
❄ ⊕ ✲
❄ ✲ ✲
21 1, 37 8
0 1 1 0 1 0 1 0 0 0 0 1 1 1 1 0
In general a linear-feedback shift-register with k bits of memory has an impulse
response that is periodic with a period that is at most 2 k − 1, corresponding
to the filter visiting every non-zero state in its state space.
Incidentally, cheap pseudorandom number generators and cheap cryptographic
products make use of exactly these periodic sequences, though with
larger values of k than 7; the random number seed or cryptographic key selects
the initial state of the memory. There is thus a close connection between
certain cryptanalysis problems and the decoding of convolutional codes.
48.3 Decoding convolutional codes
The receiver receives a bit stream, and wishes to infer the state sequence
and thence the source stream. The posterior probability of each bit can be
found by the sum–product algorithm (also known as the forward–backward or
BCJR algorithm), which was introduced in section 25.3. The most probable
state sequence can be found using the min–sum algorithm of section 25.3
(also known as the Viterbi algorithm). The nature of this task is illustrated
in figure 48.6, which shows the cost associated with each edge in the trellis
for the case of a sixteen-state code; the channel is assumed to be a binary
symmetric channel and the received vector is equal to a codeword except that
one bit has been flipped. There are three line styles, depending on the value
of the likelihood: thick solid lines show the edges in the trellis that match the
corresponding two bits of the received string exactly; thick dotted lines show
edges that match one bit but mismatch the other; and thin dotted lines show
the edges that mismatch both bits. The min–sum algorithm seeks the path
through the trellis that uses as many solid lines as possible; more precisely, it
minimizes the cost of the path, where the cost is zero for a solid line, one for
a thick dotted line, and two for a thin dotted line.
⊲ Exercise 48.2. [1, p.581] Can you spot the most probable path and the flipped
bit?
Figure 48.6. The trellis for a k = 4
code painted with the likelihood
function when the received vector
is equal to a codeword with just
one bit flipped. There are three
line styles, depending on the value
of the likelihood: thick solid lines
show the edges in the trellis that
match the corresponding two bits
of the received string exactly;
thick dotted lines show edges that
match one bit but mismatch the
other; and thin dotted lines show
the edges that mismatch both
bits.