Information Theory, Inference, and Learning ... - Inference Group

Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/0521642981You can buy this book for 30 pounds or $50. See http://www.inference.phy.cam.ac.uk/mackay/itila/ for links.208 13 — Binary CodesFigure 13.4. Schematic picture ofpart of Hamming space perfectlyfilled by t-spheres centred on thecodewords of a perfect code.1 2t. . .tt13.3 Perfect codesA t-sphere (or a sphere of radius t) in Hamming space, centred on a point x,is the set of points whose Hamming distance from x is less than or equal to t.The (7, 4) Hamming code has the beautiful property that if we place 1-spheres about each of its 16 codewords, those spheres perfectly fill Hammingspace without overlapping. As we saw in Chapter 1, every binary vector oflength 7 is within a distance of t = 1 of exactly one codeword of the Hammingcode.A code is a perfect t-error-correcting code if the set of t-spheres centredon the codewords of the code fill the Hamming space without overlapping.(See figure 13.4.)Let’s recap our cast of characters. The number of codewords is S = 2 K .The number of points in the entire Hamming space is 2 N . The number ofpoints in a Hamming sphere of radius t ist∑w=0( Nw). (13.1)For a code to be perfect with these parameters, we require S times the numberof points in the t-sphere to equal 2 N :t∑( Nfor a perfect code, 2w)Kor, equivalently,w=0t∑( Nw)w=0= 2 N (13.2)= 2 N−K . (13.3)For a perfect code, the number of noise vectors in one sphere must equalthe number of possible syndromes. The (7, 4) Hamming code satisfies thisnumerological condition because( 71 + = 21)3 . (13.4)