U. Glaeser

Theorem 1 (Shannon) For any � > 0 and R < C(p), there is an [n, k] binary code of rate k/n ≥ R with
P err < �.
For a proof of Theorem 1 and some of its generalizations, the reader is referred to [5], or even to
Shannon’s original paper [6].
Theorem 1 has enormous theoretical importance. It shows that reliable communication is not limited
in the presence of noise, only the rate of communication is. For instance, if p = .01, the capacity of the
channel is C(.01) = .9192. Hence, there are codes of rate ≥.9 with P err arbitrarily small. It also tells us not
to look for codes with rate .92 making P err arbitrarily small.
The proof of Theorem 1, though, is based on probabilistic methods and the assumption of arbitrarily
large values of n. In practical applications, n cannot be too large. The theorem does not tell us how to
construct efficient codes, it just asserts their existence. Moreover, when we construct codes, we want them
to have efficient encoding and decoding algorithms. In the last few years, coding methods approaching the
Shannon limit have been developed, the so-called turbo codes. Although great progress has been made
towards practical implementations of turbo codes, in applications like magnetic recording their complexity
is still a problem. A description of turbo codes is beyond the scope of this introduction. The
reader is referred to [2].
Codes over Bytes and Finite Fields
So far, we have considered linear codes over bits. Next we want to introduce codes over larger symbols,
mainly over bytes. A byte of size ν is a vector of ν bits. Mathematically, bytes are vectors in GF(2) ν . Typical
cases in magnetic and optical recording involve 8-bit bytes. Most of the general results in the previous
sections for codes over bits easily extend to codes over bytes. It is trivial to multiply bits, but we need a
method to multiply bytes. To this end, the theory of finite fields has been developed. Next we give a
brief introduction to the theory of finite fields. For a more complete treatment, the reader is referred
to chapter 4 of [4].
We know how to add two binary vectors, we simply exclusive-OR them componentwise. What we
need now is a rule that allows us to multiply bytes while preserving associative, distributive, and multiplicative
inverse properties, i.e., a product that gives to the set of bytes of length ν the structure of a field.
To this end, we will define a multiplication between vectors that satisfies the associative and commutative
properties, it has a 1 element, each nonzero element is invertible and it is distributive with respect to the
sum operation.
Recall the definition of the ring Zm of integers modulo m: Zm is the set {0, 1, 2,…, m − 1}, with a sum
and product of any two elements defined as the residue of dividing by m the usual sum or product. It is
not difficult to prove that Zm is a field if and only if m is a prime number. Using this analogy, we will
give to (GF(2)) ν the structure of a field.
Consider the vector space (GF(2)) ν over the field GF(2). We can view each vector as a polynomial of
degree ≤ν − 1 as follows: the vector = (a0, a1,…, aν −1) corresponds to the polynomial a(α) = a0 + a1α +…+ αν−1 α ν−1 .
The goal is to give to (GF(2)) ν the structure of a field. We will denote such a field by GF(2 ν ). The sum
in GF(2 ν ) is the usual sum of vectors in (GF(2)) ν . We need now to define a product.
Let f(x) be an irreducible polynomial (i.e., it cannot be expressed as the product of two polynomials
of smaller degree) of degree ν whose coefficients are in GF(2). Let a(α) and b(α) be two elements of
GF(2 ν ). We define the product between a(α) and b(α) in GF(2 ν ) as the unique polynomial c(α) of degree
≤ν − 1 such that c(α) is the residue of dividing the product a(α)b(α) by f(α) (the notation g(x) � h(x)
(mod f(x)) means that g(x) and h(x) have the same residue after dividing by f(x), i.e., g(α) = h(α)).
The sum and product operations defined above give to GF(2 ν ) a field structure. The role of the
irreducible polynomial f(x) is the same as the prime number m when Zm is a field. In effect, the proof
that GF(2 ν ) is a field when m is irreducible is essentially the same as the proof that Zm is a field when m
is prime. From now on, we denote the elements in GF(2 ν a
) as polynomials in α of degree ≤ν − 1 with
coefficients in GF(2). Given two polynomials a(x) and b(x) with coefficients in GF(2), a(α)b(α) denotes
© 2002 by CRC Press LLC