Data Compression: The Complete Reference

2.8 Huffman Coding 79p 5 =(a + b) +(a +2b) =p 3 + p 4 , p 6 =(a +2b) +(2a +3b) =p 4 + p 5 ,andsoon(Figure 2.26c). These probabilities form a Fibonacci sequence whose first two elementsare a and b. As an example, we select a = 5 and b = 2 and generate the 5-numberFibonacci sequence 5, 2, 7, 9, and 16. These five numbers add up to 39, so dividingthem by 39 produces the five probabilities 5/39, 2/39, 7/39, 9/39, and 15/39. TheHuffman tree generated by them has a maximal height (which is 4).5a+8b03a+5b102a+3b1101110a+2ba+b11110 11111a b(a) (b) (c)Figure 2.26: Shortest and Tallest Huffman Trees.000 001 010 011 100 101 110 111In principle, symbols in a set can have any probabilities, but in practice, the probabilitiesof symbols in an input file are computed by counting the number of occurrencesof each symbol. Imagine a text file where only the nine symbols A–I appear. In orderfor such a file to produce the tallest Huffman tree, where the codes will have lengthsof from 1 to 8 bits, the frequencies of occurrence of the nine symbols have to form aFibonacci sequence of probabilities. This happens when the frequencies of the symbolsare 1, 1, 2, 3, 5, 8, 13, 21, and 34, (or integer multiples of these). The sum of thesefrequencies is 88, so our file has to be at least that long in order for a symbol to have8-bit Huffman codes. Similarly, if we want to limit the sizes of the Huffman codes of aset of n symbols to 16 bits, we need to count frequencies of at least 4180 symbols. Tolimit the code sizes to 32 bits, the minimum data size is 9,227,464 symbols.If a set of symbols happens to have the Fibonacci probabilities and therefore resultsin a maximal-height Huffman tree with codes that are too long, the tree can be reshaped(and the maximum code length shortened) by slightly modifying the symbol probabilities,so they are not much different from the original, but do not form a Fibonaccisequence.2.8.6 Canonical Huffman CodesCode 2.24c has a simple interpretation. It assigns the first four symbols the 3-bit codes0, 1, 2, 3, and the last two symbols the 2-bit codes 2 and 3. This is an example ofa canonical Huffman code. The word “canonical” means that this particular code hasbeen selected from among the several (or even many) possible Huffman codes becauseitspropertiesmakeiteasyandfasttouse.Table 2.27 shows a slightly bigger example of a canonical Huffman code. Imaginea set of 16 symbols (whose probabilities are irrelevant and are not shown) such thatfour symbols are assigned 3-bit codes, five symbols are assigned 5-bit codes, and theremaining seven symbols, 6-bit codes. Table 2.27a shows a set of possible Huffman