B. P. Lathi, Zhi Ding - Modern Digital and Analog Communication Systems-Oxford University Press (2009)

Problems 797
3. H. Nyquist, "Certain Factors Affecting Telegraph Speed," Bell Syst. Te ch. J. , vol. 3, pp. 324-346,
April 1924.
4. N. Abramson, Information Theory and Coding, McGraw-Hill, New York, 1963.
5. R. G. Gallager, Information Theory and Reliable Communication, Wiley, New York, 1968.
6. D. A. Huffman, "A Method for Construction of Minimum Redundancy Codes,'' Proc. IRE, vol. 40,
pp. 1098-110 I, Sept. 1952.
7. R. W. Hamming, Coding and Information Theory, 2nd ed., Prentice-Hall, Englewood Cliffs, NJ,
1986.
8. C. E. Shannon, "Communication in the Presence of Noise," Proc. IRE, vol. 37, pp. 10-21, Jan. 1949.
9. J. M. Wozencraft and I. A. Jacobs, Principles of Communication Engineering, Wiley, New York,
1965, Chapter 5.
10. A. J. Viterbi, Principles of Coherent Communication, McGraw-Hill, New York, 1966.
11. A. Paulraj, R. Nabar, and D. Gore, lntroduction lo Space-Time Wireless Communications, Cambridge
University Press, Cambridge, 2003.
12. T. M. Cover and J. A Thomas, Elements of Information Theory, Wiley-Interscience, New York, 1991.
PROBLEMS
13.1-1 A message source generates one of four messages randomly every microsecond. The probabilities
of these messages are 0.4, 0.3, 0.2, and 0.1. Each emitted message is independent of the other
messages in the sequence.
(a) What is the source entropy?
(b) What is the rate of information generated by this source (in bits per second)?
13.1-2 A standard television picture is composed of approximately 300,000 basic picture elements
(about 600 picture elements in a horizontal line and 500 horizontal lines per frame). Each of
these elements can assume 10 distinguishable brightness levels (such as black and shades of
gray) with equal probability. Find the information content of a television picture frame.
13.1-3 A radio announcer describes a television picture orally in 1000 words from his vocabulary of
10,000 words. Assume that each of the 10,000 words in the announcer's vocabulary is equally
likely to occur in the description of this picture (a crude approximation, but good enough to
give an idea). Determine the amount of infotmation broadcast by the announcer in describing
the picture. Would you say the announcer can do justice to the picture in 1000 words? ls the
old adage "A picture is worth a thousand words" an exaggeration or an understatement of the
reality? Use data in Prob. 13.1-2 to estimate the information of a picture.
13.1-4 From the town of the Old North Church in Boston, Paul Revere's friend was to show one lantern
if the British army began advancing overland and two lanterns if they had chosen to cross the
bay in boats.
(a) Assume that Revere had no way of guessing ahead of time what route the British might
choose. How much information did he receive when he saw two lanterns?
(b) What if Revere were 90% sure the British would march overland? Then, how much
information would the two lanterns have conveyed?
13.1-5 Estimate the information per letter in the English language by various methods, assuming that
each character is independent of the others. (This is not true, but is good enough to get a rough
idea.)
(a) In the first method, assume that all 27 characters (26 letters and a space) are equiprobable.
This is a gross approximation, but good for a quick answer.
(b) In the second method, use the table of probabilities of various characters (Table Pl3.l-5).