View File - University of Engineering and Technology, Taxila

To have a feel for this expression, an illustration in the form of Example 3.5will be examined after the concept of channel capacity has been introduced.Channel Capacity. Channel capacity is the maximum value of mutualinformation IðY ; X Þ. As an illustration, the BSC as the DMC in Fig. 3.21 isused to evaluate the channel capacity. The average information transferred isgiven by (3.71). The output entropy isHðY Þ¼ PN¼2j¼1pðy j Þ log 21pðy j Þ¼ pðy 1 Þ log 21pðy 1 Þ þ pðy 2Þ log 21pðy 2 Þð3:81ÞBy allowing the crossover probabilities to be equal ðpðy 1 j x 2 Þ¼pðy 2 j x 1 Þ¼aÞ and the states probabilities to be pðx 1 Þ¼b and pðx 2 Þ¼ð1 bÞ, we can write the receiving self-probability using (3.68) aspðy 1 Þ¼ð1 aÞb þ að1 bÞ ð3:82ÞTherefore, pðy 2 Þ¼1pðy 1 Þ. So, the output entropy becomesHðY Þ¼Oðpðy 1 ÞÞ ¼ Oðb þ a 2abÞ ð3:83ÞFollowing the iterative approach, we can show that the conditional outputentropy isHðY j X Þ¼OðaÞð3:84Þwhere Oð Þis a functional abstract term. This expression depends on the noisein the channel. Substituting (3.83) and (3.84) in (3.71), the BSC averagemutual information isIðX ; Y Þ¼Oðb þ a 2abÞ OðaÞ ð3:85ÞIn real life a channel is fixed and the channel’s noise is difficult to control. Theobjective, therefore, will be to maximize the first Oð Þ-term, the nonnoisecomponent term, on the right-hand side of (3.85). It was noticed in Fig. 3.20that the entropy HðbÞ is maximum when the symbols’ source probabilities areequally probable; that is, b ¼ 0:5. Therefore, we want the nonnoise componentof (3.85) to be0:5 ¼ðb þ a 2abÞ ð3:86ÞCopyright © 2002 by Marcel Dekker, Inc. All Rights Reserved.