13.07.2015 Views

View File - University of Engineering and Technology, Taxila

View File - University of Engineering and Technology, Taxila

View File - University of Engineering and Technology, Taxila

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

FIGURE 3.19A generalized model <strong>of</strong> a digital communication system.symbols from a finite source states x i , which are traversed via a discretememoryless channel (DMC) to produce data y i .The output data y i , which contain N symbols, can be viewed as a timesequence <strong>of</strong> source states. The term memoryless implies that the source outputis independent <strong>of</strong> all preceding <strong>and</strong> succeeding source outputs. For convenience,we write the input <strong>and</strong> output states asX ¼ðx 1 ; x 2 ; x 3 ; ...; x M ÞY ¼ðy 1 ; y 2 ; y 3 ; ...; y N Þð3:58ÞThe theory <strong>of</strong> effective encoding <strong>of</strong> signals from various sources is a branch <strong>of</strong>modern information theory. The results <strong>of</strong> the theory descend from theShannon theorem, which states that the minimum average number <strong>of</strong> symbolsnecessary for data transmission from some sources equals its entropy, H. Thus,entropy implies the average information associated with the sources; theaverage is over all source states. The information I associated with sourcestates x i is1Iðx i Þ¼log a ð3:59Þpðx i Þwhere a, the base <strong>of</strong> the logarithm, determines the unit in which theinformation is measured. Also, pðx i Þ is the probability that the source stateis transmitted. From elementary mathematics, the sum <strong>of</strong> the source probabilitiesis unity:P Mi¼1Note thatpðx i Þ¼1log a k ¼ log 10 klog 10 að3:60aÞð3:60bÞEquation (3.59) satisfies these three conditions:1. Iðx i Þ0 within the bound 0 pðx i Þ1.2. As pðx i Þ approaches unity, Iðx i Þ approaches zero.Copyright © 2002 by Marcel Dekker, Inc. All Rights Reserved.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!