B. P. Lathi, Zhi Ding - Modern Digital and Analog Communication Systems-Oxford University Press (2009)

8
FUNDAMENTALS OF
PROBABILITY THEORY
Thus far, we have been studying signals whose values at any instant t are determined
by their analytical or graphical description. These are called deterministic signals,
implying complete certainty about their values at any moment t. Such signals, which
can be specified with certainty, cannot convey information. It will be seen in Chapter 13 that
information is inherently related to uncertainty. The higher the uncertainty about a signal (or
message) to be received, the higher its information content. If a message to be received is
specified (i.e., if it is known beforehand), then it contains no uncertainty and conveys no new
information to the receiver. Hence, signals that convey information must be unpredictable.
In addition to information-bearing signals, noise signals that perturb information signals in a
system are also unpredictable (otherwise they can simply be subtracted). These unpredictable
message signals and noise waveforms are examples of random processes that play key roles
in communication systems and their analysis.
Random phenomena arise either because of our partial ignorance of the generating mechanism
(as in message or noise signals) or because the laws governing the phenomena may
be fundamentally random (as in quantum mechanics). Yet in another situation, such as the
outcome of rolling a die, it is possible to predict the outcome provided we know exactly all
the conditions: the angle of the throw, the nature of the surface on which it is thrown, the
force imparted by the player, and so on. The exact analysis, however, is so complex and so
sensitive to all the conditions that it is impractical to carry it out, and we are content to accept
the outcome prediction on an average basis. Here the random phenomenon arises from our
unwillingness to carry out the exact and full analysis because it is impractical to amass all the
conditions precisely or not worth the effort.
We shall begin with a review of the basic concepts of the theory of probability, which
forms the basis for describing random processes.
8.1 CONCEPT OF PROBABILITY
To begin the discussion of probability, we must define some basic elements and important terms.
The term experiment is used in probability theory to describe a process whose outcome cannot
be fully predicted because the conditions under which it is performed cannot be predetermined
with sufficient accuracy and completeness. Tossing a coin, rolling a die, and drawing a card
393