Wireless Network Design: Optimization Models and Solution ...

276 Marina Aguado, Jasone Astorga, Nerea Toledo and Jon Matias
12.3 Validating Discrete Event Simulations: the Random
Number Seed and the Confidence Interval
A simulation process is a statistical experiment. In order to represent a behavior
that is not known for a specific element, the simulation environment makes use of
stochastic modeling with random number streams to introduce changes in the element’s
operating environment (i.e., its input). However, it is impossible for a computer
program, by its very nature, to generate truly random numbers. Thus, the simulation
framework relies on starting a pseudo-random number generator in different
states to exhibit unpredictable behavior. The initial state of the generator is known
as the random number seed. Each particular simulation run with a specific random
number seed value will produce a different behavior and yield new and different
results. To account for all statistical behavior that the system might exhibit, the simulation
model must be run multiple times with different seed values. As more seeds
are chosen, standard behavior should be observed more frequently than anomalous
behavior. Intuitively, the more simulation runs performed, the more confidence we
have in the results obtained from the study. In other words, the goal is to ensure
that a simulation system produces valid statistical behavior by making a sufficient
number of runs, each with a different random seed value. One of the important tasks
a simulation designer must carry out is that of deciding how many separately seeded
trials to run in a simulation experiment so that there will be enough data to make a
strong statement about the system’s typical behavior.
Suppose that a number of simulations of a system have been run with different
random number seeds to obtain n samples of the statistic X, (x1, x2, x3, ..., xi, ...,
xn). Statistic X may take on many values and its precise distribution is unknown. In
order to calculate, µ, the true mean of the random variable X, which represents the
typical behavior of the modeled system with regard to the statistic X, we should run
a large number of simulations (theoretically, an infinite number would be required).
However, since it is not usually possible to run a very large number of simulations
to determine µ, it is interesting to determine the degree of precision with which
the mean value, x, of an n-sample set approximates µ. This determines whether the
value can be used with confidence to make statements about the typical behavior of
the modeled system.
Consider an experiment consisting of a collection of samples of X, each of size
n, where x is the average value of the n samples corresponding to one trial of the
experiment. If the experiment is performed many times with different seed values,
then the resulting average value of x, which we denote by X, has its own distribution.
In accordance with the central limit theorem, regardless of X’s actual distribution,
as the number of samples grows large, the random variable X has a distribution that
approaches that of a normal random variable with mean µ, the same mean as the
random variable X itself. The theorem further states that if the true variance of X
is σ 2 , the variance for statistic X is
σ 2
n
. To conclude, if sufficient sample size n
has been chosen, normally n > 30, it is possible to work with a normal distribution
which has known properties instead of working with an unknown distribution.