DARPA ULTRALOG Final Report - Industrial and Manufacturing ...

focusing on the computational complexity of individual orbits. By not requiring a Hamiltonian,
computational mechanics can be applied in a wide range of contexts, including those where an
energy function for the system may not manifest like for the supply chains. Notions of
Complexity, Emergence and Self-Organization have also been formalized and quantified in terms
of various information measures (Shalizi 2000).
Given a time series, the (unknowable) exact states of an observed system are translated into
sequence of symbols via a measurement channel (Crutchfield 1992). Two histories (i.e., two
series of past data) carry equivalent information if they lead to the same (conditional) probability
distribution in the future (i.e., if it makes no difference whether one or the other data-series is
observed). Under these circumstances, i.e., the effects of the two series being indistinguishable,
they can be lumped together. This procedure identifies causal states, and also identifies the
structure of connections or succession in causal states, and creates what is known as an “epsilonmachine”.
The ε -machines form a special class of Deterministic Finite State Automata (DFSA)
with transitions labeled with conditional probabilities and hence can also be viewed as Markov
chains. However, finite-memory machines likeε -machines may fail to admit a finite size model
implying that the number of casual states could turn out to be infinite. In this case, a more
powerful model than DFSA needs to be used. One proceeds by trying to use the next most
powerful model in the hierarchy of machines known as the casual hierarchy (Crutchfield 1994),
in analogy with the Chomsky hierarchy of formal languages. While “ε -machine reconstruction”
refers to the process of constructing the machine given an assumed model class, “hierarchical
machine reconstruction” describes a process of innovation to create a new model class. It detects
regularities in a series of increasingly accurate models. The inductive jump to a higher
computational level occurs by taking those regularities as the new representation.
ε -machines reflect a balanced utilization of deterministic and random information processing
and this is discovered automatically during ε -machine reconstruction. These machines are
unique and optimal in the sense that they have maximal predictive power and minimum model
size (hence satisfy Principle of Occam Razor i.e. causes should not be multiplied beyond
necessity). ε -machine provides a minimal description of the pattern or regularities in a system in
the sense that the pattern is the algebraic structure determined by the causal states and their
transitions. ε -machines are also minimally stochastic. Hence computational mechanics acts as a
method for automatic pattern discovery.
ε -machine is the organization of the process, or at least of the part of it which is relevant to
our measurements. The ε -machine being a model of the observed time series from a system can
be used to define and calculate macroscopic or global properties that reflect the characteristic
average information processing capabilities of the system. Some of these include Entropy rate,
Excess entropy and Statistical Complexity (Feldman and Crutchfield 1998) and (Crutchfield and
Feldman 2001). The entropy density indicates how predictable the system is. Excess entropy on
other hand provides a measure of the apparent memory stored in a spatial configuration and
represents how hard it is the prediction.ε -machine reconstruction leads to a natural measure of
the statistical complexity of a process, namely the amount of information needed to specify the
state of the ε -machine i.e. the Shannon Entropy. Statistical Complexity is distinct and dual from
information theoretic entropies and dimension (Crutchfield and Young 1989). The existence of
chaos shows that there is rich variety of unpredictability that spans the two extremes: periodic and
random behavior. This behavior between two extremes while of intermediate information content
is more complex in that the most concise description (modeling) is an amalgam of regular and
stochastic processes. Information theoretic description of this spectrum in terms of dynamical
entropies measures raw diversity of temporal patterns. The dynamical entropies however do not
measure directly the computational effort required in modeling the complex behavior, which is
what statistical complexity captures.