Architecture of Computing Systems (Lecture Notes in Computer ...

54 J. Zeppenfeld and A. Herkersdorf
migrating tasks. In the current implementation, the task to be migrated is chosen randomly,
as no monitoring is done to determine the workload of individual tasks, and
therefore all tasks appear identical to the AE. The task is migrated to the CPU with
the lowest workload. This targeting method requires no additional exchange of information,
since workload information is already shared over the AE interconnect to
allow calculation of the system’s average workload. For simulations in which no
global information is available, the target CPU is chosen randomly.
2.3.3 Evaluator
Choosing an appropriate action to be performed based on the incoming monitor
signals is the responsibility of a learning classifier table (LCT) evaluator. LCT is a
reinforcement based machine learning technique specifically designed for efficient
integration in a HW system. LCT can be considered a “light-weight” XCS [9], the
state of the art SW method for reinforcement learning classifier systems. LCT has
previously been successfully applied for SoC robustness optimization under intermittent
IP component failures [8]. In the following we will describe the rules and objective
function used by the LCT for the network processor system presented in this
paper.
Before a completed autonomic system can be deployed, an initial set of rules must
be created and loaded into the system’s LCTs. This initial rule set should cover the
full input range of the utilization, frequency and relative workload monitors described
above, with each rule proposing either a task migration or an increase or decrease in
frequency. Overlapping rules are expressly permitted, allowing the LCT to choose
which rule to apply. By keeping track of how well a certain rule performs, and by
updating the fitness of that rule accordingly, the LCT learns to use the rule that has
previously shown the best results in a certain situation.
Although the LCT can learn which rules are detrimental and can thereby avoid using
them, it is preferable if the initial rule set has already undergone a filtering process
to maximize the effectiveness of a newly deployed system. Figure 3 shows the general
method that was used to generate the initial rule set for this paper. A non-critical
sample system is initialized with random rules, and subjected to various representative
traffic scenarios that might be expected in the field. When the system stabilizes,
indicating that beneficial rules can be distinguished from counterproductive ones by
means of their fitness, the current rule sets are extracted and stored. Thereafter, the
system is loaded with new random rules and the process is repeated. If the system
does not stabilize within a certain amount of time, the system is reset without storing
the rule tables.
After a sufficient number of successful runs, the rules of all resulting rule sets are
sorted by fitness, and a selection of the fittest rules is made for inclusion in the resulting
rule set. This step is currently performed manually, but will eventually be automated
with appropriate tools as part of our future work. The resulting rule set can then
be used to replace the random initial rules and repeat the process. After one or more
iterations of this process (just one was sufficient for generation of the initial rule set
used by the simulations in section 3), the resulting rule set is ready for use as the initial
rule set of the deployed system.
Determining the fitness of a rule, both during the initialization phase described
above and during regular MP-SoC operation, is accomplished through the use of an