DARPA ULTRALOG Final Report - Industrial and Manufacturing ...

More documents

Recommendations

Info

Coordinating Control Decisions of Software Agents for Adaptation to Dynamic Environments Y. Hong 1 , S. R. T. Kumara 1 1 Harold and Inge Marcus Department of Industrial and Manufacturing Engineering The Pennsylvania State University, University Park, PA, 16802, USA Abstract We suggest a design for an infrastructure-level load control mechanism of a multiagent system, Cougaar. The purpose of control is to strengthen the robustness of a software multiagent system with respect to load balancing such that the system can keep working without disastrous performance degradation even under occasional harsh running environments. Resource control in multiagent systems is carried out mainly by agent’s self-control, which makes the control problem very difficult. We suggest a hierarchical control structure in order to reduce complexity of control while inducing coherent movement of agents. Keywords: load balancing, hierarchical control, multi-agent system 1 INTRODUCTION Multiagent systems have significant advantages in the development of complex distributed software system [1]. Agents are naturally matched to components in complex systems. Therefore, complicated interactions among the subcomponents can be represented by agent interactions. Due to the modularity and autonomy of agents, the application could be composed by assembling the agents. The multiagent systems are flexible in design. Partial changes in the system could be localized for a few agents without affecting the rest of the system. Thus, constructing or altering a large software system could become easier with agent technology. In addition to the advantages in designing and constructing a large system, robustness is also an important factor for a multiagent system to be a good software construction technology. Robustness of a software represents “the ability of software to react appropriately to abnormal circumstances” [2]. Like many biological or man-made systems, through feedback controls and redundancy of components (agents), the software system can also cope with uncertainties in dynamic environments and improve its robustness at the expense of increasing complexity [3][4]. The time varying computational load could be one of threats to robustness. A sudden excessive workload could degrade performance to an extent in which the system cannot meet minimum requirements on response time. This is in specific very critical for real time applications. Because agent systems are distributed and decentralized, it is hard to build a control mechanism by which agents can adapt to the changing environments effectively and coherently. In order to resolve this problem, we suggest an infrastructure-level load control mechanism for a multiagent system, Cougaar. The reason we consider infrastructure level control mechanism is that the application developers’ efforts to secure robustness of software with respect to the load control could be much reduced. Multiagent systems such as Cougaar [5] and Jade [6] provide many infrastructure level services, which save the application developers efforts required to build basic functions of the multiagent system. Load control function can be included in the infrastructure and its necessity has been emphasized [7]. Infrastructure can hide the complexity of controlling resource allocation such that application developers tune the performance using highlevel abstract parameters for load control. 2. LOAD BALANCING IN MULTIAGENT SYSTEMS In multiagent systems, system functions are decomposed into software agents. Agents carry out system functions by exchanging services with each other [7]. Agents have their own work and specialize in a specific service. Agents request some service from another agent who is specialized in that service. Providing the service requires the use of some computational resource such as CPU time. Agents are distributed on multiple machines, which are connected through communication networks. More than one agent can be on a machine and share the CPU time. The frequency of service request of each agent is time varying depending on real world, which the application deals with. Considerable research has been done on dynamic load balancing for computer clustering. However, we cannot apply this directly to a multiagent system [7]. As noted by chow and kwok [7], multiagent systems (MAS) are different from computer clustering with respect to load balancing. Firstly, in MAS, agents are continuously running while in computer clustering, jobs submitted by users are killed after completion. Secondly, communications between agents in multiagent system are highly variable, whereas, communications between jobs usually has static patterns. Another difference, which is not pointed out by chow and kwok, is that agents could proactively manage their workload.
Load balancing issues have not been paid much attention in MAS studies [7][8]. There are few papers in multiagent load balancing. Schaerf et. al. [9] studies how an agent can adapt to the environment. They separated the resources from the agents. In their model, agents assign their jobs to these resources. Using reinforcement learning, they showed that agents could adapt to each other under fixed or even for dynamical loads. Chow and kwok [7] devise an agent reallocation algorithm, called ‘Comet’ algorithm, which select agents to be moved to other machines. Agents are distributed on multiple machines. The comet algorithm chooses agents based on credits, which are continuously evaluated for each agent. The agent with low credit will be moved. The credit will decrease as the agent’s workload increases or the agent has more communication with other agents on other machines. We consider a similar agent system environment with chow and kwok. However, we added a feature of agent’s selfregulation on the workload. 3 QUEUEING MODEL FOR WORKLOAD DYNAMICS We conjecture that workload dynamics could be modeled as a queueing system. A service request from outside or other agents could be seen as a customer in a queueing system. While a request is being served, the later incoming service requests will wait in the queue. We consider a situation in which agents have multiple alternative algorithms to provide their service. Those algorithms trade off between computation time and quality of solution. Thus, depending on the workload in the queue, an agent can choose an optimal algorithm to improve the overall performance measure. This is similar to anytime algorithm composition [10]. In anytime algorithm, we have to determine time duration in which the algorithm solves the problem. Here, we assume that problem solving time is not predetermined. Instead, it is a statistical characteristic of the algorithm. From the queueing model perspective, this could be seen as a service rate control problem [11]. Multiagent system infrastructure can have a facility, where each machine works as a server by assigning computational resources (run right) to the agents for CPU time-sharing. We call the server as a node. This could be seen as a polling model, which has been used to model for time-sharing in a computer operating system or link sharing in a communication network. The node could give priority to a certain agent by visiting the agent more frequently. The node could monitor the amount of workload or arrival rate of service requests through agents. Based on the detected changes, it can change the priority of the agent. Imbalance among machines can be controlled by reallocation of agents from a high loaded machine to a low loaded machine for better performance. However, in this paper, we considered only agent and machine level control. 4 DECENTRALIZED CONTROL In view of the above mentioned workload dynamics, load control in multiagent systems could be seen as a decentralized stochastic control problem [12]. The decentralized control system consists of multiple control posts. They locally sense and control some part of the system they take charge of. However, their controls influence the system dynamics collectively. Thus, in order to operate the system optimally, decisions taken by controllers should be compatible and coherent. The information sharing between agents is treated as the main issue. In order for the controllers to obtain global optimal control decisions, exchange of all the local information is inevitable. Each controller makes a decision by solving a larger problem in which other agents’ movement is considered. However, it is unrealistic in the case of multiagent systems because of the long communication time. Finding optimal controls might be very difficult due to the size of the problem. It is difficult or almost impossible to find a purely decentralized optimal control policy for a multiagent system in this way. There are few systems in which locally made decisions could be globally compatible [13]. However this is very limited to some specific problem only. Thus, we need a control structure in which each control component (for example, an agent) takes control decisions by communicating only with closely connected components, while well coordinated decisions could be generated. In this paper, we suggest a hierarchical control structure which aims at achieving the above mentioned expectations. 5 HIERARCHICAL CONTROL 5.1 General Description In order to manage the complexity of large-scale problems, hierarchical control approach have been studied in various areas [14][15]. For multiagent systems, hierarchical control has been adopted as an intermediate form between centralized and decentralized control as a tradeoff between the advantages of the two approaches [14]. Hierarchical control can reduce the computation of gathering information and finding an optimal control than centralized control. On the other hand, it has better coordination capabilities than decentralized control. We consider three levels of hierarchy – the entire system, nodes and agents. There are usually multiple subcomponents under a higher-level controller i.e. there are multiple agents under a node and multiple nodes under a top controller. Agents and a node controller have direct communication connection by which they share information. The information sharing is restricted between the components, which are connected in the hierarchy. In this case, an agent reports its workload and performance (see 5.2) and the node controller announces control (the visit order and frequency) or state information such as estimates about environment parameters, which could be more effectively observed by the node rather than agents. We could also consider similar information exchanges between nodes and the top controller. Here, nodes report their node level workload trend to the top controller. On the other hand, the top controller could inform the system level environment parameters and order to move agents from one node to another. We assume the control frequency to be different at different levels. It is higher in lower levels compared to the higher levels. Control decisions on service rate are more frequent than the changes on the CPU time assignment policy in the node level. For a given arrival rate and configuration, CPU time assignment policy in node level will not be changed until the arrival rate and configuration change. At higher level, the frequency of events is less and the time intervals between events become longer. This could be said to have multi-time scale depending on the level [16]. The difference of our problem from other multitime scale problems is that there are multiple components in the lower levels. In addition, it is reasonable to assume that the environment does not change very frequently, in such a manner that the system may not be able to estimate environment parameters and control it. A Higher-level controller has coarser information than the lower level subsystems. A higher-level controller could have better global information over its territory because it collects information from its subcomponents. However, it will not use the information gathered as a system state
Page 1 and 2:
Ultra*Log PSU/IAI Final Report for
Page 3 and 4:
Contents Contents .................
Page 5:
Executive Summary Ultra*Log is a De
Page 8 and 9:
2.3 Gnanasambandam, N., Lee, S., Ku
Page 10 and 11:
6 Characterization and analysis of
Page 12 and 13:
timizing simultaneously the link de
Page 14 and 15:
where ∆ 1 (j) ≥ 0 and ∆ 2 (i,
Page 16 and 17:
Table 2. GA Results Agent N A D max
Page 18 and 19:
connected component, in which a pat
Page 20 and 21:
Random Small-world Scale-free with
Page 22 and 23:
Growth mechanisms Start with a smal
Page 24 and 25:
Table 2. The proposed network’s c
Page 26 and 27:
¡ ¡ ¢ £ ¤ ¥ ¦ £ § ¨ © ¥
Page 28 and 29:
© © ¤ ¢ ¨ ¤ £ ¦ ¨ © §
Page 30 and 31:
DMAS Controller. The functional uni
Page 32 and 33:
1000 500 0 -500 -1000 0.2 0.4 0.6 0
Page 34 and 35:
tems. Proceedings of the Second Joi
Page 36 and 37:
Proceedings of the 1st Open Cougaar
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
1 SITUATION IDENTIFICATION USING DY
Page 46 and 47:
3 3.2 Behavior In SSC society an ag
Page 48 and 49:
5 All the behavior parameters may n
Page 50 and 51:
Estimating Global Stress Environmen
Page 52 and 53:
chaotic deterministic time series.
Page 54 and 55:
100% 1 95% 2 14 15 64% TAO 4 64% 62
Page 56 and 57:
interactions as a dynamical system
Page 58 and 59:
ehavior states under varying system
Page 60 and 61:
Extensive testing and validation of
Page 62 and 63:
5 Conclusions and Future Research T
Page 64 and 65:
2 to function critically even under
Page 66 and 67:
4 points into a corresponding inner
Page 68 and 69:
6 stresses well. The Warehouse 1 ag
Page 70 and 71:
© £ ¨ ¥ ¤ § ¢ ¥ ©
Page 72 and 73:
Table 1: Notation Symbol Descriptio
Page 74 and 75:
4 Conclusions and Future Work 4.1 C
Page 76 and 77:
O, U Stresses Physical/Infrastructu
Page 78 and 79:
Manuscript for IEEE Transactions on
Page 80 and 81:
Page 82 and 83:
Page 84 and 85:
Page 86 and 87:
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
Page 94 and 95:
Page 96 and 97:
Page 98 and 99:
Page 100 and 101:
Page 102 and 103:
Page 104 and 105:
Page 106 and 107: 2 discuss problem domain and in Sec
Page 108 and 109: 4 t Si t ∫ + ( ) i ) t When RA i
Page 110 and 111: 6 S UB i ∑ = P LI / LI . (16) i p
Page 112 and 113: 8 policies while underutilizing in
Page 114 and 115: architecture based on both the Grid
Page 116 and 117: to machines (network topology) and
Page 118 and 119: component is a member of one of the
Page 120 and 121: denotes the immediate predecessors
Page 122 and 123: limit of large number of tasks. If
Page 124 and 125: Min-min heuristic algorithm Step 1:
Page 126 and 127: We set up eight different experimen
Page 128 and 129: 0. 4 8057 8237 0.5 6439 6635 0.6 53
Page 130 and 131: pp. 191-200. [3] I. Foster, C. Kess
Page 132 and 133: Technology, Cambridge, MA, 1995. [2
Page 134 and 135: Manuscript for IEEE TRANSACTIONS ON
Page 158 and 159: directly. It has coarser scale. It
Page 160 and 161: Understanding Agent Societies Using
Page 162 and 163: • For tasks, the UID of the direc
Page 164 and 165: for tracking the dependencies betwe
Page 166 and 167: within the monitored enclave. With
Page 168 and 169: Figure 1. Agent Hierarchy in CPE So
Page 170 and 171: Figure 3. The World Model CPY Agent
Page 172 and 173: Table 2. TechSpecs: Infrastructure
Page 174 and 175: terms of the average waiting times
Page 176 and 177: Acknowledgements The work described
Page 178 and 179: key realization to tackle this prob
Page 180 and 181: are or ignored perturbations. The e
Page 182 and 183: sustained oscillations. If the inst
Page 184 and 185: solved by biological systems for li
Page 186 and 187: non-linear and non-stationarity. Ne
Page 188 and 189: D k : Outflow from high priority qu
Page 190 and 191: the system starts to perform self-s
Page 192 and 193: sustained in a supply chain. Resour
Page 194 and 195: which are necessary to make good mo
Page 196 and 197: focusing on the computational compl
Page 198 and 199: line of research that deals with ne
Page 200 and 201: ealizable. But the inherent complex
Page 202 and 203: Min, H. and Zhou, G., 2002, Supply
Page 204 and 205: In the rest of this paper we report
Page 206:
shows relatively irregular behavior
show all

DARPA ULTRALOG Final Report - Industrial and Manufacturing ...

Create successful ePaper yourself

Delete template?

Save as template?