Here - Agents Lab - University of Nottingham
Here - Agents Lab - University of Nottingham
Here - Agents Lab - University of Nottingham
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Evaluation <strong>of</strong> a Conversation ManagementToolkit for Multi Agent ProgrammingDavid Lillis, Rem W. Collier, and Howell R. JordanSchool <strong>of</strong> Computer Science and Informatics<strong>University</strong> College Dublin{david.lillis,rem.collier}@ucd.iehowell.jordan@lero.ieAbstract. The Agent Conversation Reasoning Engine (ACRE) is intendedto aid agent developers with the management <strong>of</strong> conversationsto improve the management and reliability <strong>of</strong> agent communication. Toevaluate its effectiveness, a problem was presented to two groups <strong>of</strong> undergraduatestudents, one <strong>of</strong> which was required to create a solutionusing the features <strong>of</strong> ACRE and one without.This paper describes the requirements that the evaluation scenario wasintended to meet and how these motivated the design <strong>of</strong> the problemthat was presented to the subjects. The solutions were analysed usinga combination <strong>of</strong> simple objective metrics and subjective analysis,which indicated a number <strong>of</strong> benefits <strong>of</strong> using ACRE. In particular, subjectiveanalysis suggested that ACRE by defaults prevents some commonproblems arising that would limit the reliability and extensibility <strong>of</strong>conversation-handling code.1 IntroductionThe Agent Conversation Reasoning Engine (ACRE) is a suite <strong>of</strong> components andtools to aid the developers <strong>of</strong> agent oriented s<strong>of</strong>tware systems to handle interagentcommunication in a more structured and reliable manner [1]. To date,ACRE has been integrated into the Agent Factory multi-agent framework [2]and is available for use with any <strong>of</strong> the agent programming languages supportedby Agent Factory’s Common Language Framework [3]. Related work is presentedin Section 2, followed by an overview <strong>of</strong> ACRE itself in Section 3.The principal focus <strong>of</strong> this paper is to describe an experiment that was conductedto evaluate the benefits that ACRE can provide in a communicationheavyMAS. Section 4 outlines the motivations underpinning the design <strong>of</strong> theexperiment and discusses how a scenario was designed with these in mind. Inparticular, we identified a number <strong>of</strong> variables that should be eliminated so asto ensure that comparisons <strong>of</strong> ACRE and non-ACRE code would be fair. Thisscenario was given to a class <strong>of</strong> undergraduate students, who were required todevelop agents that could interact with a number <strong>of</strong> provided agents in orderto trade virtual stocks and properties for pr<strong>of</strong>it. The subjects were divided intotwo groups: one using ACRE and one working without.85