ACO - Undergraduate

Ant Colony OptimisationAlexander Mathews, Angeline Honggowarsito & Perry Brown1Image Source: http://baynature.org/articles/the-ants-go-marching-one-by-one/

Contents●●●Introduction to Ant Colony Optimisation●●Natural InspirationAnt SystemAdvanced Ant Colony Optimisation●●●Extensions to Ant SystemMax-Min Ant SystemBest-Worst Ant SystemApplications of Ant Colony Optimisation●●The Generalised Assignment Problem (Static)Packet-switched network Routing (Dynamic)2

ACO:Introduction● Originally proposed by Marco Dorigo in 1992● To solve the Travelling Salesman Problem (TSP)● By simulation the movements of ants● The original ACO was not competitive with specialisedTSP algorithms● Many improvements have been proposed● ACO can be extended to solve generic optimisationproblems● Most applications are to routing and schedulingproblems3

Natural InspirationModel the behaviour of ants searchingfor food●●●●Model based on experimental observationsAnts mark a path to food using pheromoneOther ants can detect the pheromone trailAnts tend to follow paths with morepheromoneThis is location based communicationAn ant trail.Source: Wikipedia, Ants,Retrieved 12/9/124

Natural Inspiration Example●Ants leave the nest atregular intervals●Ants which find food return tothe nest by a random path●Ants leaving the nest tend totake the top path●●50% take the top path50% take the lower path●●Ants taking the top pathreturn firstThe top path starts toaccumulate pheromone at A●Has more pheromoneGraphics adapted from: Dorigo et al. 20065

Towards an AlgorithmWant to solve different problems those solved by real ants● Intermediate nodes: are components of a solution● Edges: are possible connections between components● Food Sources: are a complete solutions (with a fitness)Virtual ants are allowed:● Perfect memory: ants can exactly retrace their steps● Variable pheromone strength: strength related to quality ofsolution found6

Towards an AlgorithmTypical example Travelling Salesman Problem (TSP)(Find the shortest cycle that visits every city once and only once)●●●Intermediate nodes: are citiesEdges: are paths between citiesFood: A completed cycle of the graphNote: that nodes are cities not partially constructed tours.7

Ant System (AS)We have:●●●●A population of ants (starting at random cities)Ants which construct one solution per iterationA simple heuristic giving the 'fitness' of an edgeOnly pheromone information kept to the nextiteration8

Ant System: OverviewWhile not done:randomise ant positionsFor each ant: Construct a random tourFor each edge: Evaporate pheromoneFor each ant: Lay pheromone on each edge in tour9

Ant System: Overview●Initialise thepheromone for everyedge●Construct a randomtour●●Evaporate pheromonefrom every edgeIncrease thepheromone on edges inthe random tour10

Ant System: OverviewAfter a number ofiterations we get theoptimal tour.Hopefully.11

Ant System: Ant MovementUntil a solution is completed:For each edge to a node not in the tour:Calculate heuristic for edgeCheck pheromone strength for edgeCalculate attractiveness of edgeRandomly select edge with probability based on attractivenessAttractiveness is calculated as:(Pheromone) α ∗( Heuristic) β(need normalise across outgoing edges to get a probability)12

ParametersN : Number of antsα : Influence of the pheromone trailβ : Influence of the heuristicρ : Fraction of pheromone that evaporates●Low evaporation leads to slower convergence & possiblybetter solutionsQ : scales the strength of the pheromone laidPheromone Strength=QLength of Tour(Pheromone) α ∗( Heuristic) β14

Termination Conditions●●●Iteration/Time limitSolution is good enoughStagnation: Only one tour will be tried as thepheromone is too strong.Or: run forever to maintain a good solution to adynamic problem.15

Contents●●●Introduction to Ant Colony Optimisation●●Natural InspirationAnt SystemAdvanced Ant Colony Optimisation●●●Extensions to Ant SystemMax-Min Ant SystemBest-Worst Ant SystemApplications of Ant Colony Optimisation●●The Generalised Assignment Problem (Static)Packet-switched network Routing (Dynamic)16

Elitist Ant System• First improvement on Ant System• Also called Elitist Strategy for Ant System• Provide strong additional reinforcement tothe arcs belonging to the best tour found• Pheromone update rule17

Elitist Ant System• Appropriate value for parameter e allowsEAS to:• Find better tours• Find them in a lower number of iterations• Evaporation rule is implemented as in AntSystem18

Rank-Based Ant System• Introduced by Bullnheimer, Hartl and Straussin 1997• Ranking of pheromone update• The m ants are ascending ranked based onsolutions’ quality . (S’1, S’2, ....... , S’m)• Each ants deposit an amount of pheromonethat decrease with its rank• Experimental evaluation suggests that thissystem performs slightly better than EAS andsignificantly better than AS.19

Ant Colony System• Another extensions of Ant System• Three major improvements• Different transition rule, pseudo-randomproportional rule• Probability Distribution20

ACS - Improvement• Daemon triggers pheromone update• Ant with global best solution, Sglobal-best• Pheromone evaporation before update• Online step-by-step pheromone trail update• Rule:21

Max-Min Ant System (MMAS)• Developed by Stuzzle and Hoos in 1996• AS extension• Similar to ACS, offline pheromone trail update• Best Ant chosen to add pheromone• Iteration-best ant or best-so-far ant• Improve ant’s solutions using local optimisers22

MMAS - Extensions• Pheromone trails range [ Tmin,Tmax ]• Tmax :• Tmin : constant factor lower thanTmax• Pheromone trails are initialised tomaximum allowed value• Re-initialization of pheromone trails23

Best-Worst Ant System• Proposed by Cordon et al. in 1999• Incorporates Evolutionary Computation• Transition rule and pheromone evaporation• Similar with MMAS, use local optimisers toimprove ant’s solutions24

Best-Worst Ant System• Improvements:• The best-worst pheromone trailupdate rule.• Penalise all connection of theworst solution. Scurrent-worse• Update rule:25

BWAS Improvement• Pheromone trail mutation for diversity in searchprocess.• Pheromone is mutated with probability Pm.• Adding or substracting the same amount in eachiteration• mut(it,Tthreshold) depend on the average ofpheromone trails• Re-initialization of the pheromone trail, setting everypheromone to T026

Contents●●●Introduction to Ant Colony Optimisation●●Natural InspirationAnt SystemAdvanced Ant Colony Optimisation●●●Extensions to Ant SystemMax-Min Ant SystemBest-Worst Ant SystemApplications of Ant Colony Optimisation●●The Generalised Assignment Problem (Static)Packet-switched network Routing (Dynamic)27

Further applications of ACO• Two major types of problem that ACO is particularly suited to• (Static) NP-hard problems, including:– Routing problems (e.g. TSP)– Assignment problems (e.g. university timetabling)– Scheduling problems (e.g. scheduling project tasks)– Subset problems (e.g. multiple knapsack problem)• Dynamic shortest path problems– e.g. routing in packet-switched networks• N.B.: The terms static and dynamic here refer to whether ornot the problem instance changes at run-time28

Defining the pheromone trails• Key consideration for applications: decide what the pheromonetrails should signify• Relative positioning– Trails represent the desirability of choosing a particular solution component jdirectly after some other component i– Intuitively suited to problems for which the position of components in theoverall sequence is inconsequential (e.g. in TSP, no preference as to whichcities get visited sooner or later)• Absolute positioning– Trails represent the desirability of choosing a particular solution component jat the ith position in the sequence– Intuitively suited to problems for which the position of components in theoverall sequence is significant (e.g. in scheduling project tasks, different tasksare due by different dates)• Either approach is possible for any problem, but the intuitivechoices described above generally give much better performance29

Example application: GeneralisedAssignment Problem (GAP)• Example of a static NP-hard problem• A number of tasks to be distributed among a number ofagents, each with a limited capacity• Assigning a particular task to a particular agentinvolves using a certain amount of that agent’scapacity• Each task-to-agent assignment also incurs a specificcost• Goal is to minimise the total cost of assigning all taskswithin the agents’ capacity constraints30

Representing the problem graphically• Both tasks and agents represented by nodes• Ants must walk from task to agent to signifyassignment, then back to next task until all tasks havebeen visited and assignedTasksAgents31

Representing the problem graphically• Both tasks and agents represented by nodes• Ants must walk from task to agent to signifyassignment, then back to next task until all tasks havebeen visited and assignedTasksAgentsAssignment32

Representing the problem graphically• Both tasks and agents represented by nodes• Ants must walk from task to agent to signifyassignment, then back to next task until all tasks havebeen visited and assignedTasksAgentsAssignmentMove on to nexttask33

Representing the problem graphically• Both tasks and agents represented by nodes• Ants must walk from task to agent to signifyassignment, then back to next task until all tasks havebeen visited and assignedTasksAgentsAssignmentMove on to nexttask34

Aspects of the problem representation• Two types of pheromone deposits– One to determine which agent to assign a particular task to(relative)– Other to determine best order in which to assign tasks (absolute)• Heuristic information (static)– Our pre-existing knowledge of this particular problem suggests thatassignments with low cost and/or capacity usage should bepreferred• Range of feasible solutions limited by capacity constraints– Strategy: assign a task to an agent with sufficient remainingcapacity whenever possible– Ants that produce an infeasible solution (i.e. one that violatescapacity constraints) may have their pheromone contributionreduced35

Example application: packet-switchednetwork routing (dynamic)• Problem involves a network with links connecting pairsof nodes, reaching one node from another may involvetraversing a number of links in sequence• Each link has a cost associated with using it• Goal to find minimum cost path between every pair ofnodes• Normally an easy problem, except that in this instancethe link costs are subject to change over time• ACO compared with several state-of-the-art algorithmsfor this application, performed superior to all of them36

Aspects of the problem representation• Two types of pheromone deposits– One to determine which agent to assign a particular task to(relative)– Other to determine best order in which to assign tasks (absolute)• Heuristic information (static)– Our pre-existing knowledge of this particular problem suggests thatassignments with low cost and/or capacity usage should bepreferred• Range of feasible solutions limited by capacity constraints– Strategy: assign a task to an agent with sufficient remainingcapacity whenever possible– Ants that produce an infeasible solution (i.e. one that violatescapacity constraints) may have their pheromone contributionreduced37

References• M. Dorigo and T. Stutzle, "The ant colony optimizationmetaheuristic: algorithms, applications, and advances," inHandbook of Metaheuristics, Eds. F. Glover and G. Kochenberger,New York: Springer, 2003, pp. 250-285.• M. Dorigo et al., "Ant colony optimization," IEEE Comput. Intell.Mag, vol. 1, (4), pp. 28-39, Nov. 2006.• G. Di Caro and M. Dorigo, "AntNet: distributed stigmergetic controlfor communications networks," J. Artif. Int. Res., vol. 9, (1), pp.317-365, Aug. 1998.38

References• B. Bullnheimer, R. F. Hartl and C. Strauss, ANew Rank Based Version of the Ant System:A Computational Study. Central EuropeanJournal for Operations Research andEconomics, 7(1):25-38, 1999.• O. Cordon, F. Herrera, and T. Stützle, AReview on the Ant Colony OptimizationMetaheuristic: Basis, Models and NewTrends. Mathware and Soft Computing, 9(2-3), pp. 141--175, 2002h39

References• M.Dorigo and L. M. Gambardella. AntColony System: A cooperative learningapproach to the travelling salesman problem.IEEE Transaction on EvolutionaryComputation, 1(1):53-66, 1997.• T. Stützle and H. H. Hoos, MAX-MIN AntSystem. Future Generation ComputerSystems40

References• O.Cordon, I. Fernandez de Viana and F. Herrera andL. Moreno. A new ACO model integrationevolutionary computation concepts: The best-worstAnt System. In M. Dorigo, M. Middendorf, andT.Stutzle, editors, Abstract proceedings ofANTS2000 - From Ant Colonies to Artificial Ants: ASeries of International Workshops on Ant Algorithms,pages 22 - 29, IRIDIA, Universite Libre de Bruxelles,Belgium, 2000.41