POLITECNICO DI MILANO - DCSC

POLITECNICO DI MILANO 

Facoltà di Ingegneria Industriale 

Corso di Laurea Specialistica in Ingegneria Spaziale 

DISTRIBUTED AND ROBUST CONTROL FOR SPACE 

MULTI AGENT SYSTEMS 

Relatore: Prof. Michèle R. LAVAGNA 

Co-relatore: Prof. Paolo MAGGIORE 

Aprile 2008 

Anno Accademico 2006/07 

Tesi di Laurea di: 

Andrea SIMONETTO 

Matr. 680744

Cite as 

@MASTERSTHESIS{Simonetto_ms_08, 

AUTHOR = {Simonetto, Andrea}, 

c○2008 Andrea Simonetto 

Submitted March 26th, 2008 

TITLE = {{Distributed And Robust Control For Space Multi 

Agent Systems}}, 

} 

SCHOOL = {Department of Aerospace Engineering, Politecnico di Milano}, 

YEAR = {2008}, 

month = {April},

M.C. Escher (1898 - 1972), Moebius Strip II (Red Ants) 1963 

To who will have the patience to understand my words 

i

ABSTRACT 

Nowadays Multi Agent Systems are increasingly being studied in space applications, mainly 

for their reliability, robustness, fault tolerance and cost effectiveness. Although the research 

in this area started in Computer Science field at least twenty years ago, the challenges and 

open issues are still several, since the application scenario is completely different. This thesis 

main aim is, first, to understand the Space Multi Agent System state of the art, starting from 

the proposed mission by NASA and ESA, second, to study deeply two applicative scenarios, 

namely a formation flying mission and an asteroid exploration one. Different algorithms and 

formal techniques have been analyzed in order to built a fully distributed and robust controller 

for the system. The overall agent architecture embodies both a high level control and a low 

level one; the former uses a suitable extension of a potential field formulation, called Artificial 

Physics, which let the basic approach capable of dealing with multiple tasks and skills; on 

the other hand, the latter is a trajectory control and a communication network assurance 

one. The first is a non linear Lyapunov control, formulated in a H∞ framework, extending 

a previous approach to let it robust and reliable to perturbations and model uncertainties. 

The second is a dynamic potential field/token-based algorithm, which is a modification of 

standard potential field techniques and it increases the overall performances in terms of global 

connectivity and efficiency. Several simulation scenarios have been tested, both in formation 

flying contests, like Prisma ESA mission, and in asteroid belt environment, showing good 

results for reliability, robustness and wide application areas. 

Keywords: Multi Agent Systems, Distributed Control, Robust Control, Artificial Physics, 

Non Linear Lyapunov Control, Sensor Networks. 

iii

ACKNOWLEDGEMENTS 

A Master Thesis means a lot of mental effort, both to be able to focus on a particular and 

very specific topic for long time, and not to feel either disappointed or upset if something 

does not work as it should. Since the latter occurred very often I would like to thank all the 

people who have helped me during these long eight months, or maybe more. 

First of all, my advisor Michèle, who is a very busy person, but she made me understand 

how to work by myself and how to develop a scientific research. She was always kind and 

she gave me a lot of advices, supporting me in the hardest moments. Moreover she made the 

amazing experience to work abroad possible for me, thus I could not thank her a lot. Then, 

of course, the other professors who helped me developing my thesis, Paul and Katia, both 

US researchers. Paul was great, he kept pushing me to do better than my best and finally I 

managed to do something he liked. He is the person I would like to have as a friend and he 

is also a great researcher. Thanks Paul. Katia has a lot of experience, thus her advices were 

deep and very helpful for me to understand better how to formulate my problem. I remember 

the first meeting with them, they were so suspicious about me, but by the end, they were so 

glad to have worked with me. It is something to be proud of. Thanks. 

I want to thank my parents, who supported me in this adventure abroad, I imagine they 

missed me a lot, or at least I hope, thus thanks dad and mommy, I missed you too. I am 

very proud of them, they are very open minded people, I cannot ask for better ones. 

Then, of course, my friends. I am very pleased to have a lot of names on this list. And you, 

the reader, please don’t miss a name! First my friends from Carnegie Mellon and US: Kevin, 

the funniest German I have ever met, I am looking forward for other amazing pool matches; 

Giuseppe, very awesome times together, Philadelphia, the swimming pool, the canoeing, I 

hope to work with you someday; Joe, Pras, Robin, Sean, Jack I really like those guys. And 

then, more: Joe, Becca, Marie, Paul, Eric, Holly, Mike, Garret, Alex Sasha, I had really great 

time with you. Thanks for the birthday party and all the attentions you gave me, the moka, 

all the ride back to my place, all the fabulous dishes, the skiing stuff, Thanksgiving party 

and the superbowl party and even more. Finally, Andrea, a good guy who pretends to get 

his PhD next year, isn’t he fun? 

I want to mention all my great friends of ASP, another amazing thing I had the pleasure 

to do. Thanks Elena, who is very smart and cute, I really like her, even if I cannot remember 

v

vi 

the exact date of her birthday, sorry girl. Francesca, very long mails, very nice talks, she 

is very cool doing very amazing math stuff. I really like her as well. Then in some order, 

my forever roomy Lo, Gianma the easy-going guy, Francesco and Franz the normative boys, 

Martino Daniele and Marco from Chicago, Davide and Pietro who know how to make a 

wash machine work, Alessandro the physician, Stefania and Umberto the mathematicians, 

Giacomo, Giorgio, Andrea the open-door man, who met me a lot of time ago, in a certain 

competition. I have to tell you the truth, I let you win that time. 

How could I forget my friends all over the World? Like Paul in France, Thanos, Fani and 

Kostas in Greece, Tamara in Croatia, and even more. Roberta, Goran, Antonio, Ines, Bene, 

Nico and other funny guys and ladies. We had great times wherever we were. 

What about my Uni friends? Very pleased to have met them all. Mauro now at Toulouse, 

very easy man, I remember the days just at the end of July, nobody at school, only me and 

you. Who on Earth made us do that? Maffez, very smart man, I really like him, and he is 

very impressive in skiing, somehow. Paolo, very very nice, sorry for that girl man. Giorgio 

and Ricki and some testosterone moments. Alfu, the most particular one, he really thinks 

Armani could have a chance to win something. Sorry, but anyway I like you. Bart, now 

in German, who was a true surprise for me, you are so smart! Great. Castel, in ESA, the 

most easy-going person I have ever met, always drinking beer, always optimizing something, 

amazing. Luca, very deep person, now at Atlanta, the man who helped me with my mind 

most in those days at Pitt. I still cannot understand how a tiramisù could cost thirty bucks! 

Great NYC. And then all the other: Davide, thanks a lot for the – unfortunately not used 

– asteroid model; Fede, thanks for your support, your good words, your friendship; Monica, 

in France, very cool to work with you; Dano, very cool discussions, you are so funny; Fabri, 

Francesco, Nazi, Michele, Matteo, Marco, Alessandro, Roberto, Gabriele, Marcello, Luca, 

Giulio, Elisa, Fabio, Daniele, Luca, Marco, Michele, Alberto, Davide, Dario. I think there is 

somebody missing, but I hope they could forgive me. 

Then, of course my friend of adventures at university: Guido. I barely understand you, 

but I like you after all. I am very pleased to have met you, although I cannot agree on 

everything you say, and I have some difficulties to work with you. I am joking, of course. 

Thanks for your practical perspective on life and science, it served me a lot to understand 

better what was going on. Thanks for all the discussions, the ideas, the funny trips, the nice 

girls, thanks man, I hope you could do whatever you want. 

I am so pleased to have met you all, friends, you can make it as I had. I remember three 

things. The first, an economics book with a long long bibliography, the second, an Iranian 

who kept writing third order integrals, finally, some Fusion conference papers I had to read 

to understand the research at CMU in October. Well, I have a very long biblio, I could have 

had sixth order integrals, but I found a better solution and, by the end, I managed to submit 

my own paper to the Fusion conference. Everything is possible guys, just go out and do it. 

Finally, my closest friends. Vale, who helped me feeling at home also when I was overseas. 

Very funny moments together, the lake over all. Eleonora, from the coolest high school 

ever, thanks for the long discussions, your friendship and the basketball matches. You have

absolutely to come to my thesis party. Then, the three people I care most. My best friend, 

as a brother for me, Roberto, Bob, who is actually an architect. No one is perfect after all. 

Thanks for everything, really everything. Since your English is worse than mine, I put here 

just a ⋆, and we will discuss later on what I would have liked to write to thank you. Thanks 

thanks thanks a lot, your are a great friend, although you have a lot of defects. Then my 

sister, my true sister, Tiziana. I really love here, she is very cute, smart, carina e coccolosa. 

Thanks Tita, for everything, for just being you. I hope the thesis is intriguing enough for 

you, I really hope, because I do not want write it another time! 

Last on this list, You. I don’t know what is going to happen, but I want to thank You 

for all your mind, your Love, everything you gave me in such a short period of time. I will 

never forget how You made me feel. Thanks. 

Thanks to everybody, 

vii 

andrea =)

viii

CONTENTS 

ABSTRACT iii 

ACKNOWLEDGEMENTS v 

SUMMARY xiii 

1 INTRODUCTION 1 

1.1 Multi Agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

1.2 Proposed Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 

1.2.1 Prisma and Swarm missions . . . . . . . . . . . . . . . . . . . . . . . . 3 

1.2.2 Terrestrial Planet Finder Interferometer . . . . . . . . . . . . . . . . . 5 

1.2.3 ANTS project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 

1.2.4 APIES feasibility study . . . . . . . . . . . . . . . . . . . . . . . . . . 7 

2 FORMULATION OF THE PROBLEM 11 

2.1 Typical scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 

2.1.1 Formation flying in Prisma Mission . . . . . . . . . . . . . . . . . . . . 12 

2.1.2 Main asteroid belt exploration . . . . . . . . . . . . . . . . . . . . . . 12 

2.2 Agent architecture and assumptions . . . . . . . . . . . . . . . . . . . . . . . 15 

2.3 Multi agent techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 

2.3.1 Space multi agent system . . . . . . . . . . . . . . . . . . . . . . . . . 18 

2.3.2 Formal methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 

2.3.3 Artificial Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 

2.4 Work’s framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 

3 THE DECISIONAL LEVEL 23 

3.1 Artificial Physics Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 

3.1.1 AP state of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 

3.2 Extension of AP formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 

3.3 Formal methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 

3.4 Information propagation and knowledge bounded agents . . . . . . . . . . . . 28 

ix

x CONTENTS 

3.4.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 

3.4.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 

3.5 Distributed scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 

4 THE PHYSICAL PART 31 

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 

4.2 Possible solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 

4.2.1 Potential field approach . . . . . . . . . . . . . . . . . . . . . . . . . . 32 

4.2.2 SDRE approach for non linear systems . . . . . . . . . . . . . . . . . . 32 

4.2.3 Non linear Lyapunov control . . . . . . . . . . . . . . . . . . . . . . . 33 

4.3 Long period dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 

4.3.1 Robust non linear Lyapunov control . . . . . . . . . . . . . . . . . . . 36 

4.3.2 HJI equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 

4.3.3 R as a function of the state . . . . . . . . . . . . . . . . . . . . . . . . 38 

4.4 Short period dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 

4.5 Final Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 

5 THE COMMUNICATION PART 41 

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 

5.2 Artificial Physics and Token based algorithm . . . . . . . . . . . . . . . . . . 42 

5.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 

5.4 Positioning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 

5.4.1 Standard algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 

5.4.2 Dynamic Potential Fields . . . . . . . . . . . . . . . . . . . . . . . . . 46 

5.5 Potential Field and Motion Control . . . . . . . . . . . . . . . . . . . . . . . . 49 

6 PERTURBATION MODELS 51 

6.1 Formation Flying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 

6.1.1 Atmospheric drag effect . . . . . . . . . . . . . . . . . . . . . . . . . . 51 

6.1.2 Gravitational effects, J2 and J22 . . . . . . . . . . . . . . . . . . . . . 52 

6.2 Asteroid belt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 

7 RESULTS 57 

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 

7.1.1 Scalability proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 

7.2 Goal Manager example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 

7.3 Formation flying scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 

7.3.1 Unperturbed and perturbed results . . . . . . . . . . . . . . . . . . . . 60 

7.3.2 Montecarlo analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 

7.4 Communication network deployment . . . . . . . . . . . . . . . . . . . . . . . 66 

7.4.1 2D Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 

7.4.2 3D Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

CONTENTS xi 

7.5 Asteroid belt scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 

7.5.1 The physical part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 

7.5.2 The communication part . . . . . . . . . . . . . . . . . . . . . . . . . . 73 

8 FINAL REMARKS 77 

8.1 Thesis final remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 

8.2 Future developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 

8.2.1 Decisional Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 

8.2.2 Physical part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 

8.2.3 Communication part . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 

8.2.4 Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 

REFERENCES 79 

A SISTEMI MULTI AGENTE PER APPLICAZIONI SPAZIALI i 

A.1 Introduzione . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i 

A.1.1 Principali Contributi . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii 

A.2 Formulazione del Problema . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii 

A.3 Controllo di Alto Livello . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii 

A.4 Controllo Robusto della Traiettoria . . . . . . . . . . . . . . . . . . . . . . . . iii 

A.5 Architettura di Comunicazione . . . . . . . . . . . . . . . . . . . . . . . . . . iii 

A.6 Modelli per le Perturbazioni . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv 

A.7 Risultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv 

A.7.1 Controllo di alto livello . . . . . . . . . . . . . . . . . . . . . . . . . . iv 

A.7.2 Controllo della traiettoria . . . . . . . . . . . . . . . . . . . . . . . . . iv 

A.7.3 Dispiegamento della rete di comunicazione . . . . . . . . . . . . . . . . vii 

A.7.4 Cintura degli asteroidi . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 

A.8 Sviluppi Futuri . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

xii CONTENTS

SUMMARY 

This work deals with multi agent system and, since they are a quite new research topic in 

space area, the project is in first instance a collection of ideas, then a detailed analysis of two 

typical scenarios: formation flying and asteroid belt exploration missions. 

The focus is on understanding how these kind of systems could be applied in space envi- 

ronments, in particular how to assure reliability, verifiability and thus robustness. First the 

problem is outlined with some proposed mission in this direction, e.g. NASA ANTS mission 

to the asteroid belt, second the agent control is shown. Both the high level control, i.e. a 

distributed goal manager, and a low level control, an H∞ non linear control for the physical 

part, and a dynamical potential field-token based control for the communication architec- 

ture, are developed and tested, showing high reliability, wide area of applicability and good 

performances. 

Chapter division 

Chapter 1. Here the framework of the work is presented with some proposed mission in 

the multi agent systems contest, both from ESA and NASA. 

Chapter 2. The problem is formulated, outlining both the high level control and the low 

level control for each agent. The chosen approach for the goal manager is an Artificial 

Physics/ Potential Fields approach. 

Chapter 3. The high level control is developed extending the Artificial Physics formulation 

to more complex scenarios with multiple goals and capabilities. 

Chapter 4. The low level control for the physical part is examined and derived. The final 

control is a long dynamic - short dynamic control. For the long dynamic one a H∞ non linear 

Lyapunov control is used, and its robustness is proved. Then for the short dynamic control 

two solutions are proposed. 

Chapter 5. The communication network deployment control is developed here, using a 

dynamical potential fields approach, in which the communication among the agents is dictated 

xiii

xiv CONTENTS 

by tokens, which are information packets. 

Chapter 6. The perturbation models are briefly discussed here. 

Chapter 7. Main results are shown here. First, for goal manager, second for the physical 

part, third for the communication network deployment and finally for the whole control. 

Main contributions 

The original contributions in this thesis are several: 

� The Potential Field - Artificial Approach is firstly extended to the multiple goals and 

skills case, using matrices algebra (Chapter 3). 

� The non linear Lyapunov control is applied to a perturbed system, the spacecraft tra- 

jectory control, extending it using an H∞ control. The necessary and sufficient con- 

ditions for stability is then derived. This application resulted in a paper submitted to 

AIAA/AAS Conference 2008 (Chapter 4). 

� The Potential Fields approach is integrated in a token - based algorithm to assure min- 

imal communication and high value of connectivity. Then this is tested in environment 

with obstacles, which has never been done before. This results in a paper submitted to 

IEEE Fusion Conference 2008 (Chapter 5). 

Acknowledgments 

This Master Thesis has been developed partly in Italy, in Milan, within the Politecnico 

di Milano, somehow even in collaboration with Politecnico di Torino, but mostly in US at 

Carnegie Mellon University. I would like to thank everyone who helped me in these three 

top-level universities and who made this possible. 

Milano, March 2008

CHAPTER 

1 

INTRODUCTION 

[...] to boldly go where no one has gone before. 

Gene Roddenberry (1921 - 1991) 

The chapter is completely dedicated to understand how the new concepts of Multi Agent 

Systems and Distributed Systems enter in the space research. First of all, why they are studied 

and their advantages are discussed, then an overview of the current proposed missions is 

presented. 

1.1 Multi Agent System 

Space science always leads Man to think to the most challenging ideas to try to catch 

the beauty of the cosmos. It is not new that many technological breakthroughs come from 

space research. Moreover, the multidisciplinarity, the fusion of different types of knowledge, 

the never similar problems are only some of its several amazing pros. 

Nowadays, new interpretations of what a space system actually is are appearing in the 

scene, in particular the concepts of distributed architecture, formation of satellites and swarm 

of agents. The advantages behind approaches of this kind are several, at least many as the 

challenges involved in. First of all, the relative simplicity of the single agent – spacecraft, 

robot or satellite – in terms of manufacturing, testing, electronics. The smaller the agent is, 

the simpler the making and, moreover, the cheaper. Second, having in mind an architecture 

of many agents, the reliability is improved and this is a key feature for a space mission to be 

founded. The biggest communication satellites have to be tested in many ways to be reliable 

enough to fly, moreover they are less fault tolerant as a multi spacecraft system may be. It 

is then worth to mention that, in a vision like this, the idea of series productions could be 

thought even in space research. 

1

2 CHAPTER 1. INTRODUCTION 

Distributed architectures could be a reasonable, maybe the only one, option when the 

missions to be performed are particularly risky, as the exploration of asteroid belt, or they 

involve formation flying concepts, as a large interferometric telescope. 

In Table 1.1 the differences between the two types of approaches are shown. As it seems 

evident, the distributed system key disadvantages, or challenges to handle, are related to the 

design of the system as a whole. 

Single agent systems Multi agent systems 

Design Complex Simple for the single agent 

Software Could have several functions 

Communications Important 

Basic functions, but complex global 

verification 

Important for the whole system and 

critical between the agents 

Manufacturing Complex Simpler and possible in series 

Tests Several and Complex 

Reliability 

Fault tolerance 

Cost High 

Reasonable for the several test and 

internal redundancy 

Reasonable for internal redundancy, 

but critical 

Quite impossible for the system as a 

whole but simpler for single agents 

High for the intrinsic redundancy 

High for the intrinsic redundancy 

Lower and dependent on the num- 

ber of agents 

Table 1.1: Comparison between single agent and multi agent approaches 

The fact that the single agent itself could be very basic and simple hides the global complexity 

of the complete system. The communication architecture and the control design has to be 

faced, in particular the control has to be as simpler as possible, but it has to include some 

functions to handle the system as a whole. Moreover, the idea that the total architecture can 

not be tested on the ground is quite critical. 

The present work deals with the agent control to assure both robust task execution and 

communication network building. 

Space system design involves many research areas; in fact, the concept of distributed 

systems is not new in the literature, in particular in computer science, where problems like 

those are called usually M ulti Agent Systems, or MAS, and they have been studied since 

1980 [Woo02], [Syc98]. There are several novelties in trying to apply the computer science 

methods to space science: first of all the scenario, which is completely different. Typically, 

the MAS are robots which have to perform some scheduled tasks, as explore a region, rescue

1.2. PROPOSED MISSIONS 3 

people, play soccer. Thus the scenario is basically bi-dimensional and the relative distances 

are small in comparison to the involved dynamic. Furthermore the system is composed usually 

by relatively few agents, less than 100, and the communication is thought as complete and 

faster than the decisional process of the robots. These features are not those a space engineer 

could expect from a Space MAS, SMAS. The scenario is three dimensional, the distance 

could be large respect to the dynamic, thus the environment may be addressed as sparse, the 

agents would be more than 1000 and, in the end, the communication could not be complete, 

see Table 1.2. 

Dimensions 

Distances in compar- 

ison to dynamics 

MAS SMAS 

Typically 2, could be 3 in UAVs sce- 

narios 

Small 

3 

Large 

Number of agents 10 - 100 10 - 1000 

Communication Often complete Often incomplete 

Environment Dense Typically sparse 

Table 1.2: Comparison between MAS and SMAS 

Thus, the challenges are several: using the methods of computer science, try to model a 

SMAS and, eventually, change the tools in order to fit them to the different scenario. It is 

also possible, and highly recommended in some cases, to develop completely new algorithms. 

1.2 Proposed Missions 

This is the framework in which the most important space agencies are in. In particular, 

in the following sections an overview of the most promising missions of both NASA and ESA 

is provided. This list is not complete but it could give a good insight on what is going on in 

space research. 

1.2.1 Prisma and Swarm missions 

Prisma Mission 

Prisma mission [1] provides a technology demonstration mission for the in-flight validation 

of sensor technologies and guidance/navigation strategies for spacecraft formation flying and 

rendezvous. Prisma is originating from an initiative of the Swedish National Space Board 

(SNSB) and the Swedish Space Corporation (SSC) and provides a precursor mission for 

critical technologies related to advanced formation flying and In-Orbit-Servicing. Prisma 

mission launch window is fixed for the beginning of 2009.


Figure 1.1: The Prisma mission 

The Prisma test bed comprises the fully maneuverable micro-satellite Main as well as 

the smaller passive sub-satellite Target. The two spacecraft will be injected into a Sun- 

synchronous dusk-dawn orbit at an altitude of 700 km. 

The mission objectives of Prisma may be divided into the validation of sensor and actu- 

ator technology related to formation flying as well as the demonstration of experiments for 

formation flying and rendezvous. 

ESA’s magnetic field mission Swarm 

The objective of the Swarm mission [2], scheduled for 2010, is to provide the best ever survey 

of the geomagnetic field and its temporal evolution, and gain new insights into improving the 

knowledge of the Earth’s interior and climate. 

The Swarm concept consists of a constellation of three satellites in three different polar 

orbits between 400 and 550 km altitude. High-precision and high-resolution measurements 

of the strength and direction of the magnetic field will be provided by each satellite. In 

combination, they will provide the necessary observations that are required to model various 

sources of the geomagnetic field. GPS receivers, an accelerometer and an electric field instru- 

ment will provide supplementary information for studying the interaction of the magnetic 

field with other physical quantities describing the Earth system - for example, Swarm could 

provide independent data on ocean circulation. 

The multi-satellite Swarm mission will be able to take full advantage of a new generation 

of magnetometers enabling measurements to be taken over different regions of the Earth 

simultaneously. Swarm will also provide monitoring of the time-variability aspects of the 

geomagnetic field, this is a great improvement on the current method of extrapolation based 

on statistics and ground observations. The geomagnetic field models resulting from the 

Swarm mission will further our understanding of atmospheric processes related to climate 

and weather and will also have practical applications in many different areas, such as space


weather and radiation hazards. 

Figure 1.2: The Swarm mission concept 

1.2.2 Terrestrial Planet Finder Interferometer 

The Terrestrial Planet Finder Interferometer (TPF-I) mission, by NASA, will search for 

habitable worlds around nearby stars and look for indicators of the presence of life. Working 

with infrared wavelengths, TPF-I complements the search made by the Terrestrial Planet 

Finder Coronagraph (TPF-C) in visible wavelengths. This combination provides the strongest 

possible confirmation of the presence of indicators of habitable worlds, Figure 1.3. 

TPF-I is in the pre-formulation phase of its development. The observatory mission concept 

includes five formation flying spacecraft: four 4-meter-class mid-infrared telescopes and one 

combiner spacecraft to which the light from the four telescopes is relayed to be combined and 

detected. The observatory will be deployed beyond the Moon’s orbit for a mission life of 5 

to 10 years. 

New technologies are being developed to allow spectroscopic measurements of light from 

extrasolar planets, including: 

1. formation flying telescopes to work together as one extended observatory, providing 

unprecedented angular detail and sensitivity that no telescope on the ground could 

ever achieve; 

2. starlight-suppression technology so that light from a planet’s star will be dimmed by a 

factor of a million, making the planet’s light visible;


3. new cryogenic coolers, making it possible for a new generation of detectors to find 

Earth-like planets. 

In this context it is important to note that the formation flying concept is one of the main 

challenge of the project, [LLJB07], [3]. 

1.2.3 ANTS project 

Figure 1.3: The TPF interferometer 

The Autonomous Nano-Technology Swarm (ANTS) concept mission, by NASA, will involve 

the launch of a swarm of autonomous pico-class (approximately 1kg) spacecraft that will 

explore the asteroid belt for asteroids with certain scientific characteristics. Figure 1.4 gives 

an overview of the ANTS mission. In this mission, a transport ship, launched from Earth, 

will travel to a point in space where net gravitational forces on small objects (such as pico- 

class spacecraft) are negligible (a Lagrangian point). From this point, 1000 spacecraft, that 

have been manufactured en route from Earth, will be launched into the asteroid belt. This 

environment presents a large risk of destruction for large (traditional) spacecraft. Even with 

pico-class spacecraft, 60 to 70 percent of them are expected to be lost. Because of their small 

size, each spacecraft will carry just one specialized instrument for collecting a specific type 

of data from asteroids in the belt. 

To implement this mission, a heuristic approach is being considered, which provides for 

a social structure to the spacecraft that uses a hierarchical behavior analogous to colonies or 

swarms of insects, with some spacecraft directing others. Artificial intelligence technologies 

such as genetic algorithms, neural nets, fuzzy logic, and on-board planners are being investi- 

gated to assist the mission to maintain a high level of autonomy. Crucial to the mission will 

be the ability to modify its operations autonomously, to reflect the changing nature of the 

mission and the distance and low bandwidth communications back to Earth. Approximately 

80 percent of the spacecraft will be workers that will carry the specialized instruments (e.g., 

a magnetometer, x-ray, gamma-ray, visible/IR, neutral mass spectrometer) and will obtain 

specific types of data. Some will be coordinators (called rulers or leaders) that have rules that


decided the types of asteroids and data the mission is interested in and that will coordinate 

the efforts of the workers. The third type of spacecraft are messengers that will coordinate 

communication between rulers and workers, and communications with the mission control 

center on Earth. 

Figure 1.4: An overview of ANTS mission 

The swarm will form sub-swarms, each under the control of a ruler, which contains models 

of the types of science that are to be pursued. The ruler will coordinate workers, each of which 

uses its individual instrument to collect data on specific asteroids and feeds this information 

back to the ruler, who will determine which asteroids are worth examining further. If the 

data matches the profile of a type of asteroid that is of interest, an imaging spacecraft will be 

sent to the asteroid to ascertain the exact location and to create a rough model to be used 

by other spacecrafts for manœuvering around the asteroid. Other teams of spacecrafts will 

then coordinate to finish mapping the asteroid to form a complete model, [RHRT06], [4]. 

1.2.4 APIES feasibility study 

APIES (Asteroid Population Investigation & Exploration Swarm) is a mission developed by 

EADS Astrium in response to an European Space Agency (ESA) Call for Ideas for swarm 

missions, based on the utilization of a large number of spacecrafts working cooperatively 

to achieve the mission objectives. APIES is intended to be the first interplanetary swarm


mission, designed to explore the asteroid main belt. This is one the least known parts of the 

Solar System, yet holding vital information about its evolution and planet formation. APIES 

aims to characterize a statistically significant sample of asteroids, exploring the main belt in 

great detail, measuring mass and density and imaging over 100 of these objects, at a stroke 

more than doubling the number of Solar System bodies visited by man-made spacecraft. 

Using the latest advances in system miniaturization, propulsion, onboard autonomy and 

communications, the APIES mission can achieve these ambitious goals within the framework 

of a standard ESA mission. APIES has completed a Mission Feasibility Study as part of 

the General Studies Programme (GSP) of ESA, whose purpose is to evaluate novel missions, 

concepts, methods, and to identify their research and development needs beyond currently 

running programmes. 

Figure 1.5: The HIVE carrier spacecraft with the BEEs 

In the baseline concept, the target orbit for the APIES swarm is based on a HIVE helio- 

centric circular orbit at 2.6 AU. This orbit selection is the result of a trade-off between the 

need of achieving a high rate of asteroid flybys (hence targeting a high density region of the 

asteroid belt) and that of adequately sampling the diversity of the asteroid population (and 

so targeting a Main Belt zone where the population is mixed, with representatives of most 

of the known asteroid spectral classes). To achieve the final operational orbit, it is envisaged 

that the APIES swarm will be transported by the HIVE carrier spacecraft, with the BEEs, 

the spacecrafts, deployed only after reaching the asteroid belt, figure 1.5. APIES is designed 

for a Soyuz/Fregat launch, capable of injecting a mass of up to 1420 kg into a Mars flyby 

trajectory. The HIVE, still carrying the BEEs, will take advantage of a Mars gravity assist 

and then use its own Solar Electric Propulsion (SEP) system to reach its 2.6 AU final circu- 

lar heliocentric orbit. It has been estimated that with the Soyuz/Fregat launcher and Mars 

gravity assist, an SEP system can deliver about 850 kg of payload (total ad-up mass for the 

BEEs, which are thought to be less than 50 kg each) to a circular orbit at 2.6 AU within a


3-year transfer time. An additional 3-4 years may then be needed for the deployment of the 

BEEs swarm to its nominal operational formation. 

After reaching the asteroid belt, the BEEs will separate from the HIVE and create a 

swarm ’cloud’ centered on the HIVE, [D’A04], [2]. 

ANTS APIES 

Launch date 2025 − 

Objective Asteroid belt Asteroid belt 

Spacecraft mass 1 kg 50 kg 

Spacecraft number 1000 19 

Table 1.3: Comparison between ANTS and APIES missions

10 CHAPTER 1. INTRODUCTION

CHAPTER 

2 

FORMULATION OF THE PROBLEM 

It is possible to make things of great complexity out of things that are very simple. 

There is no conservation of simplicity. 

Stephen Wolfram (1959 - ) 

In this chapter the formulation of the problem is described. First of all, the chosen sce- 

narios for SMAS are outlined, then connected critical issues are presented. To try to make 

first design hypotheses, several MAS techniques are analyzed in details, in particular in the 

framework of formal methods. In the end, the work is outlined and described in its parts. 

2.1 Typical scenarios 

As stated, the proposed missions are basically of two type: 

1. formation flying systems; 

2. exploration missions. 

These scenarios have characteristic features which have to be understood before a rea- 

sonable design could start. This is typical: given a problem, the solution could be sought. 

In space research, however, it is not so obvious. Often for cost reasons a good design is not 

the one perfect for one and only one mission, but it is the one which could be used for many 

different aims. It is crucial, in this view, to try to develop a solution which is general enough 

to be used for a family of problems. 

Another very peculiar issue in space research is the system scalability. As it has been 

shown, the future SMAS are going to be very large in comparison with MAS. Thus the 

scalability of the algorithms has to be proved. 

11

12 CHAPTER 2. FORMULATION OF THE PROBLEM 

To analyze the algorithms which have to be developed, two missions have been selected, 

one per type, namely 

1. Prisma mission; 

2. main asteroid belt exploration. 

2.1.1 Formation flying in Prisma Mission 

The mission objectives of Prisma may be divided into the validation of sensor and actuator 

technologies related to formation flying and the demonstration of experiments for formation 

flying and rendezvous. It will support and enable the demonstration of autonomous space- 

craft formation flying, homing, and rendezvous scenarios, as well as close-range proximity 

operations. The mission schedule foresees a launch of the two spacecraft in 2009. Both Main 

and Target will be injected by a Dnepr launcher into a sun-synchronous orbit at 700-km 

altitude and 98.2 deg inclination. A dusk-dawn orbit with a 6 or 18 h nominal local time at 

the ascending node (LTAN) is targeted. Following a separation from the launcher, the two 

spacecraft will stay in a clamped configuration for initial system checkout and preliminary 

verification. Once the spacecraft are separated from each other, various experiment sets for 

formation flying and in-orbit servicing will be conducted within a minimum targeted mission 

lifetime of eight months. Spacecraft operations will be performed remotely from Solna, near 

Stockholm, making use of the European Space and Sounding Rocket Range (Esrange) ground 

station in northern Sweden. The Sband ground-space link to Main supports commanding 

with a bit rate of 4 kbps and telemetry with up to 1 Mbps. In contrast, communication with 

the Target spacecraft is only provided through Main acting as a relay and making use of 

a Main-Target intersatellite link (ISL) in the ultrahigh-frequency (UHF) band with a data 

rate of 19.2 kbps. The Main spacecraft has a wet mass of 150 kg. In contrast to the highly 

maneuverable Main spacecraft, Target is a passive and much simpler spacecraft, with a mass 

of 40 kg [GM07], [PG05], [DM06]. Hence the mission key features can be summarized as 

follows 

� 2 s/cs; 

� communication architecture: sensor → relay → Earth; 

� Earth environment/perturbations. 

2.1.2 Main asteroid belt exploration 

ANTS proposed spacecraft 

The proposed spacecraft for ANTS mission has, [4] 

� power: 100 mW battery; 

� material: 1 kg, 100 m 2 /kg;

2.1. TYPICAL SCENARIOS 13 

� locomotion: solar sail. 

Sail achieves dynamic attitude control through capability for dynamic change in its mor- 

phology, thus changes the effective area and distribution of solar reflectivity to change its 

acceleration and momentum vectors to achieve required orbit and orientation. 

Asteroid belt 

The asteroid belt is the region of the Solar System located roughly between the orbits of 

the planets Mars and Jupiter. It is occupied by numerous irregularly shaped bodies called 

asteroids or minor planets. The asteroid belt region is also termed the main belt to distinguish 

it from other concentrations of minor planets within the Solar System, such as the Kuiper 

belt and scattered disk. 

Figure 2.1: Main asteroid Belt 

More than half the mass within the main belt is contained in the four largest objects: 

Ceres, 4 Vesta, 2 Pallas, and 10 Hygiea. All of these have mean diameters of more than


400 km, while Ceres, the main belt’s only dwarf planet, is about 950 km in diameter. The 

remaining bodies range down to the size of a dust particle. 

The asteroid belt formed from the primordial solar nebula as a group of planetesimals, 

the smaller precursors of the planets. Between Mars and Jupiter, however, gravitational 

perturbations from the giant planet imbued the planetesimals with too much orbital energy 

for them to accrete into a planet. Collisions became too violent, and instead of sticking 

together, the planetesimals shattered. As a result, most of the main belt’s mass has been 

lost since the formation of the Solar System. Some fragments can eventually find their way 

into the inner Solar System, leading to meteorite impacts with the inner planets. Asteroid 

orbits continue to be appreciably perturbed whenever their period of revolution about the 

Sun forms an orbital resonance with Jupiter. At these orbital distances, a Kirkwood gap 

occurs as they are swept into other orbits [1]. 

Even if perturbed, main asteroid orbital parameters are known with a reasonable uncer- 

tainty; for example Ceres semi-axis is known with 10 −9 1 − σ relative error. Nonetheless, 

since mass measurement is more complex, the physical parameters have significatively more 

uncertainty, also in the order of 50%, thus this is the most relevant problem in the perturba- 

tion model determination. In Table 2.1, the first ten more massive asteroids, which will be 

included in the perturbation model [2] [3]. 

Name Mass [ M⊙ × 10 −10 ] a [AU] e [deg] i [deg] Ω [deg] ω [deg] θ [deg] 

Ceres 4.39 ± 0.04 2.7659 0.07976 10.58 80.40 73.15 215.80 

Vesta 1.69 ± 0.11 2.3619 0.08936 7.13 103.91 150.18 341.59 

Pallas 1.59 ± 0.05 2.7716 0.23075 34.84 173.13 310.34 199.72 

Hygiea 0.47 ± 0.23 3.1367 0.11790 3.84 283.45 313.03 91.71 

Psyche 0.087 ± 0.026 2.9193 0.13953 3.09 150.34 227.80 141.36 

Eunomia 0.042 ± 0.011 2.6436 0.18728 11.73 293.27 97.90 354.91 

Hermione 0.0305 ± 0.0013 2.7208 0.08195 6.60 147.93 85.36 154.86 

Parthenope 0.0258 ± 0.001 2.4521 0.10019 4.62 125.62 195.25 230.35 

Massalla 0.024 ± 0.004 2.4088 0.14276 0.70 206.50 255.49 38.98 

Table 2.1: Main asteroids masses and orbital parameters at 2007 – April – 10.0. 

The mission 

The key features of this mission include 

� ≫ 2 s/cs – scalability proof; 

� multi communication architecture, typical: sensor → relay → ... → hub → Earth, but 

it could be different depending on the agent design; 

� asteroid belt environment/perturbations.

2.2. AGENT ARCHITECTURE AND ASSUMPTIONS 15 

And the main assumptions on the environment 

� reduced two-body gravitational field; 

� the gravitational effects of asteroids will be considered as a perturbation force and this 

is reasonable if the agents are far from them; 

� the rendezvous will not be considered and the agents will be always out of the spheres 

of influence of asteroids. 

2.2 Agent architecture and assumptions 

As it could be understood from these two scenarios, there are different problems to handle. 

First of all the spacecrafts have to perform particular missions, they have targets and they 

do have not to collide with other objects or each other. This can be called the physical part, 

PP, of the control. 

Then the spacecrafts are supposed to communicate, both among each other and with 

Earth, therefore, since communications are one of the most critical issue in space design, the 

communication network deployment and maintenance part, CP, of the control is, at least, as 

important as the PP. 

Since the multi agent system has to be highly autonomous, both parts have to be thought 

in a way they could be very flexible and simple enough to permit real time control by the 

agent itself. This is quite obvious since the environment in which the agents will operate is 

very dynamic and the Earth – agent communications could take too long. Moreover, this 

leads to reduce the communication among the agents as much as possible. 

Of course the agents, before even switching on the control, have to decide what to do, 

therefore they have to plan in which way they have to act. As stated, this decision has to 

be taken with the less communication possible among the agents, then it has to be real time 

and robust enough. Hence, the architecture is the one in Figure 2.2. 

The main assumptions on the agents are 

� they are knowledge and resource bounded, thus they do not have the knowledge of the 

whole environment and they have to cooperate to fulfill the tasks; 

� they can move use continuous thrust motors, electrical driven in the formation flying 

scenario, solar sail driven in the asteroid exploration one, [4]; 

� they can communicate, both broadcasting or using a peer to peer protocol, but the 

range of communication is limited; 

� they have sensors to determine their current state and the state of what they sense; 

� they can be either explorers or communicators, which means that some agents can have 

the capabilities to perform scientific researches – the explorers – but they have lim- 

ited communication capabilities; on the contrary other agents cannot perform scientific


Low Level Control High Level Control 

Goal Manager 

✄ 

✄ 

✄ 

✄ 

✄ 

✄ 

❈ ❈❈❈❈❈ 

PP CP 

Agent 

Figure 2.2: The control architecture. 

measurements, but they can communicate in a better way, maybe also with Earth – the 

communicators. This implies that explorers cannot communicate each other and they 

need communicators to relay data back to Earth; communicators can send information 

to other communicators and some of them, the communicator Hubs, can send data back 

to Earth, Table 2.2. 

Explorers Hubs Communicators 

Scientific payload yes no no 

Communication only with communicators with all and Earth with all but not with Earth 

Table 2.2: Agent composition. 

Since, the goal manager and the couple (PP,CP) have to be though in the context of 

MAS, the next sections will review some basic concepts of this topic. 

2.3 Multi agent techniques 

Two paradigms dominate the design of multi agent systems. The first, that will be called 

the traditional paradigm, is based on deliberative agents and (usually) central control, while 

the second, the swarm paradigm, is based on simple agents and distributed control. In the 

past two decades, researchers in the Artificial Intelligence and related communities have, for 

the most part, operated within the first paradigm. They focused on making the individual 

agents, be they software agents or robots, smarter and more complex by giving them the 

ability to reason, negotiate and plan action. In these deliberative systems, complex tasks can 

be done either individually or collectively. If collective action is required to complete some

2.3. MULTI AGENT TECHNIQUES 17 

task, a central controller is often used to coordinate group behavior. The controller keeps 

track of the capabilities and the state of each agent, it decides which agents are best suited 

for a specific task, it assigns it to the agents and coordinates communication between them. 

Deliberative agents are also capable of collective action in the absence of central control; 

however, in these cases agents require global knowledge about the capabilities and the states 

of other agents whom they may form a team with. Acquiring such global knowledge may be 

expensive and thus impractical for many applications. For instance, a multi agent system 

may break into a number of coalitions containing several agents, each coalition being able to 

accomplish some tasks more effectively than a single agent can. In one approach to coalition 

formation, the agents compute the optimal coalition structure and form coalitions based on 

this calculation [LJGM05]. 

Swarm Intelligence represents an alternative approach to the design of multi agent sys- 

tems. Swarms are composed of many simple agents. There is no central controller directing 

the behavior of the swarm, rather, these systems are self-organizing, meaning that construc- 

tive collective (macroscopic) behavior emerges from local (microscopic) interactions among 

agents and between agents and the environment. Self-organization is ubiquitous in nature, 

bacteria colonies, amoebas and social insects such as ants, bees, wasps, termites, among oth- 

ers, are all examples of this phenomenon. Swarms offer several advantages over traditional 

systems based on deliberative agents and central control: specifically robustness, flexibility, 

scalability, adaptability, and suitability for analysis. Simple agents are less likely to fail than 

more complex ones. If they do fail, they can be entirely pulled out or replaced without signif- 

icantly impacting the overall performance of the system. Distributed systems are, therefore, 

tolerant of agent error and failure. They are also highly scalable, increasing the number of 

agents or task size does not greatly affect performance. In systems using central control, 

the high communication and computational costs required to coordinate agent behavior limit 

the system size to at most a few dozen agents. Finally, the simplicity of agent’s interactions 

with other agents makes swarms amenable to quantitative mathematical analysis. The main 

difficulty in designing a swarm is understanding the effect individual characteristics have on 

the collective behavior of the system. 

Traditional paradigm Swarm paradigm 

Control (usually) central distributed 

Agent software complex simple 

Communication intensive minimal 

Time to act long short 

Scalability no yes 

Reliability poor very good 

Table 2.3: Comparison between traditional and swarm paradigm.


2.3.1 Space multi agent system 

In space multi agent systems challenges could be still more. Not least amongst these are 

the complex interactions between heterogeneous components, the need for continuous re- 

planning, re-configuration and re-optimization, the need for autonomous operation without 

intervention from Earth, and the need for assurance of the correct operation of the mission. 

In missions such as ANTS [RHRT06], that will be highly autonomous and out of contact 

with ground control for extended periods of time, errors in the software may not be observable 

or correctable after launch. Because of this, a high level of assurance is necessary for these 

missions before they are launched. Testing of space exploration systems is done through 

simulations, since it would be impractical or impossible to test them in their final environment. 

Although these simulations are of very high quality, often very small errors get through and 

can result in the loss of the entire mission, as it is thought to have happened with Mars Polar 

Lander Mission [CS00]. 

Complex missions like these exacerbate the difficulty of finding errors, and will require 

new mission verification methods to provide the level of software assurance that for example 

NASA requires to reduce risks to an acceptable level. Errors under such conditions can rarely 

be found by inputting sample data and checking for correct results. To find these errors 

through testing, the software processes involved would have to be executed in all possible 

combinations of states (state space) that the processes could collectively be in. Because 

the state space is exponential (and sometimes factorial) to the number of states, it becomes 

intestable with a relatively small number of processes. Traditionally, to get around the 

state explosion problem, testers artificially reduce the number of states of the system and 

approximate the underlying software using models. This reduces the fidelity of the model 

and may mask potential errors. A significant issue for specifying (and verifying) swarms is 

support for analysis and identification of emergent behavior. The idea of emergence is well 

known from biology, economics, and other scientific areas. It is also prominent in computer 

science and engineering, but the concept is not so well understood by computer scientists 

and engineers, although they encounter it regularly. Emergent behavior has been described 

as system behavior that is more complex than the behavior of the individual components, [...], 

often in ways not intended by the original designers [PV97]. This means that when interacting 

components of a system whose individual behavior is well understood are combined within a 

single environment, they can demonstrate behavior that can be unforeseen or not explained 

from the behavior of the individual components. 

2.3.2 Formal methods 

Formal methods [Rou06] are proven approaches for assuring the correct operation of complex 

interacting systems, being them mathematically-based tools and techniques for specifying and 

verifying systems. They are particularly useful for specifying complex parallel and distributed 

systems, where the entire system is difficult for a single person to fully understand and 

when more than one person was involved in the development. With formal methods, certain

2.3. MULTI AGENT TECHNIQUES 19 

properties may be proposed to hold, and prove that they hold. In particular, this is invaluable 

for properties that cannot be tested on Earth. By its nature, a good formal specification can 

guide researchers to propose and verify certain behaviors (or lack of certain behaviors) that 

they would often not think of when using regular testing techniques. Moreover, if properly 

applied and used in the development process, a good formal specification can guarantee 

the presence or absence of particular properties in the overall system well in advance of 

mission launch, or even implementation. Indeed, various formal methods offer the additional 

advantage of support for simulation, model checking and automatic code generation, making 

the initial investment well worth while. It has been stated that formal analysis is not feasible 

for emergent systems, due to their complexity and intractability, and that simulation is the 

only viable approach for analyzing emergence of a system [BP03]. For space missions in 

general, relying on simulations and testing alone is not sufficient even for systems that are 

much simpler than the swarm missions, as noted above. The use of formal analysis would 

complement the simulation and testing of these complex systems giving additional assurance 

of their correct operation. Given that one mistake can be catastrophic to a system and result 

in the loss of money and years of work, development of a formal analysis tool, even at a great 

cost, could have huge returns also if only one mission is kept from failing. 

Verifying emergent behavior is an area that has been addressed very little by formal meth- 

ods, though some work has been done in this area by computer scientists, analyzing biological 

systems [SB01] [Tof91]. However, formal methods may provide guidance in determining pos- 

sible emergent behaviors that must be considered. Formal methods have been widely used 

for test case generation to develop effective test cases. Similar techniques may be used with 

formal methods, not to generate a test plan, but to propose certain properties that might or 

might not hold, or certain emergent behaviors that might arise. 

The Formal Approaches to Swarm Technologies, FAST, project has surveyed formal meth- 

ods and formal techniques to determine whether existing formal methods, or a combination 

of existing methods, could be suitable for specifying and verifying swarm-based missions and 

their emergent behavior [RR04] [RH05] [RR06]. Various methods have been surveyed based 

on a small number of criteria that were determined to be important in their application to 

intelligent swarms. These include: 

� support for concurrency and real time constraints; 

� formal basis; 

� (existing) tool support; 

� past experience in application to agent based and/or swarm based systems; 

� algorithm support. 

A large number of formal methods, that support the specification of one between, but 

not both, concurrent and algorithmic behavior, have been identified. In addition, there


is a large number of integrated or combination formal methods that have been developed 

over recent years, with the goal of supporting the specification of both concurrency and 

algorithms. Although the survey identified a few formal methods, to used to specify swarm 

based systems, initially only two formal approaches were found that had been used to analyze 

the emergent behavior of swarms, namely Weighted Synchronous Calculus of Communicating 

Systems (WSCCS) [Tof91] and Artificial Physics [SG99] [SY99a]. 

The following is a brief description of some specification techniques that have been used 

for specifying social, swarm, and emergent behavior: 

� Weighted Synchronous Calculus of Communicating Systems (WSCCS), a process alge- 

bra, was used by Tofts to model social insects. WSCCS was also used in conjunction 

with a dynamical system approach for analyzing the non-linear aspects of social insects. 

� X-Machines have been used to model cell biology and modifications, such as Commu- 

nicating Stream X-Machines that also seem to have potential for specifying swarms 

[Hol88]. 

� Dynamic Emergent System Modeling Language (DESML), a variant of UML, has been 

suggested for use in modeling emergent systems [Kin98]. 

� Cellular Automata have been used to model systems that exhibit emergent behavior 

(land use) [vN96]. 

� Artificial Physics, which uses physics-based modeling to gauge emergent behavior, has 

been used to provide assurance for formation flying as well as other constraints on 

swarms. 

NASA is currently developing its own formal language based on a mix of the first two 

languages and some others, since it thinks that the two are not sufficient by their own. 

DESML, though very interesting, has not been chosen because it had not been used or 

evaluated outside of the thesis it was developed under. Cellular Automata have not been 

selected because they do not have any built in analysis properties for emergent behavior and 

because they have been primarily used for simulating emergent systems. Artificial physics, 

which is very promising, has not been selected by NASA because of the newness of the 

approach [RHRT06]. 

2.3.3 Artificial Physics 

Although Artificial Physics, AP, has not been considered by NASA, it is worth enough to be 

analyzed in details, because it could offer several advantages upon the other formal methods. 

Moreover, the fact it is quite new is not really a problem, since its newer approach could lead 

to reconsider old issues under different perspectives. 

AP is a physics oriented approach to construct a coordinated task-allocation algorithm 

for cooperative goal-satisfaction [SY99a]. This has to be used by the single agent within the 

system and it enables coordination without negotiation and with limited communication.

2.4. WORK’S FRAMEWORK 21 

Basically the approach consists in the calculation of a potential function, based on what 

a single agent can sense, and in a derivation of the actions that agent has to make. For 

this reason the name “Potential Field approach” will be used as well as “Artificial Physic 

approach”. 

The AP idea can be summarized as 

� physics oriented approach, so AP is in the context of formal methods and emergent 

behavior can be verified by mathematical tools as statistical mechanics; 

� complete reactive behavior, since agents act only as a response of what they sense; this 

leads to both scalability and real time algorithms; 

� potential function calculation, therefore the approach is very close to the ones used 

for example in robot/rover path planners [ZW02], [GC00] [Rei92], [ZS04], [BL92], this 

means that space researchers could have a good background to deal with AP. 

These properties can respond to the ones expressed before, and since AP could be com- 

plete, simple and close enough to previous tested algorithms, it is chosen for this work. 

Although the approach is appealing, it has to be kept in mind that it is quite new, thus 

only basic test cases have been analyzed in previous works. 

2.4 Work’s framework 

The work deals with the development of a distributed control for a SMAS using an AP 

approach. First of all, the high level control has to be developed, in chapter 3. This is not 

the mere application of AP, but a suitable extension of this approach has to be derived to 

take into account different capabilities and goals. It is clear that in specifying these properties 

some assumptions have to be made. Then, the PP and CP of the control have to be analyzed, 

the former in a way which leads to a robust and reliable motion control, chapter 4, the latter 

in order to assure the communication network deployment minimizing the communication 

effort among the agents, chapter 5. Finally, the perturbation models, which describe the 

environments, have to be developed, chapter 6.


Low Level Control High Level Control 

Goal Manager (chap. 3, Algorithm 1) 

✄ 

✄ 

✄ 

✄ 

✄ 

✄ 

PP 

(chap. 4, Algorithm 2) 

❈ ❈❈❈❈❈ 

Agenti 

CP 

(chap. 5, Algorithm 3) 

Figure 2.3: Work outline. 

✛ 

✛ 

✲ 

Agentj 

Information sharing 

(chap. 3 and chap. 5) 

✲ 

Environment 

Targets and Perturbations 

(chap. 3 to 6)

CHAPTER 

3 

THE DECISIONAL LEVEL 

People just naturally assume that dogs would be incapable of working together on some sort 

of construction project. But what about just a big field full of holes? 

Jack Handey (1949 - ) 

In this chapter the Goal Manager is developed in the framework of Artificial Physics. 

First, AP is extended in a suitable way to take into account also multiple tasks and capabilities, 

then information sharing mechanism is outlined pointing out key issues. 

3.1 Artificial Physics Formulation 

Let A = {a1, a2, . . . aN} be the set of agents, which could be spacecrafts but also robots or 

whatever. Let G = {g1, g2, . . . , gM} be a set of goals, possibly dynamically changing. These 

two sets can be located in a goal space, G, thus A ∈ G and G ∈ G. Let x be the coordinates 

of agents and goals in the goal space. Using this notation, both a displacement vector, Dij, 

and a metric, dij, can be defined on G as 

Dij ≡ xi − xj 

dij ≡ ||xi − xj|| 2 

(3.1) 

(3.2) 

Since, in some domains, goals do not have physical properties, the components of Dij are 

not necessarily physical distances. 

Then it is to assume that: 

� the agents have the ability to perceive the displacement vector in the goal space and 

they can perceive the properties of other adjacent agents and goals. This may be done 

by sensors, integrated into the agents; 

23

24 CHAPTER 3. THE DECISIONAL LEVEL 

� each agent knows about the types of resources that other agents may have. 

These two assumptions are necessary since agents who progress within the goal space need 

some information regarding properties of other agents and goals. Moreover it is also to assume 

that: 

� each agent has a performance capability, mi ∈ R, that can be measured by standard 

measurement units, which enable quantification of the agents’ task execution; 

� there is a scaling method which is used to represent the displacement of the agents in 

the goal space and to evaluate the mutual distances between goals and agents within 

this space. 

These assumptions are necessary since distances are a significant factor in the AP model. 

Moreover it is to assume that goal satisfaction can be achieved progressively. That is, a 

goal may be partially satisfied at one instant, and its remaining non satisfied part may be 

completed at another point in time. 

Then, another basic assumption in AP is that each agent, ai, has a fixed capability, 

mi, and each goal requires only a capability to be satisfied. Therefore agents can do only a 

single task for goals which require only a single task. This assumption is quite strong and 

in the following sections will be removed and the AP approach will be extended in a very 

straightforward way. 

G 

A 

• a1 

• a3 

• an ◦ g1 

• an 

� 

�� 

• a2 

d2n 

❅ 

❅❅❅❅❅ 

dn4 

G 

◦ g4 

◦ gm 

Figure 3.1: Goal space, sets and metric. 

Given these assumptions, each agent, ai, in every time instant can compute a potential 

function as 

◦ g2 

◦ g3 

Φi = � 

Φa(mn, din) + � 

Φg(mm, dim) (3.3) 

n 

m

3.1. ARTIFICIAL PHYSICS FORMULATION 25 

where Φa and Φg are suitable potentials, the former takes into account the mutual repulsion 

between the agents, since it is not reasonable that many agents perform the same goal, and 

it could be in the form 

Φa ∝ mn 

din 

(3.4) 

The latter represents the natural attraction towards the goal, which requires a certain amount 

of capacity to be fulfilled, mm, and it could be in the form 

Φg ∝ − mm 

dim 

The n and m are the sensed agents and goals. 

(3.5) 

Example. Just to clarify, an example can be made. Let the goals be the fixing of holes 

on a surface, bigger the hole, bigger the value of m. The agents have a certain capability 

which is their size, bigger the agent bigger its mi. During time, the agents can eventually fix 

a certain amount of a hole, so m for that hole can decrease, and in fact m = m(t). 

Once computed the potential function, the agents have to derive the force, since they 

have to move towards the minimum of the potential field, which is of course time dependent. 

The calculation is straightforward being 

F i = −mi∇Φi 

(3.6) 

The dimensions of vector F i depend on the dimensions of the coordinate vector, x, which 

depends on the goal space representation. For example in the scenario of the holes, G is a 

subset of R × R and F i is a bidimensional vector. 

In Table 3.1 the match between MAS and the shown physical approach is shown. 

Multi agent system Physical model 

Agent Dynamic particle 

Goal Static particle 

Agent’s capability Particle’s mass 

Agent’s location Particle’s location 

Algorithm for goal allocation Formal method for calculating 

the evolution of displacement 

Table 3.1: The match between MAS and physical model. 

The force vector, whom the agent has to act accordingly to, has to be projected by the 

same agent in a meaningful space for itself. Let ◦ be the projection transformation, hence 

each agent has to compute 

Ai = F i ◦ Vi 

(3.7)


where Ai is the vector of the final actions and Vi is the suitable space for the agent ai. 

In the given example, the projection is straightforward since it is only an identity. In fact, 

the force vector is indeed a control force, and the example could be regarded as a feedback 

control. 

3.1.1 AP state of the art 

AP has been developed independently by two groups of researchers. The first belongs to 

University of Wyoming at Laramie, this is the Spears group; the second is located at Carnegie 

Mellon University, Pittsburgh, and it is represented by the person of Onn Shehory. Basically 

the approach is the same, but the former group is more focused on real physical goal space, 

while the latter is interested in very large MAS for both web and market applications. 

Spears. The main idea of Spears group is to try to use the AP approach to move a cluster 

of robots in a sort of formation. Therefore the goal space has a complete physical meaning 

[SG99]. In Figure 3.2 an example of artificial potential calculation for moving robots: as it 

can be seen, the introduction of a virtual agent, which represents the mean motion of the 

cluster, is exploited. The Spears group is not the only one which uses the AP in this way, 

[EK06]. 

Figure 3.2: Artificial physics for formations of robots 

Shehory. The studies of this second group are both more interesting and more general. 

Focused both on web services and market transactions, they imply distributed algorithms 

among very large teams of agents. As stated in some papers, the AP approach is practically 

the only one capable to deal with very large MAS [SY99a].

3.2. EXTENSION OF AP FORMULATION 27 

3.2 Extension of AP formulation 

The main strong assumption of AP is that each agent, ai, has a fixed capability, mi, and 

each goal requires only a capability to be satisfied. This can be easily removed. 

Let the agents, ai ∈ A, i = 1, . . . , n, have multiple capabilities and let C be the matrix of 

the total A set capabilities. Then let the goals, gi ∈ G, i = 1, . . . , m, require more tasks to 

be fulfilled and let T be the matrix of the total G set tasks. 

C can be seen as 

⎡ 

⎢ 

C ≡ ⎢ 

⎣ 

c11 c12 c1k 

c21 c22 c2k 

. .. 

cn1 cn2 cnk 

⎤ 

⎥ 

⎦ 

(3.8) 

thus C ∈ R n × R k , being k the total number of the capabilities the agents have. Each 

coefficient of the C matrix, cij is nothing else but a real number which states how well the 

agent ai could perform the task j. 

Whereas T can be seen as 

⎡ 

⎢ 

T ≡ ⎢ 

⎣ 

t11 t12 t1q 

t21 t22 t2q 

. .. 

tm1 tn2 tmq 

⎤ 

⎥ 

⎦ 

(3.9) 

thus T ∈ R m × R q , being q the total number of the tasks the goals require. Each coefficient 

of the T matrix, tij is nothing else but a real number which states how much of the task j 

serves to fulfil the goal gi. It is straightforward that T is time dependent. 

Clearly k � q since otherwise the agents could not perform the goals. Let k = q, with 

no loss of generality. By the assumptions of extended AP approach, each agent can write k 

potentials which live in the goal space G as 

Φ w i = � 

αnicnwciw Φa(din) + � 

βnitnwciw Φg(dim) , w = 1, . . . , k (3.10) 

n 

m 

where, as before, Φa and Φg are suitable potentials, the former takes into account the 

mutual repulsion between the agents, since it is not reasonable that many agents perform 

the same goal; the latter represents the natural attraction towards the goal, which requires a 

certain amount of capacity to be fulfilled. Note that the potentials have lost their dependence 

on capabilities. In the end α and β are coefficients. 

For each Φ w i and for each ai, a force vector can be computed as 

F w i = −∇Φ w i (3.11) 

Then, by a suitable projection transformation, each agent can compute the real action. 

Note that, in this case, force vectors could be mutually contrasting, thus the transformation 

has to include some conflict detection/solver techniques, F. 

� 

Ai = F F 1 i ◦ V 1 i , F 2 i ◦ V 2 i , . . . , F k i ◦ V k � 

i 

(3.12)


3.3 Formal methods 

As stated, AP formulation is chosen for this work mainly because it could be seen as a formal 

method, thus it could verify the presence of emergent behavior. In the new extended AP 

version, since classical mechanics does not provide many different properties a body can have 

more than inertia properties, statistical mechanics has to be used as suggested by [SY99a], 

[DM06]. This parallelism, although it is not difficult to develop, will not be explored in this 

work. 

3.4 Information propagation and knowledge bounded agents 

Since the agents are knowledge bounded, they cannot have a global representation of the 

environment and thus they may not have any target. This can overcome through a suitable 

information sharing architecture, which have to use the communication less as possible. 

Different techniques have been proposed to solve the target assignment problem via in- 

formation sharing, for example [SB07a] shows how a broadcast coupled with a distributed 

algorithm could be optimal in some cases, namely when the environment is sparse. Other 

approaches, [SN08], [YS06], use randomly peer to peer sent packets of information, which 

have the advantage to decrease the communication effort. 

Even if, the suitable algorithm could be a mix of the two, since the target assignment 

frequency is supposed to be low compared to motion dynamics and typical communications, 

the broadcast approach is adopted. Moreover, this approach is simpler than the peer to peer 

one. 

3.4.1 Example 

To try to understand how the information sharing mechanism works an example can be made. 

Let g1, g2, g3 be three asteroids in a three dimensional space and let ai ∈ A be the agents, the 

spacecrafts, randomly located in the space, which have to choose which asteroid reach. Each 

ai has three different capabilities, {ci1, ci2, ci3}, which could be, for example: take photos, 

communicate, detect water. Each asteroids, gj requires, at the beginning, a certain amount 

of capabilities, {tj1, tj2, tj3}, which can be thought as proportional to the asteroid surface. At 

the initial time τ0, only a subset of the agents, A ⊆ A, can actually sense the asteroids, and, 

supposing the could have a complete knowledge of the object, they can set the {tj1, tj2, tj3} 

for each asteroid they sense. Thus each ai ∈ A can define its own TS, which is the sensed T 

matrix as: 

where j ∈ {sensed object}. 

TS = 

� 

tj1 tj2 tj3 

. . . 

Then, using the Goal Manager, namely the potential field 3.10 and the following formulae, 

with T = TS, they can decide which asteroid reach. Chosen their target, namely j, they 

�

3.5. DISTRIBUTED SCHEDULING 29 

change the {tj1, tj2, tj3} accordingly, subtracting their capacities 

{tj1, tj2, tj3}τ1 = {tj1, tj2, tj3}τ0 − {ci1, ci2, ci3} (3.13) 

and broadcast the information adding to the packet also the time in which they decided, thus 

where TRi = TS at τ1. 

Msgi ≡ {TRi , τ1} 

When other agents receive the information arriving from different sources, they added it 

to what the have sensed, TS, they form a new T as 

� 

� 

T = TS ∪ 

where the i–th agent is the one who has broadcast them the information. 

Then they decide, subtract and broadcast. This process of receiving and sending has a 

i 

TRi 

fixed frequency, depending on the sparsity of the systems. 

It has to be noted that in some cases agents have to decide upon conflicting messages, 

since the information path is not unique. To solve these conflicts policies could be added 

to the agent control, but by now, only a simple policy has been developed: the time policy. 

The newest message has to be followed and if more messages than one are the newest, a 

random choice is performed. Using this policy the greater the agent number is, the more 

accurate the target assignment is. Defining a Sum operator which embodies the policy, the 

T determination has to be changed accordingly as 

3.4.2 Algorithm 

� 

T = Sum(TS, SumiTRi ) 

The final, Goal Manager algorithm is, for each agent, ai the Algorithm 1 

3.5 Distributed scheduling 

As stated, in most of the cases, in the context of multi agent systems conflicts appear. This is 

the case of overlapping requests of the same resource, for example, when two explorers would 

like to use the same communication antenna to relay data back to Earth. This problem can be 

solved in the framework of distributed scheduling using different approaches, like decoupling 

strategies [BL08], but it will not face in this work.


Algorithm 1 Goal Manager and sharing mechanism, at τ = τ⋆ 

1: TS = Sense(Environment) 

2: TR = {} 

3: repeat 

4: Td =GetMsg 

5: TR = Sum(TR, Td) 

6: until (No more messages) 

7: T = Sum(TS, TR) 

8: /*Goal Manager Start*/ 

9: Φw i , eq. 3.10 

10: F w i , eq. 3.11 

11: Ai, eq. 3.12 

12: /*Goal Manager End*/ 

13: Form TR, eq. 3.13 

14: BroadCast Msg ≡ {TR, τ⋆+1}

CHAPTER 

4 

THE PHYSICAL PART 

Go, Traveler. Go anywhere. The universe is a big place, perhaps the biggest. 

Philip J. Farmer (1918 - ) 

In this chapter the physical part of the control is developed. First of all, the tracking 

problem is presented as a reasonable solution in the scenarios which have to be explored. 

Then different approaches to track the targets are shown, and it is explained how a suitable 

anna algorithm for long period dynamics, coupled with a short period dynamic control, is a 

reasonable solution which mixes both lightweight and robust features. 

4.1 Introduction 

The physical part of the control is in charge of moving agents towards dynamic evolving 

targets and since, the scenarios involve only physical goals, the problem which has to be 

solved could be formulated as: 

Given an initial condition on the state variables, x0, of each agent, ai, at the time t0, find 

the optimal control u which leads x0 → xf , the target state variables, at the time tf . 

The problem which each agent has to solve could be seen in two main ways: 

� as a Lambert problem, but, since this involves a lot of computational effort, it is not 

reasonable for a real time calculation; 

� as a tracking problem/feedback control, to be resolve either using AP approach or, that 

is the same in this case, with a suitable feedback control law. In this way, although it 

is not necessary, it is convenient for the control effort, to set tf → ∞, thus the problem 

becomes an infinite horizon control. 

31

32 CHAPTER 4. THE PHYSICAL PART 

4.2 Possible solutions 

4.2.1 Potential field approach 

The potential field approach has been used by ESA in different demonstrative softwares for 

SMAS [IP07]. The target position global knowledge is the key assumption of an equilibrium 

shaping technique, in order to find an optimal set of parameters which can lead the system 

towards a final equilibrium. Since in the scenarios of this work the information is not global 

but distributed, this assumption has to be removed and that parameter set has to be tuned 

in another way. 

Although this could be easily done, problems arising from the use of such a formulation 

could be difficult to overcome. The main one is that the control u is far from being optimal, 

as long as a suitable term in the potential field is not included. This term would shape 

the geometry of the space according to the geodetic lines of the dynamical system, thus it 

would exploit the space properties, instead of involving useless control effort. Although this 

approach could be easily developed dealing with a linearized system, it is not straightforward 

in a complete non linear one, since it involves a sort of dynamic inversion. For that reason 

this approach will not be used. 

4.2.2 SDRE approach for non linear systems 

The use of a feedback control, which takes into account the dynamical system non linearity, 

could be a suitable approach for the tracking problem. Initially developed for intelligent 

missiles applications, it has revealed some capabilities in space area since few years. 

The basic idea is to write the non linear dynamical system in an affine way [Bra04] [PB04], 

assuring that the dynamic matrices are not singular for x ∈ {x0, . . . , xf }. Thus, since the 

reduced two-body problem in a cartesian coordinate system is 

¨r = − κ 

r + u + w (4.1) 

r3 where κ is the planetary constant and w is the perturbation acceleration vector, it can be 

rewritten in the affine shape, introducing the state vector x = {r, ˙r} T as 

⎡ 

03 

I3 

⎤ 

⎡ 

⎢ 

˙x = ⎢ 

⎣ 

− − −− − − −− 

− κ 

⎥ ⎢ ⎥ 

⎦ x + ⎣− 

− −−⎦ 

(u + w) (4.2) 

I3 03 

r3 I3 

� �� 

A(x) 

B 

Since it is clear that the the matrix A can not be singular, the control problem can be 

formulated using a State Dependent Riccati Equation, SDRE, introducing an error variable 

e ≡ x − xf and writing a suitable cost function J as 

J ≡ 

� ∞ 

t0 

03 

⎤ 

e ′ Q e + u ′ R u − γ 2 w ′ w dt (4.3)

4.2. POSSIBLE SOLUTIONS 33 

Then u has to be found in a H∞ context thus it satisfies a particular steady state Riccati 

Equation. Two main problems arise from this kind of approach: 

� the use of cartesian coordinates lead to worse results than the use of more suitable set 

of coordinates; this has a numerical explanation; 

� the control u does not take into account collision avoidance. 

Both the first and the second problem can be fixed; in fact for the former a different set 

of coordinates has to be chosen, whereas for the latter two main ideas could be used: 

� a dynamical separation between long period and short period, thus the long takes in 

consideration the tracking problem, and the short the collision avoidance problem. The 

latter has to be formulated in a suitable way. 

� The control u could be written as 

u = −Ky, with y = f(ρa, ρT ) (4.4) 

where f(ρa, ρT ) is a suitable function which depends on the distance among agents ρa 

and with the target ρT . This leads to an output feedback control [Gad07] which is both 

too complex to resolve real time and possibly ill conditioned. 

The above considerations lead to the choice of a dynamical separation and the use of a 

different set of variables. 

4.2.3 Non linear Lyapunov control 

Although state dependent, the SDRE approach is nothing else but an extention of the usual 

linear control to non linear systems. It could offer some advantages but, since the system 

is highly non linear, the use of a non linear control, whose approach is very close to SDRE, 

could be more suitable. 

First of all, using the non singular equinoctial variables, which are chosen as the set of 

coordinate system, to have better numerical results, the equations of motion 4.1, neglecting 

the perturbations, can be written as [Naa02] 

where 

˙x = B(x) u (4.5)


� the state vector x = {a, P1, P2, Q1, Q2, l0} T is related to the classical elements set as 

� 

⎧ 

⎪⎨ 

⎪⎩ 

a = a 

P1 = e sin ¯ω 

P2 = e cos ¯ω 

Q1 = tan i 

2 

Q2 = tan i 

2 

sin Ω 

cos Ω 

l0 = ¯ω + M − nt 

⎧ 

⎪⎨ 

⎪⎩ 

a = a 

e = 

� 

P 2 1 + P 2 2 

i = 2 tan−1 � 

Q2 1 + Q22 Ω = tan −1 

ω = tan −1 

� Q1 

Q2 

� P1 

M = l − tan −1 

P2 

� 

� 

− tan −1 

with ¯ω = ω + Ω and l = l0 + nt. Moreover the true longitude, L, can be defined starting 

from the mean longitude l as L = ¯ω + θ. In the end l0, which is the mean longitude 

at epoch is set in a way so that the equations of motion loose the constant term and 

l0 → 0 for t → ∞; in fact l0,0 = l − nlf /nf ; 

⎡ 

⎢ 

B(x) = ⎢ 

⎣ 

with 

2a 2 (P2 SL − P1 CL) 

h 

− pCL 

h 

pSL 

h 

2a 2 p 

hr 

r � P1 + � 1 + p� 

� 

r SL 

h 

r � P2 + � 1 + p� 

� 

r CL 

h 

0 0 

0 0 

− pa � (P1SL + P2CL) + 2b 

� 

a 

h(a + b) 

� 

b = a 1 − P 2 1 − P 2 2 , h = nab , 

r 

h = 

h 

κ(1 + P1SL + P2CL) 

� P1 

P2 

p 

ra(1 + r − )(P1CL − P2SL) 

h(a + b) 

In the framework of Lyapunov control, this theorem holds 

� 

� Q1 

Q2 

p 

r = 1 + P1SL + P2CL 

, C ≡ cos , S ≡ sin 

� 

(4.6) 

0 

− P2(Q1CL − Q2SL)r 

h 

P1(Q1CL − Q2SL)r 

h 

r(1 + Q2 1 + Q2 2)SL 

2h 

r(1 + Q2 1 + Q2 2)CL 

2h 

− r(Q1CL 

⎤ 

⎥ 

− Q2SL) ⎦ 

h 

(4.7)

4.2. POSSIBLE SOLUTIONS 35 

Figure 4.1: Classical orbital parameters


Theorem. Given a positive definite function, V = V (x) in Ω, which can be called a Lya- 

punov function, if the time derivate of V is negative definite in a subset ¯ Ω of Ω, then the 

system is asymptotical stable in ¯ Ω. 

Thus choosing V as 

V = 1 

2 δx′ Q δx , δx = x − xf (4.8) 

since the time derivate can be written as 

D 

Dt V = δx′ Q δ ˙x = δx ′ Q Bu (4.9) 

because ˙xf = 0, taking u as 

u = − 1 

2 K B′ Q ′ δx (4.10) 

with K ≻ 0, the time derivate of V is negative definite. 

This approach has different advantages, both it involves simple calculations which could 

be done in real time and it assures the asymptotical stability of the control. Of course, it has 

to be extended for the case in which also perturbations are taken into account and, moreover, 

it has to be united to a short period dynamic control for collisions avoidance. 

It has to be noticed that the dynamic separation is reasonable since the space environment 

is supposed to be sparse, this means that defining a typical free path length, λ, as the path 

length an agent has to travel before it encounters another agent, and defining the typical 

averaged length which the agent travels in a time step, δℓ, the relation λ ≫ δℓ holds. 

4.3 Long period dynamic 

4.3.1 Robust non linear Lyapunov control 

In this section the work of [Naa02] will be extented, taking into account the perturbations. 

The equations of motion are 

The problem can be reformulated in the H∞ approach as: 

˙x = B(x) [u + w] (4.11) 

Find both a definite positive Lyapunov function V whose time derivate 

D 

V = ∇V ˙x = ∇V B (u + w) (4.12) 

Dt 

is negative definite, and the minimal u which makes the cost function 

L(x, u, w) = 

� ∞ 

stationary with the maximum value of perturbations w. 

This statement leads to solve 

t0 

δx ′ ˜ Q δx + u ′ R u − γ 2 w ′ w dt (4.13) 

0 = min 

u max 

w [L(x, u, w) + ∇ V B (u + w)] (4.14) 

which is called the Hamilton – Jacobi – Isaacs, HJI, equation [GG96].

4.3. LONG PERIOD DYNAMIC 37 

4.3.2 HJI equation 

The equation 4.14 can be resolved with iterative techniques and, in fact, the resulting algo- 

rithm is not so different from a SDRE approach. 

The result is that [AKL06] 

⎧ 

u = − 1 

2 R−1 B ′ (∇V ) ′ 

⎪⎨ 

⎪⎩ 

w = 

where 

1 

2γ 2 B′ (∇V ) ′ 

0 = (∇ V ) B ′ 

It is simple to rewrite the last equation as 

� � 

I3 

− R−1 B (∇ V ) 

γ2 ′ + δx ′ Q˜ δx 

(4.15) 

0 = 2∇ V B ′ (u + w) + δx ′ ˜ Q δx (4.16) 

and seek an iterative solution for ∇ V , starting from u 0 and w 0 [BM00]. For this solution 

also a Galerkin approximation could be used, but, since this way leads to sixth order integrals 

for the dimension of the state space, it is not selected. 

The algorithm 4.15 has two main disadvantages, the first is to find a good starting point 

for the convergence, the second is that it includes a pseudo-inverse calculation. The way in 

which these problems can be fixed is by adapting the control to the case A = 0 in a new way. 

The first step is seeking a solution for the Lyapunov control in the form 

V = 1 

2 δx′ Q δx (4.17) 

which is reasonable, then substituting it in the solution of HJI equation 

0 = δx ′ Q B ′ 

� � 

I3 

− R−1 B Q δx + δx 

γ2 ′ Q˜ δx (4.18) 

thus leading to a particular symmetric equation, 

0 = δx ′ 

� 

Q B ′ 

� � 

I3 

− R−1 B Q + 

γ2 ˜ � 

Q δx = δx 

� �� 

W 

′ W δx (4.19) 

which is nothing but a particular Riccati equation, which can be called for simplicity anna 

equation for the palindrome. Resolving anna implies determining Q, so that 

Q B ′ 

� 

� 

B Q = ˜ Q (4.20) 

− I3 

+ R−1 

γ2 which could be found using a Newton algorithm approach, since the equation is non linear, but 

it would lead to the same problems as before. Since it is basically the same using as control


weights the couple ( ˜ Q, R) or (Q, R), the problem can be reformulated, without numerical 

problems, assuring that both � 

and 

− I3 

+ R−1 

γ2 w ≥ wmax 

where wmax is the maximum expected value of the perturbations. 

� 

≻ 0 (4.21) 

The algorithm, which can be called anna algorithm, can be summarized as follows 

(4.22) 

� Choice a couple (Q, R) and γ which satisfy the definite positiveness of the matrix 4.21 

� Calculate both u and w using 

⎧ 

� verify w ≥ wmax 

⎪⎨ 

⎪⎩ 

u = − 1 

2 R−1 B ′ Q δx 

w = 1 

2γ 2 B′ Q δx 

(4.23) 

As stated, this approach is the optimal solution of an H∞ problem in which the anna 

equation holds 

˜Q = Q B ′ 

� 

− I3 

+ R−1 

γ2 � 

B Q 

It is important to note that since wmax is known, and u is bounded, in fact u < ū, it is 

possible that for some perturbation the control effort could not be sufficient to overcome the 

perturbation. However this is a design problem and not really a control problem. 

4.3.3 R as a function of the state 

In most of the cases, it is reasonable to have a thrust which is almost constant over time. This 

can be achieved using a control weight, R, which is modulated on the state, thus R = R(x). 

Moreover if R is symmetric and definite positive for each x the control can be proved to be 

stable, thus 

(∀x(R(x) ≻ 0) ∧ (R(x) = R(x) ′ )) ⇒ 

� � 

D 

V ≺ 0 

Dt 

Proof. The proof is straightforward, since R is symmetric and definite positive, there exists 

a Cholesky decomposition thus 

R = HH ′ 

with H = H(x), hence the time derivative of the Lyapunov function V is 

D 

V = −1 

Dt 2 δx′ Q B H H ′ B ′ Q ′ δx 

� �� 

y 

= − 1 

2 y′ y ≺ 0 

�

4.4. SHORT PERIOD DYNAMIC 39 

4.4 Short period dynamic 

As stated, the collision avoidance can be handle using a suitable short dynamic control, thus 

each agent trajectory is modified in an optimal way to face possible conflicts. Designing such 

a control law is not trivial because it has to be, at least 

� real time; 

� robust enough to overcome perturbations and model/sensor errors. 

It could be thought that a suitable repulsion force, in a potential field way, could solve the 

problem easily. This is not the case, since potential functions have local minima which cannot 

assure a optimal control. Mainly there are two reasonable approaches, the first is to use an 

optimal feedback control, in the fashion of LQR or H∞, coupled with a velocity Potential 

field, as in [IP07] or [MY07]. This leads to good results and lightweight control. The key idea 

is to define a velocity field that make the spacecraft move far from the obstacles and close to 

the targets. Then a feedback control is added to obtain the required velocity. 

Another reasonable approach is the one of [IH02], [Arm04], even if it has to be extended 

to let it be fully distributed and moreover it has to be integrated into the long dynamic 

control law. The concept of [Arm04] approach, which it will be called AA now on, could be 

summarize as follows: 

Given the initial and the final positions of all the agents, find the optimal control law which 

enables them to reach their goals without any collisions. 

The AA involves linear simplifications, the global knowledge of where each agent is, and 

moreover it assumes to have a reliable perturbation model. Since these three main points are 

not acceptable for the problem which has to be faced, some extensions have to be thought 

and developed. First of all, the linear behavior assumption can be adopted if the reference 

orbit is the actual one, thus it is dynamically changing by the long dynamic control; second, 

the algorithm has to be thought as distributed, thus a policy, of a suitable decisional function 

has to be found and added. In the end, the control law has to be robust enough to face 

uncertainty in the perturbation model. 

4.5 Final Algorithm 

In this section only the long dynamic algorithm is exploited. Each agent, every time step, 

knowing its own target use the Algorithm 2.


Algorithm 2 Long dynamic control 

1: x ← (Actual State) 

2: xf ← (Target) 

3: δx = x − xf 

4: B = f(x), eq. 4.7 

5: R = R(x), section 4.3.3 

6: u, eq. 4.23

CHAPTER 

5 

THE COMMUNICATION PART 

Democracy means government by discussion, 

but it is only effective if you can stop people talking. 

Clement Attlee (1883 - 1967) 

This chapter deals with the communication network assurance part of the control, thus, 

first of all, the problem is formulated in the context of graph theory and sensor networks. 

Then, a standard potential field/ distributed algorithm is presented and extended using a 

dynamic – token based approach. Various policies which determine the way in which tokens 

are sent and kept are examined on the basis of local connectivity. 


Communications are critical in space environment since, once they are lost, the entire mis- 

sion could be lost, or at least out of control. Although SMAS are designed to be highly 

autonomous, trying to have a almost constant connection with the agents is at least reason- 

able and perhaps required, thus the communication network has to be deployed. 

Communications can be divided in two main group: 

� peer to peer, or P2P, in which the information is sent to one and only one agent per 

time; 

� broadband, in which the information is just spread everywhere, therefore every agent in 

the communication range can have access to it. 

Of course the former is less time consuming, given a certain amount of byte to be sent, 

or it could send more information, given the time interval. On the contrary, the latter is 

simpler since it does not involve complex mechanism to individuate where the other agents 

41

42 CHAPTER 5. THE COMMUNICATION PART 

are, although this is not a really hard problem. In fact agents know where the others are and 

with an electronic phasing antenna, also the mechanical difficulties could be overcome. 

Defining: 

� graph: a set of vertices V connected together by edges E; therefore the a graph G can 

be represented by a V and a connectivity matrix KE, which states which vertice is 

connected to which. Hence 

G ≡ (V, KE); (5.1) 

� k-connectivity: the graph G is k – connected if and only if for each vertice Vi there 

exists at least k connections. 

7 

6 

5 

5 

4 

5 

Figure 5.1: A 3-connected graph 

The control problem for the communication part can be written as: 

Given, in every time instant, the position of the explorers, built a k-connected graph with the 

communicators. 

Where k has to be chosen to assure a good fault tolerance property. 

5.2 Artificial Physics and Token based algorithm 

Dynamically deploying effective networks is difficult for a variety of reasons. First, the com- 

municators will not have a priori knowledge of where the explorers will go, nor of the environ- 

ment in which they must deploy. Second, to coordinate their deployment they must maintain 

communication with each other or coordinate without communication. Even if communica- 

tion between the agents is available, its use has to be minimized, both to make bandwidth 

5 

3 

5 

3

5.3. PROBLEM STATEMENT 43 

available for explorers to relay information back to a possible hub and to allow commands to 

be relayed to agents. Third, there is typically no clearly defined deployment stage, thus the 

ad hoc network needs to be maintained for the explorers while the communicators deploy to 

their positions. Finally, the agents may need to constantly rearrange to adjust to explorers 

movement or failed communicators, since typically they cannot provide coverage to the whole 

environment. 

A variety of approaches have been developed for this problem. In ad hoc networks and 

sensor networks, distributed algorithms, which can assure k - connected graphs [BR05], allow 

robust robot positioning [SL02] and provide good coverage [MS01], have been applied in rela- 

tively open environments. However those efforts largely ignore situations in which signals are 

impeded by obstacles, like walls or asteroids, or in which only a small dynamically changing 

part of environment needs coverage. 

Artificial Physics and Potential field are lightweight and robust way of positioning agents 

in a clustered and complex environment [HS02], often not requiring any communication to 

coordinate. However, potential fields are best suited for spreading agents out across an 

environment, not focusing them on dynamically changing areas. Hence, for this application, 

key extensions to Artificial Physics and potential fields were required to take advantage of 

their strengths while meeting problem constraints. 

The central idea of this work is to dynamically change the applicable potential fields based 

in the current overall needs of the team. If the potential fields can be appropriately varied, 

the agents will robustly move to locations where a connected network can be formed. 

The key to the dynamic potential field approach is to ensure that each communicators is 

influenced by appropriate fields at appropiate times. Specifically, the team must configure 

itself so that some communicators move near to the explorers, while others position themselves 

to relay massages to and from the hub. To achieve this, each agent sends out requests for 

other agents to connect it back to the hub or in the hub’s case, sends out requests to be 

connected to the network. These requests are in the form of tokens. When an agent receives 

a token it either keeps the token, adds a potential field corresponding to the request for 

support represented by the token, or passes the token on to another agent (which faces the 

same choice). By controlling the number of tokens each agent sends out, the number of links 

the team tries to form with the requester can effectively be controlled. The policy by which 

an agent decides to keep a token, and add the corresponding field, or pass the token on, 

dictates the effectiveness and the nature of the network. 

5.3 Problem Statement 

Let S = {S1, . . . , Sn, H} be a set of moving agents, Si, the explorers, and a hub H, and let 

C = {C 1 , . . . , C m } be a set of communication agents, the communicators. The basic aim is 

to position C to create a network which connects each Si to H. 

Si is assumed to be independent of C i but both can move at the same speed. Si and C i 

have a maximum range of communication, dc. It is assumed that every agent can sense where


the others are if they are within their communication range and agents can distinguish be- 

tween explorers and communicators. This may be done by overhearing messages broadcasted 

by other agents. Let S ⊆ S and C ⊆ C be the subsets of explorers and communicators a 

agent can sense respectively. 

Let 

Let x be the position of a generic agent at a given time, while the hub, H is stationary. 

Pi(Sk) = {Sk, C i , . . . , C q , H} 

be a possible communication path from Sk ∈ S to H, thus the distance between two consec- 

utive elements, pi, of Pi(Sk) is at most dc, 

|x(pi) − x(pi−1)| ≤ dc 

Two paths from the same explorer are different if they involve different communicators: 

Pi(Sk) �= Pj(Sk) ⇔ ∄ C i |(C i ∈ Pi(Sk)) ∧ (C i ∈ Pj(Sk)) 

Among all the Pi(Sk) which have at least a C i in common, a minimal path can be defined 

as the one which involves the minimum number of communicators: 

minPi(Sk) = Pi(Sk) if |Pi(Sk)|is minimum 

All the different paths from the same explorer can be grouped in the local subset of different 

paths, P (Sk): P (Sk) = {. . . , Pi(Sk), . . . } where 

∀Pi(Sk), Pj(Sk) | (i �= j) ∧ (Pi(Sk), Pj(Sk) ∈ P (Sk)) ⇒ 

⇒ Pi(Sk) �= Pj(Sk) 

Let the local connectivity c be c = |C| and let the connectivity explorer - Hub at a given 

time t, be 

Ki(t) = |P (Si)| 

Thus, Ki(t) = 0 means there are no communication paths from Si to H, thus Si is not 

connected. The primary goal of C is to avoid this happening. In Figure 5.2 the connectivity 

is 1 for S1 and 2 for S2. Let the global connectivity at a given time be 

K(t) is 1 in Figure 5.2. 

K(t) = min 

i Ki(t) 

A communicator is useful if it is on a minimal path. Let the used communicators subset 

be the subset: 

� 

� 

� 

U = minPi /S 

i 

i.e., the useful communicators. Define efficiency, E, as the ratio of useful communicators to 

total communicators: 

E = |U|/|C|

5.4. POSITIONING ALGORITHM 45 

H 

C 2 

C 3 

C 4 

Figure 5.2: An example of possible network which connects two explorers, S1 and S2 and 

communicators C 1 , . . . , C 4 to the hub H. The dashed line are communication links, the black 

line is a wall. 

In Figure 5.2, E = 3/4 , since C 1 is not on a minimal path. 

C 1 

Let < K > and < E > be the average global connectivity over time and the average 

efficiency over time respectively. 

Finally let v be the environment change rate, characterized as the maximum rate a com- 

municator has to move to prevent the network breakdown. This gives a rough measure of 

the environment difficult for C. 

The problem is to: 

5.4 Positioning Algorithm 

� 

� 

max min K(t) 

0≤t≤tmax 

The basic concept of AP and potential field is to overlap fields representing different influences 

on the agent. The agent then simply follows the gradient down the resulting field. The basic 

potential function, Jj, utilizes the Lennard - Jones formulation [SY99a], resulting in: 

Jj( � S, � C) = α � 

Si∈ � S 

� �6 � �12 dcfs dcfs 

−2 + 

+ β 

rCj Si rCj Si 

� 

Cq∈ � � �6 � �12 dcfc dcfc 

−2 + 

rCj Cq rCj Cq C 

S 1 

S 2 

(5.2) 

where � S ⊆ S and � C ⊆ C are the subsets of explorers and communicators which influence a 

agent, r C j Si and r C j C q are the relative distances between Cj and the agents in those subsets. 

The communication distance dc and the fs and fc coefficients determine the function shape. 

The coefficients α and β state whether to move further from explorers or communicators. If 

| � S| = 1, | � C| = 0, Jj( � S, � C) would have a minimum at a distance dcfs from Si; below that 

distance Jj( � S, � C) would increase not to prevent agents from being too near one another,


while above that distance it would increase to keep agents in the communication range. Once 

the potential function has been evaluated, the agent moves toward the local minimum. 

In the next sections different versions of this approach will be presented. What varies 

among the versions are � S and � C, i.e., which agents effect the potential field. First the basic 

potential field algorithm, in which � S = S and � C = C, i.e., where the potential field is 

influenced by all agents in sensor range. Second, a version where tokens are passed around 

the team, with the agent represented by the token being an influence in � S and � C. 

5.4.1 Standard algorithm 

In the basic algorithm, referred to as standard, � S = S and � C = C, thus every sensed agent 

influences the potential field shape. This leads to the agents spreading out the environment, 

since Jj( � S, � C) makes the relative distances among agents almost the same. The main problem 

with the standard approach is that, when the environment is large, spreading out is not an 

acceptable solution, since coverage can not be assured. 

5.4.2 Dynamic Potential Fields 

The key is to have the communicators move to the parts of the environment where explorers 

are, not just anywhere. Since in the standard approach the balance between attractive 

and repulsive force, i.e., the potential field gradient, determines the spreading pattern, it is 

reasonable that if communicators could cooperate they could turn off useless repulsive forces, 

which avoid them to move in critical position, and they could move in better locations. This 

approach, which dynamically changes the potential field to follow, will be called dynamic. 

The algorithm works as follows: every agent sends a message to N randomly chosen 

C i ∈ C, in which they request help maintaining the network. The information is packed in 

a token, τ = {x, c, explorer/communicator}. This could be accepted by the other agents or 

resent. 

By a intelligent choice based on the information on the token, this dynamic token potential 

field approach can overcome the problems of the standard approach. 

A simple example can show the token algorithm features and better performance over the 

standard approach. Let the situation be the one in Figure 5.3 (left), and let the communica- 

tors be in equilibrium, i.e., at minima in the potential field. If the S1 moves right and S2 up, 

in the standard approach, C 1 tries to follow both sensors breaking the network; moreover C 2 

and C 3 repulse each other and they can not help C 1 . In the dynamic approach C 1 follows 

S 1 and informs C 2 , which moves to help it maintain the network. 

The choice of what tokens to send and, moreover, what to do with them, is made by 

defining a policy. Three different policies has been defined, namely C policy (connectivity), 

TC policy (threshold connectivity) and RC policy (resend connectivity). In following sections 

each policy will be described. 

Each communicator follows the algorithm shown in Algorithm 3. For every time step, 

they form a token τ = {x, |C|, communicator} (Algorithm 3 line 1), then they keep sending

5.4. POSITIONING ALGORITHM 47 

H 

C 3 

C 2 

C 1 

Initial situation 

S 1 

S 2 

Standard 

H 

H 

C 2 

Token Algorithm 

Figure 5.3: Standard vs. Token algorithm behavior in an example. S1 and S2 are moving in 

opposite directions causing the network break in the former but not in the latter. 

and receiving tokens imax times and they group them in T . imax is fixed by the policy, if 

imax = 1 they simply delete the tokens they do not use. The determination of which tokens 

are important is made using a policy (Algorithm 3 line 12 - 16), which forms � S, � C and, if 

imax > 1, the token τ to be resent. Then they compute the potential function and move 

(Algorithm 3 line 18 - 19). 

C policy 

Agents with a low local connectivity are in the most critical positions of the network, thus 

they need more help. For that reason, C policy is based on local connectivity encouraging 

communicators to keep the tokens of low connected agents and move toward them. It works 

as follows: let NR be the number of received tokens, the communicator sorts the received NR 

tokens so that the first has the lowest c and the last the highest (Algorithm 4 - line 3), then 

it determines the subsets � S and � C using M ≤ NR agents, which the first M tokens refer to 

(Algorithm 4 - lines 5 - 11). Then it deletes the remaining tokens since for C policy imax = 1. 

TC policy 

The policy is the same of C policy, but includes a criterion for determine whether a com- 

municator is useful or not. Since each agent can not know if it is on a minimal path, this 

criterion is based on the number of received token, NR. If this is very high it means that 

the communicator is very useful, since a lot of agents request its help; if there was only a 

C 3 

C 3 

C 2 

C 1 

C 1 

S 2 

S 2 

S 1 

S 1


Algorithm 3 Token Algorithm 

1: τ = {x, |C|, communicator} 

2: imax ← Policy 

3: for i = 0 to imax do 

4: for k = 0 to N do 

5: Send(τ) → Random(C r ∈ C) 

6: end for 

7: T = {∅} 

8: repeat 

9: TA =GetToken ← (C r ∈ C ∨ S r ∈ S) 

10: T = T ∪ TA 

11: until (No more messages) 

12: if imax > 1 then 

13: ( � S, � C, τ) ← Policy(T ) 

14: else 

15: ( � S, � C) ← Policy(T ) 

16: end if 

17: end for 

18: J = J ( � S, � C) 

19: x ← x + (∇xJ ) dx 

Algorithm 4 C policy 

1: imax = 1 

2: NR ← |T | 

3: Sort(T ): c(T1) ≤ c(TNR ) 

4: � S = {∅}, � C = {∅} 

5: for j = 1 to M ≤ NR do 

6: if communicator(Tj) = 0 then 

7: S � = { S, � S(Tj)} 

8: else 

9: C � = { C, � C(Tj)} 

10: end if 

11: end for

5.5. POTENTIAL FIELD AND MOTION CONTROL 49 

agent connected to it, it would receive at least N tokens. On the other hand, if few tokens 

are received, the communicator is in an useless position. 

Therefore, when a comm receives less tokens than a specified threshold, it computes a 

different potential function where the repulsive part is neglected. This is done to allow that 

communicator to move closer to other communicators to reach, eventually, critical location 

in the network. 

The threshold can be different for explorers and communicators, in the sense that the 

repulsive part of the � S subset is neglected if the number of received tokens NR is less than 

TS, whereas the one of the � C subset is neglected if the number of tokens is less than TC. 

Thus the potential function is 

�Jj( � S, � C, NR) = α � 

Si∈ � S 

Algorithm 5 TC policy 

1: C - policy(from line 1 to line 11) 

2: Change J to � J ( � S, � C, NR) 

RC policy 

� �6 � �12 dcfs 

dcfs 

−2 + (tokens ≥ TS) 

+ 

rCj Si 

rCj Si 

β � 

� �6 � �12 dcfc 

dcfc 

−2 + (tokens ≥ TC) 

rCj Cq rCj Cq C q ∈ � C 

The policy is the same of C policy, but imax > 1, thus, after communicator determined � S 

(5.3) 

and � C it resends some of the tokens it did not use. It keeps on sending and receiving tokens, 

until i = imax, when this condition is satisfied, it computes the potential field using the last 

determined � S and � C. This allows to better determine � S and � C, since it spreads the infor- 

mation of where the low local connectivity areas are. In fact, in low local connectivity areas 

communicators could be not sufficient to satisfy all the helping requests, while in high local 

connectivity ones, communicators are less useful. By passing the not used tokens through 

the network this can be avoided making less useful communicators move where it is needed. 

The choice of which tokens have to be resent is made by the connectivity value. In 

particular, each communicator forms a new token, which carries not only its information, but 

also Q tokens, from M + 1 to M + Q (Algorithm 6 line 13 - 15). 

5.5 Potential Field and Motion Control 

In the previous sections the shaping of suitable potential fields have dictated where each 

spacecraft has to move, which is towards the minimum of their local potential field. This 

means, that giving a fixed spatial step at every time instant, each agent knows the location 

in space which has to be reach. The key issue is, then, how to actually reach that location, 

since the environment will be perturbed and the model/sensors will be affected by errors.


Algorithm 6 RC policy 

1: imax = number > 1 

2: NR ← |T | 

3: Sort(T ): c(T1) ≤ c(TNR ) 

4: � S = {∅}, � C = {∅} 

5: for j = 1 to M ≤ NR do 

6: if communicator(Tj) = 0 then 

7: S � = { S, � S(Tj)} 

8: else 

9: C � = { C, � C(Tj)} 

10: end if 

11: end for 

12: τ = {x, |C|, communicator} 

13: for j = M + 1 to Q ≤ NR − M do 

14: τ = {τ, x(Tj), c(Tj), communicator/explorer(Tj)} 

15: end for 

Although this control will not develop here, it seems reasonable that a suitable feedback 

robust control can be an answer, since the fixed spatial step is usually small compared to the 

global dynamics. That feedback control will be not different from a relative motion control 

in the fashion of [MY07], among the others.

CHAPTER 

6 

PERTURBATION MODELS 

The true felicity of life is to be free from anxiety and perturbations 

to understand and do our duties to God and man, and to enjoy the present without any 

serious dependence on the future 

Lucius Annaeus Seneca (4 BC - 65) 

Here the perturbation models are analyzed and derived, both for Earth environment, drag, 

J2, J22, and for Asteroid main belt. 

6.1 Formation Flying 

The Prisma mission involves sun-synchronous orbits, it involves small satellites and a 700 km 

altitude perigee. For these reasons the significative perturbations are 

� atmospheric drag effect; 

� gravitational effects, namely J2 and J22. 

6.1.1 Atmospheric drag effect 

The acceleration due to atmospheric drag, aD, can be compute as [PS06] 

aD = − 1 S 

ρ 

2 m CDv 2 ˆv (6.1) 

where ρ is the density, S is the surface, CD is the drag coefficient, m the mass, v the velocity 

modulus and ˆv is the velocity unit vector. 

51

52 CHAPTER 6. PERTURBATION MODELS 

6.1.2 Gravitational effects, J2 and J22 

The external geo-potential function at any point P specified by the spherical coordinates 

(r, δ, λ) can be expressed as[Cho02], [PS06] 

U = − κ 

� 

1 − 

r 

∞� 

n=2 

Jn 

� �n RE 

r 

Pn(sin δ) + 

∞� 

n� 

n=2 m=1 

Jnm 

� � � 

n 

RE 

Pnm(sin δ) cos m(λ − λT ) 

r 

where κ is the planetary constant, RE is the Earth radius, and Pi, Pij suitable spherical 

functions. 

For the J2 term, considering the spherical triangle ABC, 

C 

A 

λ 

θ 

i 

Figure 6.1: Spherical Triangle. 

sin δ = sin i sin θ 

Thus substituting in the geo-potential function and calculating the partial derivatives in the 

orbital frame 

fr = ∂U 

∂r 

fθ = 1 ∂U 

r ∂θ 

R 

= −3κJ2 

2 E 

r4 B 

δ 

� 2 2 � 

1 − 3 sin i sin θ 

R 

= −3κJ2 

2 E 

r4 sin2 i sin θ cos θ 

fh = 1 ∂U 

r sin θ ∂i 

= −3κJ2 

r4 sin i cos i sin θ 

As regards J22, the term which needs to be written is 

R 2 E 

P22(sin δ) cos 2(λ − λT ) = 3(1 − sin 2 δ) cos 2(λ − λT ) 

2 

(6.2) 

(6.3)

6.1. FORMATION FLYING 53 

Considering the spherical triangles ABC: 

Thus 

The second part can be written as 

but 

sin δ = sin i sin θ 

sin λ = 

cos i sin θ 

cos δ 

P22(sin δ) cos 2(λ − λT ) = 3(1 − sin 2 i sin 2 θ) cos 2(λ − λT ) 

cos 2(λ − λT ) = 1 − 2 sin 2 (λ − λT ) = 1 − 2(sin λ cos λT − cos λ sin λT ) 2 = 

= 1 − 2(sin 2 λ cos 2 λT + cos 2 λ sin 2 λT − 2 sin λ cos λ sin λT cos λT ) 

sin 2 λ = cos2 i sin 2 θ 

1 − sin 2 i sin 2 θ 

and, by the spherical triangles ABC 

Hence 

cos 2(λ − λT ) = 

or also 

thus 

cos 2(λ − λT ) = 

sin λ cos λ = 

cos i sin θ 

cos δ 

cos θ 

cos δ 

= cos i sin θ cos θ 


= 1 − 2(sin 2 λ cos 2 λT + cos 2 λ sin 2 λT − 2 sin λ cos λ sin λT cos λT ) = 

= 1 − 

= 1 − 2(sin 2 λ cos 2 2λT + sin 2 λT − sin λ cos λ sin 2λT ) = 

= 1 − 2 

2 


P22(sin δ) cos 2(λ − λT ) = 

� cos 2 i sin 2 θ 

1 − sin 2 i sin 2 θ cos2 2λT + sin 2 λT − 

cos i sin θ cos θ 

1 − sin2 i sin2 sin 2λT 

θ 

� cos 2 i sin 2 θ cos 2 2λT + sin 2 λT (1 − sin 2 i sin 2 θ) 

� 

− cos i sin θ cos θ sin 2λT ) 

= 3(1 − sin 2 i sin 2 θ − 2 � cos 2 i sin 2 θ cos 2 2λT + sin 2 λT (1 − sin 2 i sin 2 θ) 

Rearranging the similar terms 

P22(sin δ) cos 2(λ − λT ) = 

− cos i sin θ cos θ sin 2λT )) 

= 3(1−2 sin 2 λT −sin 2 i sin 2 θ(1−2 cos 2 2λT −2 sin 2 λT )−2 sin 2 θ cos 2 2λT +2 cos i sin θ cos θ sin 2λT ) = 

= 3(c1 + c2 sin 2 i sin 2 θ + c3 sin 2 θ + c4 cos i sin θ cos θ)

54 CHAPTER 6. PERTURBATION MODELS 

Thus, in the orbital frame 

fr = ∂U 

∂r 

fθ = 1 ∂U 

r ∂θ 

fh = 1 ∂U 

r sin θ ∂i 

6.2 Asteroid belt 

R 

= 3κJ22 

2 E 

r4 3(c1 + c2 sin 2 i sin 2 θ + c3 sin 2 θ + c4 cos i sin θ cos θ) 

R 

= −κJ22 

2 E 

r4 3(2c2 sin 2 i sin θ cos θ + 2c3 sin θ cos θ + c4 cos i cos 2θ) 

R 

= −κJ22 

2 E 

r4 3(2c2 sin i cos i sin θ − c4 sin i cos θ) 

(6.4) 

The perturbation model for the asteroid belt can be derived considering the perturbative 

gravitational forces of the most massive objects. As shown in chapter 2 this model will 

include the most massive 10 asteroids. Thus the perturbative field in each point x induced 

by each asteroid i can be written as 

w(x) = − � 

i 

mi 

||x − xi|| 3 (x − xi) (6.5) 

The w has to be written in the orbital frame with a suitable rotation matrix. 

In Figure 6.2, the orbit of the selected asteroids. 

1 

0 

−1 

−6 

−4 

−2 

0 

2 

4 

Figure 6.2: First ten asteroid orbits, dimensions in AU. 

6 

The final distances between spacecrafts and asteroids will be in the order of 0.01 AU, 

because below that distance it is more convenient a feedback linearized approach for the 

control; therefore the maximum expected perturbation is in the order of 10 −8 m/s 2 , which 

can be handle easily with solar sail propulsion ∼ 10 −6 m/s 2 , as in [1]. 

Since the uncertainties on the masses are quite important, in closer rendezvous, a method 

to handle this problem has to be used and applied in order to make a correct valuation of the 

−4 

−2 

0 

2 

4

6.2. ASTEROID BELT 55 

maximum perturbative force, thus the correct design of the control law. The Taylor series 

method of [Ber99], [Fer08] could be chosen.

56 CHAPTER 6. PERTURBATION MODELS

CHAPTER 

7 

RESULTS 

Telling the future by looking at the past assumes that conditions remain constant. 

This is like driving a car by looking in the rearview mirror. 

Herb Brody (1957 - ) 

The main results are here shown. First, the goal manager behavior in the example pre- 

sented in chapter 3; second the non linear Lyapunov control in several tests in the framework 

of Prisma mission. Third the communication network deployment in two significative sce- 

nario; finally an asteroid belt example, in which the features of the whole control algorithm 

are shown. 


In this chapter the main results are presented, in particular 

� a goal manager simulation for the example shown in chapter 3; here the fully scalable 

behavior of task assignment is shown and the sharing information mechanism tested; 

� several tests for the non linear Lyapunov controller in the framework of Prisma mis- 

sion; here the algorithm robustness is proved using, among the others, a Montecarlo 

simulation; 

� two wall scenario, 2D and 3D, for the communication network deployment; here the 

behavior of token based algorithm is shown and proved to out-perform the standard 

one; 

� an asteroid belt scenario, in which the controls are coupled and shown. 

57

58 CHAPTER 7. RESULTS 

7.1.1 Scalability proof 

First of all the global agent algorithm has to be proved to be scalable, thus it has to be 

proved that the computational time for each agent does not increase increasing the agent 

number. This is quite straightforward, since the single pieces of the algorithm are completely 

distributed, hence the agent number do not affect the computation time. It has to be note 

that the communication and information sharing mechanism, in particular the number of 

messages each agent receives depends on the environment density and not directly on the 

number of agents. Since the scenario environments are supposed to be sparse, there is no 

scalability issue and thus the global algorithm is scalable. 

7.2 Goal Manager example 

As an example of the properties of the Goal Manager, the example of section 3.4.1, will be 

exploited. Let the environment be closed in a box, [−1, 1] × [−1, 1] × [−1, 1], as in Figure 

7.1, and let the red dots be three different asteroids, with a fixed initial T ∈ R 3×3 matrix, 

where each tij = 1/3. Let N be the agent number randomly located in the box and let C be 

a R N×3 random capacities matrix. C is normalized thus the global team capacities are equal 

to 1. Let dc = 1 be the communication/sensing range among the agents. Moreover let the 

agents move towards the chosen asteroid using straight lines. 

As a performance index can be chosen the final T matrix on the initial one, which repre- 

sents how much the goals have been satisfied. In Figure 7.1, the dynamical behavior of a set 

of N = 50 agents is depicted; the agents are the black dots, while the gray dashed lines are 

the communication links. 

In Figure 7.2, the first column of the final T on the initial one, is shown for 100 simulation 

runs. This represents how well the Goal Manager can assign a target to the agents, the lower 

the values the better the Goal Manager is; in particular a positive tij/(tij)0 means that the 

goal still needs tij/(tij)0 capacity to be fulfilled, whereas a negative tij/(tij)0 results in a 

excess of capacity. The choice of the first column does not affect the results evaluation, since 

every column is almost the same starting with a random choice on C. In Figure 7.2, the line 

thickness represents the results standard deviation. 

The shown results assure a good Goal Manager performance for N, which can fulfill the 

tasks within a 10% errors. This can be acceptable, since in the real case, the agents may 

have more capacities than necessary. Moreover, it has to be noted that the agent number 

is not significant per se but it is has to be compared to the initial connectivity, i.e. the 

communication distance. Few agents with a long communication distance could perform 

better than lot of agents with a very limited communication range.

7.2. GOAL MANAGER EXAMPLE 59 

1 

0.5 

0 

−0.5 

−1 

1 

0.5 

0 

−0.5 

−1 

1 

0.5 

0 

−0.5 

−1 

1 

1 

1 

0.5 

0.5 

0.5 

0 

0 

0 

−0.5 

−0.5 

−0.5 

−1 

−1 

(a) τ = 1 

−1 

−1 

(c) τ = 5 

−1 

−1 

(e) τ = 9 

−0.5 

−0.5 

−0.5 

0 

0 

0 

0.5 

0.5 

0.5 

1 

1 

1 

1 

0.5 

0 

−0.5 

−1 

1 

0.5 

0 

−0.5 

−1 

1 

0.5 

0 

−0.5 

−1 

1 

1 

1 

0.5 

0.5 

0.5 

0 

0 

0 

−0.5 

−0.5 

−0.5 

−1 

−1 

(b) τ = 3 

−1 

−1 

(d) τ = 7 

−1 

−1 

(f) τ = 11 

Figure 7.1: Dynamical behavior of a set of N = 50 agents: the agents are the black dots, the 

asteroids are the red dots, while the gray dashed lines are the communication links. 

−0.5 

−0.5 

−0.5 

0 

0 

0 

0.5 

0.5 

0.5 

1 

1 

1


t ij /(t ij ) 0 value 

0.25 

0.2 

0.15 

0.1 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

Goal Manager performances 

First Asteroid 

Second Asteroid 

Third Asteroid 

30 40 50 60 70 80 90 100 

Agent number, N 

Figure 7.2: Goal Manager performances, the line thickness represents the results standard 

deviation. 

7.3 Formation flying scenarios 

The Prisma mission has been used to test the physical part of the distributed algorithm, in 

particular it has been supposed that the Target spacecraft has to rendezvous with the Main 

one in operative orbit, which is the real case, from the initial orbit in which it is left by the 

launcher. In fact, the spacecrafts have impulsive rocket motors, but here low thrusters have 

been considered. At the beginning the nominal Main orbital parameters are [GM07] 

where RE is the Earth radius. 

ā ē ī ¯ω ¯ Ω ¯ θ 

RE+ 700 km 0 98.2π/180 0 0 0 

Table 7.1: Main initial orbital parameters. 

7.3.1 Unperturbed and perturbed results 

Three cases have been tested, Table 7.2 

The simulations parameters are: R = 1E − 4, ∆tI = 60 s, ∆tC = 60 s, where I and 

C mean Integration time step and Control time step, Q = diag{1/a 2 e, 1, 1, 10, 10, 1}, with 

ae = (aT + aM)/2, being aT the Target satellite semiaxis and aM the Main one. 

The drag due to the atmosphere and gravitational perturbations due to J2 and J22 have

7.3. FORMATION FLYING SCENARIOS 61 

a e i ω Ω θ 

1 st test – RE+ 600 km ē ī ¯ω ¯ Ω π/2 

2 nd test – – RE+ 500 km ē ī ¯ω ¯ Ω π/2 

3 rd test – • RE+ 400 km ē ī ¯ω ¯ Ω π/2 

Table 7.2: Tests initial conditions. 

been considered and since the actual orbit of Main has to be a sun-synchronous, the precession 

of the ascending node has not been controlled. 

Since the perturbation magnitude is ∼ 1 mm/s 2 and the control should be at least ∼ 1 

cm/s 2 , for a 100 Kg spacecraft, the use of low thrust electrical motors is, in fact, unpractical 

and perhaps infeasible. Here, only the control performances in terms of relative distances 

and Lyapunov function have been taken into account, leaving the thruster detailed design for 

further developments. 

In Figure 7.3, the unperturbed relative distances – in-track/cross-track/radial – are shown 

for the different tests. The chosen time interval is t ∈ [18, 21]h. While in Figure 7.4, the 

perturbed relative distances – in-track/cross-track/radial – are shown for the different tests. 

The chosen time interval is t ∈ [18, 21]h. 

From the graphs appear that the control can lead the Target spacecraft to a close approach 

with the Main spacecraft. The relative distance is within less than 5 km and this is reasonable 

to start a close rendezvous approach via linear feedback control. 

In Figure 7.5 the variation of Normalized Lyapunov functions for the unperturbed and 

perturbed case in the third test. The two curve cannot be distinguished. The normalization 

is made on the initial value. 

7.3.2 Montecarlo analysis 

As a final test on the algorithm quality a Monte Carlo analysis has been performed using, as 

state space, the uncertainties on the initial position of the Target spacecraft as 


RE+ 700km ± 100 km ē + 0.1 ī ± 10 ◦ ¯ω ± 10 ◦ ¯ Ω ± 10 ◦ π/2 ± 10 ◦ 

Table 7.3: Monte Carlo initial condition intervals. 

In Figure 7.6 the relative distance results in the time frame t ∈ [20, 21]h. There, to make 

the graphs readable, only discrete instant trajectories are shown; for each simulation, the 

continuous trajectory would start from the top and it would stop at the bottom, as the one 

shown in black. Thus each simulated trajectory passes through the 0 km in track point. The 

results show a good performance of the control algorithm, allowing a close approach of less 

than 5 km.


Cross track [km] 

Radial [km] 

0.4 

0.3 

0.2 

0.1 

0 

−0.1 

−0.2 

Relative unperturbed distance for t ∈ [18, 21] h 

−0.3 

25 30 35 40 45 

In track [km] 

50 55 60 65 

−0.2 

−0.25 

−0.3 

−0.35 

−0.4 

−0.45 

−0.5 

Relative unperturbed distance for t ∈ [18, 21] h 

−0.55 

25 30 35 40 45 

In track [km] 

50 55 60 65 

Figure 7.3: Results: Close approach of the spacecrafts for different initial altitude: – 600 km, 

– – 500 km, – • 400 km, unperturbed case.



Radial [km] 

2 

1.5 

1 

0.5 

0 

−0.5 

−1 

−1.5 

Relative perturbed distance for t ∈ [18, 21] h 

−2 

−100 −50 0 

In track [km] 

50 100 

2 

1.5 

1 

0.5 

0 

−0.5 

−1 

Relative perturbed distance for t ∈ [18, 21] h 

−1.5 

−100 −50 0 

In track [km] 

50 100 

Figure 7.4: Results: Close approach of the spacecrafts for different initial altitude: – 600 km, 

– – 500 km, – • 400 km, perturbed case.


Normalized Lyapunov function J 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 

0 5 10 15 20 25 

Time [h] 

Figure 7.5: Normalized Lyapunov Function



Radial [km] 

2 

1.5 

1 

0.5 

0 

−0.5 

−1 

Relative distance for t ∈ [20, 21] h 

−1.5 

−40 −20 0 20 40 

In track [km] 

60 80 100 120 

2 

1.5 

1 

0.5 

0 

−0.5 

−1 

Relative distance for t ∈ [20, 21] h 

−1.5 

−40 −20 0 20 40 

In track [km] 

60 80 100 120 

Figure 7.6: Relative positions reached after 20 h. The black line is a representative trajectory.


7.4 Communication network deployment 

Two main scenarios have been analyzed to test the performance of the algorithm, namely 

� a 2D wall scenario; 

� a 3D wall scenario. 

Other, more complex, scenarios can be found in [SS08]. 

7.4.1 2D Environment 

This apparently simple wall scenario tests the basic functionality of the algorithms. If the 

communicators do not move north of the wall, connectivity will be lost. Thus, spreading out 

is an inadequate strategy. 

The initial and a final state are shown in Figure 7.4.1, notice the explorers, black dots, 

move from west to east. In Figures 7.8 and 7.9 the obtained average results for 200 simulation 

runs are presented. The parameters for the experiments were: 

� for the scenario: |S| = 6, dc = 10, v = 0.016; 

� for the potential function: fs = 1/2, fc = 1/2, α = 100 |C|/|S|, β = 1; 

� for the policies: N = 2, (M, Q) = ([NR/2], 0) for C and TC policies, N = 2, (M, Q) = 

(1, 1) for RC policy, TS = 4, TC = 3, imax = 3. 

In the graphs B means Baseline, C C policy and so on. Bars are the value of average 

global connectivity of average efficiency. Black bars on the top of the bars are the standard 

deviations. Lines in the bars are the final value of K. 

(a) Initial State (b) Final State 

Figure 7.7: Simple wall simulations: black dots are explorers, white dots the communicators, 

dashed lines communication links, the triangle the hub. The box ticks distance is 5. 

Both < K > and < E > are significantly higher using the token algorithm than in the 

baseline. Moreover, note that in the |C| = 8, case the baseline can not assure a final global 

connectivity greater than one, while the other algorithms can. TC policy performs best 

here, which was expected since it makes communicators move in the north side of the wall 

neglecting the repulsive force of communicators already there. This was less decisive for 

14 communicators because with an increasing number of communicators resending is more 

important since it allows a better understand of critical locations.

7.4. COMMUNICATION NETWORK DEPLOYMENT 67 

Average Global Connectivity 

8 

7 

6 

5 

4 

3 

2 

1 

0 

B C TC RC B C TC RC 

|C| = 8 |C| = 14 

Figure 7.8: < K > for different |C|: B means Baseline, C C - policy and so on. Black bars 

on the top of the bars are the average standard deviations, whereas black lines in the bars 

are the final value of K. 

Average Efficiency 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 

B C TC RC B C TC RC 

|C| = 8 |C| = 14 

Figure 7.9: < E > for different |C|: B means Baseline, C C - policy and so on. Black bars 

on the top of the bars are the average standard deviations.


7.4.2 3D Environment 

This 3D wall scenario tests the basic functionality of the algorithms in an environment closer 

to the real one, since the wall could be seen as an asteroid. Moreover to test code robustness, 

agents were disabled at a rate, kR. The initial and a final state are shown in Figure 7.4.2, 

notice the explorers, black dots, move from west to east. In Figures 7.11 and 7.12 the obtained 

average results for 50 simulation runs are presented. The parameters for the experiments were: 

� for the scenario: |S| = 3, |C| = 12, dc = 10, v = 0.02; 


� for the policies: N = 2, (M, Q) = ([NR/2], 0) for C policy, N = 2, (M, Q) = (1, 1) for 

RC policy, imax = 3. 




10 

5 

0 

−5 

−10 

−10 

−5 

0 

5 

10 −10 

(a) Initial State 

−5 

0 

5 

10 

10 

5 

0 

−5 

−10 

−10 

−5 

0 

5 

10 −10 

(b) Final State 

Figure 7.10: 3D wall simulations: black dots are explorers, white dots the communicators, 

dashed lines communication links, the triangle the hub. 

Both < K > and < E > are significantly higher using the token algorithm than in the 

baseline. Moreover, note that the baseline can not assure a final global connectivity greater 

than one, while the other algorithms can. The code robustness is proven. 

−5 

0 

5 

10

7.4. COMMUNICATION NETWORK DEPLOYMENT 69 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 

B C RC B C RC B C RC 

Kill rate = 0.00 Kill rate = 0.16 Kill rate = 0.33 





1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 






7.5 Asteroid belt scenario 

An Asteroid belt scenario has been developed to test both the single controls’ performance 

and how the two spacecraft types behave together. The scenario consists in three different 

explorers which have to follow two asteroids, one communicator Hub and a set of communi- 

cators; the initial data for the explorers, hub and asteroids are summarized in Table 7.5. 

a [AU] e [ ◦ ] i [ ◦ ] ω [ ◦ ] Ω [ ◦ ] θ [ ◦ ] Target 

1 st explorer 2.75 0 2 0 0 0 1 st asteroid 

2 nd explorer 2.8 0 2 0 0 0 1 st asteroid 

3 rd explorer 2.775 0 2 0 0 0 2 nd asteroid 

Hub 2.8 0 2 0 0 0 target orbit 

1 st asteroid 2.85 0 2 0 0 -2 

2 nd asteroid 2.8 0 2 0 0 -2 

target orbit 2.9 0 2 0 0 -2 

It has been assumed that 

Table 7.4: Explorers/Hub/Asteroids initial conditions. 

� the targets were pre-imposed by the goal manager and they were minor asteroids, here 

arbitrarily located, as in [1]; 

� the spacecrafts had a mass of 1 kg and solar sail propulsion system as in [1]; 

� the time frame was 3 years as in [2]. 

7.5.1 The physical part 

To obtain smoother control results R is considered as a function of the state as 

� 

||δx|| 

R = αRI3 

||δx0|| 

where δx0 is the initial value of δx. The other simulations parameters are: αR = 0.25,∆tI = 

1 day, ∆tC = 1 day, where I and C mean Integration time step and Control time step, 

Q = diag{1/a 2 e, 1, 1, 1, 1, 1}, with ae = (aT + aM)/2, being aT the Target orbit semiaxis and 

aM the spacecraft one. 

The results are depicted in Figures 7.13 - 7.14. 

In Figures 7.15 - 7.16 the control behavior for the analyzed spacecrafts. Note that the 

control effort is almost constant and, moreover, always positive for fr, which is a necessary 

condition for solar sail propulsion. The sail area with this control is ∼ 20 m 2 , which is in the 

order of the one proposed by NASA, [1].

7.5. ASTEROID BELT SCENARIO 71 

z [AU] 

2 

Asteroid 

Hub 

Explorers 

1 

y [AU] 

0 

−1 

−2 

−2 

−1 

0 

x [AU] 

Figure 7.13: Trajectory of Asteroids/Hub/Explorers in a sun-centered inertial reference 

frame. Asteroids in blue, Hub in red, Explorers in black. 

y [AU] 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

−0.3 

−0.35 

Relative positions from Hub [AU] 

−0.1 0 0.1 0.2 

x [AU] 

0.3 0.4 

Figure 7.14: Trajectory of Asteroids/Hub/Explorers in a Hub-centered inertial reference 

frame. Asteroids in blue, Hub in red, Explorers in black. 

1 

2


Control [nN] 

Control [nN] 

10 

8 

6 

4 

2 

0 

10 

8 

6 

4 

2 

0 

HUB Control 

0 0.5 1 1.5 

Time [y] 

2 2.5 3 

0 0.5 1 1.5 

Time [y] 

2 2.5 3 

Normalized Lyapunov Function J 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

s/c 2 Control 

f r 

f θ 

f h 

f r 

f θ 

f h 

Control [nN] 

Control [nN] 

10 

8 

6 

4 

2 

0 

10 

8 

6 

4 

2 

0 

s/c 1 Control 

0 0.5 1 1.5 

Time [y] 

2 2.5 3 

s/c 3 Control 

0 0.5 1 1.5 

Time [y] 

2 2.5 3 

Figure 7.15: Control effort for Hub/Explorers. 

Normalized Lyapunov Function 

J HUB 

J s/c 1 

J s/c 2 

J s/c 3 

0 0.5 1 1.5 

Time [y] 

2 2.5 3 

Figure 7.16: Normalized Lyapunov Function for Hub/Explorers. 

f r 

f θ 

f h 

f r 

f θ 

f h


7.5.2 The communication part 

The communicators were initially spread randomly in a box, [−0.05, 0.05]×[−0.05, 0.05]×[0, 0] 

AU , around the Hub; in Figure 7.5.2, notice the initial and a final graph configuration. In 

Figures 7.18 - 7.19 the obtained average results for 50 simulation runs are presented. The 

parameters for the experiments were: 

� for the scenario: |S| = 4, |C| = 6 − 10, dc = 0.25 AU, v = 0.8; 


� for the policies: N = 2, (M, Q) = ([NR/2], 0) for C policy, N = 2, (M, Q) = (1, 1) for 

RC policy, imax = 3. 




y [AU] 

0.1 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

−0.3 

−0.35 

−0.4 

−0.2 −0.1 0 0.1 0.2 

x [AU] 

0.3 0.4 0.5 0.6 

(a) Initial State 

y [AU] 

0.1 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

−0.3 

−0.35 

−0.4 

−0.2 −0.1 0 0.1 0.2 

x [AU] 

0.3 0.4 0.5 0.6 

(b) Final State 

Figure 7.17: Asteroid belt scenario: black dots are explorers, white dots the communicators, 

dashed lines communication links, the triangle the hub. 

Both < K > and < E > are higher using the token algorithm than in the baseline. 

The differences among the algorithms are slight here because the environment is completely 

without obstacles, but, even so, only the RC policy can assure a final connectivity greater 

then one. The algorithms behavior to a change of |C| depends on the interaction between 

Hub/Explorers and Communicators, and it is not monotonic here, because the explorers 

relative velocity varies with time. In fact in more clustered environments the communicators 

receive less tokens from explorers relatively to those of other communicators, thus they are 

less reactive to rapid velocity increase. To avoid such a situation a proactive algorithm could 

be used as in [SS08]. 

As a final remarks on the scenario, a communication distance of dc = 0.25 AU can assure 

a data amount in the order of 1 MB per day – e.g. a high resolution image – with a 20 W, 1



8 

7 

6 

5 

4 

3 

2 

1 

0 


|C| = 6 |C| = 8 |C| = 10 





1 

0.99 

0.98 

0.97 

0.96 

0.95 

0.94 

0.93 

0.92 

0.91 

0.9 


|C| = 6 |C| = 8 |C| = 10 




m diameter, Ka antenna. The data rate can be improved considering different Hub trajectory 

and thus less communication range.

76 CHAPTER 7. RESULTS

CHAPTER 

8 

FINAL REMARKS 

A positive attitude may not solve all your problems, 

but it will annoy enough people to make it worth the effort. 

Herm Albright (1876 - 1944) 

The conclusions and some remarks are included in this chapter. The future developments 

are also outlined. 

8.1 Thesis final remarks 

The thesis results show good algorithm performances in terms of reliability, robustness and 

wide application areas; in particular 

� the goal manager can handle fairly complex scenario with multiple tasks and capabili- 

ties; 

� the non linear Lyapunov control framework is rich enough to be both robust and a good 

starting point to tune and optimize the trajectory control; 

� in the communication network deployment, the proposed token based algorithm out- 

performs the standard one. 

8.2 Future developments 

8.2.1 Decisional Level 

Reactive Goal Manager has been investigating for years in the computer science field, mostly 

using potential fields or other techniques. As it appears clear, to better develop the one of 

this thesis, several things have to be studied as 

77

78 CHAPTER 8. FINAL REMARKS 

� the projection operator has to be fully defined in various applicative contexts; 

� the conflict detection and solver has to be developed; one way could be the use a 

distributed scheduling [BL08]; 

� the information sharing mechanism has to be improved, perhaps using a token based 

approach, and low connected graphs have to be taken into account. 

8.2.2 Physical part 

The trajectory control, using a long dynamic approach, has been fully characterized, even if 

several parameters could be tuned to obtained some sort of optimal behavior. On the other 

hand, the short period dynamic control has to be developed, in particular 

� the approach has to be decided, focusing on reactive algorithm, like [MY07] and it has 

to be extended properly; 

� the long and short period have to be integrated. 

8.2.3 Communication part 

The communication network deployment has been studied and assured using a token based 

algorithm. Mainly the following points are open issues 

� find the optimal policy tuning the different parameters; 

� integrate the feedback control law for motion control; 

8.2.4 Missions 

Real mission scenarios have to be tested, as 

� large formation flight mission, such as TPF, in environment as a Lagrangian point; 

� asteroid scenario with goal manager integration.

1 Introduction 

REFERENCES 

[Bro99] R. A. Brooks. Cambrian Intelligence, The Early History of the New AI. The MIT Press, 1999. 

[D’A04] P. D’Arrigo. The APIES Mission. Technical report, ESA, 2004. 

[KE01] J. Kennedy and R. C. Eberhart. Swarm Intelligence. The Morgan Kaufmann, 2001. 

[LLJB07] P. R. Lawson, O. P. Lay, K. J. Johnston, and C. A. Beich. Terrestrial Planet Finder Interferometer 

Science Working Group Report. Technical report, JPL - NASA, 2007. 

[RHRT06] C. A. Rouff, M. G. Hinchey, J. L. Rash, and W. Truszkowski. Verifying Future Swarm-Based 

Missions. SpaceOps 2006 Conference AIAA 2006-5555, 2006. 

[Syc98] K. Sycara. Multiagent systems. AI - Magazine, 1998. 

[Woo02] M. Wooldridge. An introduction to MultiAgent Systems. Wiley, 2002. 

Web sites 

[1] http://www.weblab.dlr.de/rbrt/GpsNav/Prisma/Prisma.html (2008) 

[2] http://www.esa.it (2007) 

[3] http://planetquest.jpl.nasa.gov/TPF (2007) 

[4] http://ants.gsfc.nasa.gov (2007) 

2 Formulation of the problem 

[BL92] B. Barraquand, J. Langlois and J. Latombe. Numerical Potential Field Techniques for Robot Path 

Planning. IEEE Transactions on Systems, Man, and Cybernetics, 22(2), March – April 1992. 

[BP03] S. Brueckner and H. V. D. Parunak. Resource-aware exploration of the emergent dynamics of 

simulated systems. Proceedings of Autonomous Agents and Multi Agent Systems (AAMAS), pages 

781–788, 2003. 

[CS00] C. Albee A. Battel S. Brace R. Burdick G. Burr P. Dippoey D. Lavell J. Leising C. MacPherson 

D. Menard W. Rose R. Sackheim R. Casani, J. Whetsler and A. Schallenmuller. Report on the 

Loss of the Mars Polar Lander and Deep Space 2 Missions. Jet Propulsion Laboratory, California 

Institute of Technology, (JPL D-18709), 2000. 

[d’I04] M. d’Inverno. The dMARS Architecture: A Specification of the Distributed Multi-Agent Reasoning 

System. Autonomous Agents and Multi-Agent Sytems (AAMAS), pages 5–53, 2004. 

79

80 REFERENCES 

[DM06] E. D’Amico, S. Gill and O. Montenbruck. Relative Orbit Control Design For The Prisma Formation 

Flying Mission. AIAA Guidance, Navigation, and Control Conference, August 21–24 2006. 

[EK06] G. H. Elkaim and R. J. Kelbley. A Lightweight Formation Control Methodology for a Swarm of 

Non-Holonomic Vehicles. IEEE, 2006. 

[GC00] S. Ge and Y. Cui. Path Planning for Mobile Robots Using New Potential Functions. 3rd Asian 

Control Conference, July 4–7 2000. 

[GM07] S. Gill, E. D’Amico and O. Montenbruck. Autonomous Formation Flying for the PRISMA Mission. 

Journal of Guidance, Control and Dynamics, 44(3):671 – 681, May – June 2007. 

[Hol88] W.M.L. Holcombe. X-Machines as a Basis for System Specification. Software Engineering, 3(2):69– 

76, 1988. 

[HR01] J. Hinchey, M. Rash and C.A. Rouff. Verification and Validation of Autonomous Systems. NASA, 

2001. 

[Kin98] J. R. Kiniry. The Specification of Dynamic Distributed Component Systems. Master’s thesis, 

California Institute of Technology, 1998. 

[LAM02] A. Li, L. Martinoli and Y. S. Abu-Mostafa. Emergent Specialization in Swarm Systems. IDEAL, 

LNCS 2412, pages 216–266, 2002. 

[Ld95] M. Luck and M. d’Inverno. A Formal Framework for Agency and Autonomy. pages 254–260, 1995. 

AAAI Press/MIT Press. 

[LJGM05] K. Lerman, C. Jones, A. Galstyan, and M. J. Matari c. Analysis of Dynamic Task Allocation in 

Multi-Robot Systems. University of Southern California, Los Angeles, 2005. 

[PG05] B. Persson, S. Jacobsson and E. Gill. Prisma - Demonstration Mission For Advanced Rendezvous 

And Formation Flying Technologies And Sensors. IAC, 56th International Astronautical Congress, 

October 17–21 2005. 

[PV97] H. V. D. Parunak and R. Vanderbok. Managing Emergent Behaviour in Distributed Control 

Systems. Proceedings of ISA-Tech’97, 1997. 

[Rei92] M.B. Reid. Path Planning Using Optically Computed Potential Fields. NASA, Ames, 1992. 

[RH05] W. Rash J. Rouff, C.A. Truszkowski and M. Hinchey. A Survey of Formal Methods for Intelligent 

Swarms. NASA Goddard Space Flight Center, 2005. 

[RHRT06] C. A. Rouff, M. G. Hinchey, J. L. Rash, and W. Truszkowski. Verifying Future Swarm-Based 

Missions. SpaceOps 2006 Conference AIAA 2006-5555, 2006. 

[Rou00] C. A. Rouff. Experience Using Formal Methods for Specifying a Multi-Agent System. IEEE, 2000. 

[Rou06] C. A. Rouff. Agent Technology from a Formal Perspective. NASA Monographs in Systems and 

Software Engineering. Springer, 2006. 

[RR04] A. Hinchey M. Truszkowski W. Rouff, C. A. Vanderbilt and J. Rash. Formal Methods for Swarm 

and Autonomic Systems. Proc. 1st International Symposium on Leveraging Applications of Formal 

Methods (ISoLA), 2004. 

[RR06] M. Truszkowski W. Rouff, C. A. Hinchey and J. Rash. Experiences Applying Formal Approaches in 

the Development of Swarm-Based Space Exploration Systems. International Journal on software 

Tools for Technology Transfer. Special Issue on Formal Methods in Industry, 2006.

REFERENCES 81 

[SB01] G. B. Sumpter, D. J. T. Blanchard and D. S. Broomhead. Ants and Agents: a Process Algebra 

Approach to Modelling Ant Colony Behaviour. Bulletin of Mathematical Biology, 63(5):951–980, 

September 2001. 

[SG99] W. M. Spears and D. F. Gordon. Using artificial physics to control agents. Proc. IEEE International 

Conference on Information, Intelligence, and Systems, November 1999. 

[SK96] O. Shehory and S. Kraus. Cooperative goal-satisfaction without communication in large-scale 

agent-systems. ECAI 96. 12th European Conference on Artificial Intelligence, 1996. 

[Spe04a] W. Spears. Distributed, Physics-Based Control of Swarms of Vehicles. Autonomous Robots, 17:137– 

162, 2004. 

[Spe04b] W. Spears. Physicomimetics for Mobile Robot Formations. AAMAS, July 19–23 2004. 

[Spe05] W. Spears. An Overview of Physicomimetics. Computer Science Department, University of 

Wyoming, Laramie, 2005. 

[SS04] A. Strurm and O. Shehory. A Framework for Evaluating Agent-Oriented Methodologies. AOIS, 

LNAI 3030, pages 94–109, 2004. 

[SS06] W. Spears, W. Kerr and D. Spears. Fluid-Like Swarms with Predictable Macroscopic Behavior. 

Computer Science Department, University of Wyoming, Laramie, 2006. 

[SW05] W. Spears, D. Kerr and Spears W. Physics-Based Robot Swarms For Coverage Problems. 

IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), August 2005. 

[SY99b] Kraus Shehory and Yadgar. Goal-satisfaction in large-scale agent-systems: a transportation ex- 

ample. Carnegie Mellon University, 1999. 

[SY99a] O. Shehory, S. Kraus and O. Yadgar. Emergent Cooperative Goal-Satisfaction in Large-Scale 

Automated-Agent Systems. Artificial Intelligence, 110(1):1–55, 1999. 

[Tof91] C. Tofts. Describing social insect behavior using process algebra. Transactions on Social Computing 

Simulation, pages 227–283, 1991. 

[vN96] J. von Neumann. Theory of Self-Reproducing Automata. University of Illinois Press, Urbana, 

Illinois, 1996. 

[ZS04] T. Zhang, P. Lu and L. Song. Soccer robot path planning based on the artificial potential field 

approach with simulated annealing. Robotica, 22:563–566, 2004. 

[ZW02] Q. Yin B. Zhuang, X. Meng and H. Wang. Robot Path Planning by Artificial Potential Field Op- 

Web sites 

timization Based on Reinforcement Learning with Fuzzy State. 4th World Congress on Intelligent 

Control and Automation, June 10–14 2002. 

[1] http://en.wikipedia.org/wiki/Asteroid belt (2008) 

[2] http://ssd.jpl.nasa.gov/sbdb.cgi#top (2008) 

[3] http://aa.usno.navy.mil/faq/docs/asteroid masses (2008) 

[4] http://ants.gsfc.nasa.gov (2007)

82 REFERENCES 

3 The Decisional part 

[BL08] A. Brambilla and M. Lavagna. A Decentralized Approach to Cooperative Situation Assessment 

in Multi-Robot Systems. 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 

2008), May 12–16 2008. 

[DM06] A. DeMartino and M. Marsili. Statistical mechanics of socio-economic systems with heterogeneous 

agents. Journal of Physics A: Mathematical and General, 39(43):R465 – R540(1), October 2006. 

[EK06] G. H. Elkaim and R. J. Kelbley. A Lightweight Formation Control Methodology for a Swarm of 

Non-Holonomic Vehicles. IEEE, 2006. 

[SB07a] S.L. Smith and F. Bullo. Monotonic target assignment for robotic networks. IEEE Transacion on 

Automatic Control, June 2007. Submitted. 

[SB07b] S.L. Smith and F. Bullo. Target assignment for robotic networks: asymptotic performance under 

limited communication. Autonoumous Control Conference, July 2007. 

[SG99] W. M. Spears and D. F. Gordon. Using artificial physics to control agents. Proc. IEEE International 

Conference on Information, Intelligence, and Systems, November 1999. 

[SK96] O. Shehory and S. Kraus. Cooperative goal-satisfaction without communication in large-scale agent- 

systems. ECAI 96. 12th European Conference on Artificial Intelligence, 1996. 

[SN08] P. Farinelli A. Sycara K. Settembre, G. Scerri and D. Nardi. A Decentralized Approach to Coop- 

erative Situation Assessment in Multi-Robot Systems. 7th Int. Conf. on Autonomous Agents and 

Multiagent Systems (AAMAS 2008), May 12–16 2008. 

[Spe04a] W. Spears. Distributed, Physics-Based Control of Swarms of Vehicles. Autonomous Robots, 17:137– 

162, 2004. 

[Spe04b] W. Spears. Physicomimetics for Mobile Robot Formations. AAMAS, July 19–23 2004. 

[Spe05] W. Spears. An Overview of Physicomimetics. Computer Science Department, University of 

Wyoming, Laramie, 2005. 

[SS04] A. Strurm and O. Shehory. A Framework for Evaluating Agent-Oriented Methodologies. AOIS, 

LNAI 3030, pages 94–109, 2004. 

[SS06] W. Spears, W. Kerr and D. Spears. Fluid-Like Swarms with Predictable Macroscopic Behavior. 

Computer Science Department, University of Wyoming, Laramie, 2006. 

[SW05] W. Spears, D. Kerr and Spears W. Physics-Based Robot Swarms For Coverage Problems. IEEE/RSJ 

Int. Conf. on Intelligent Robots and Systems (IROS), August 2005. 

[SY99a] O. Shehory, S. Kraus and O. Yadgar. Emergent cooperative goal-satisfaction in large-scale 

automated-agent systems. Artificial Intelligence, 110(1):1–55, 1999. 

[SY99b] S. Shehory, O. Kraus and O. Yadgar. Goal-satisfaction in large-scale agent-systems: a transportation 

example. Carnegie Mellon University, 1999.

REFERENCES 83 

4 The Physical part 

[AKL06] J. Abu-Khalaf, M. Huang and F.L. Lewis. Nonlinear H2/H∞ Constrained Feedback Control. 

Springer, 2006. 

[AM90] B. Anderson and J. Moore. Optimal Control. Prentice Hall, 1990. 

[Arm04] Armellin. Formation fliying control. Master’s thesis, Politecnico di Milano, 2004. 

[BB95] T. Basar and P. Bernhard. H∞ - Optimal Control and Related Minimax Design Problems. 

Birkhaeuser, 1995. 

[BM00] R.W. Beard and T.W. McLain. Successive Galerkin Approximation Algorithms for Non Linear and 

Robust Control. International Journal of Control, 2000. 

[Bra04] A. Bracci. Nuovi sviluppi nelle tecniche di controllo nonlineare SDRE. Master’s thesis, Università 

di Pisa, 2004. 

[Clo97] J.R. Cloutier. State-Dependent Riccati Equation Techniques: An Overview. Proceedings of the 

American Control Conference, June 1997. 

[EA01] Erdem and Alleyne. Experimental Real - Time SDRE Control of an Underactuated Robot. Proced- 

ings of the 40th IEEE Conference on Decision and Control, 2001. 

[Gad07] J. Gadewadikar. H-Infinity Output-Feedback COntrol: Application to Unmanned Aerial Vehicle. PhD 

thesis, The University of Texas at Arlington, May 2007. 

[GC06] F.L. Subbarao Q. Peng L. Gadewadikar, J. Lewis and T. Chen. H-Infinity Static Output-Feedback 

Control for Rotorcraft. AIAA Guidance, Navigation, and Control Conference and Exhibit, August 

2006. 

[GG96] F. Garofalo and L. Glielmo. Robust Control via Variable Structure and Lyapunov Techniques. 

Springer, 1996. 

[GL06] J. Gadewadikar and F. L. Lewis. Aircraft flight controller tracking design using H-Infinity static 

output-feedback. Transactions of the Institute of Measurement and Control, 28(5):429 – 440, 2006. 

[IH02] M. Inalhan, G. Tillerson and J. How. Relative Dynamics and Control of Spacecraft Formations in 

Eccentric Orbits. Journal of Guidance, Control and Dynamics, 2002. 

[IP07] Dario Izzo and Lorenzo Pettazzi. Autonomous and distributed motion planning for satellite swarm. 

Journal of Guidance, Control and Dynamics, 30(2):449 – 459, 2007. 

[JB05] A. Jaganath, C. Ridley and D.S. Bernstein. A SDRE-Based Asymptotic Observer for Nonlinear 

Discrete-Time Systems. American Control Conference, June 2005. 

[MC02] T. Crawford L.S. Menon, P.K. Lam and V.H.L. Cheng. Real-Time Computational Methods for 

SDRE Nonlinear Control of Missiles. Proceedings of the American Control Conference, May 2002. 

[Mei90] L. Meirovitch. Dynamics and Control of Structures. Wiley, 1990. 

[MO03] G.D. Menon, P.K. Sweriduk and E.J. Ohlmeyer. Optimal Fixed-Interval Integrated Guidance- 

Control Laws For Hit-To-Kill Missiles. AIAA Guidance, Navigation, and Control Conference and 

Exhibit, August 2003.

84 REFERENCES 

[MY07] M. McCamish, S. Romano and X. Yun. Autonomous distributed control algorithm for multiple 

spacecraft in close proximity operations. AIAA Guidance, Navigation, and Control Conference and 

Exhibit, 20 – 23 August 2007. 

[Naa02] Bo J. Naasz. Classical Element Feedback Control for Spacecraft Orbital Manoeuvers. Master’s 

thesis, Virginia Polytechnic Institute and State University, May 2002. 

[PB04] R.E. Palumbo, N.F. Brian and R.A. Blauwkamp. Integrated Guidance and Control for Homing 

Missiles. Johns Hopkins APL Technical Digest, 25(2), 2004. 

[PJ99] N.F. Palumbo and P. Jackson. Integrated Missile Guidance and Control: a State Dependent Riccati 

Differential Equation Approach. Proceedings of the 1999 IEEE International Conference on Control 

Applications, August 1999. 

[Szn00] M. Sznaier. Receding Horizon Control Lyapunov Function Approach to Suboptimal Regulation of 

Nonlinear Systems. Journal of Guidance, Control and Dynamics, 23(3), 2000. 

[WK03] Bogdanov Wan and Kieburtz. Model Predictive Neural Control for Aggressive Helicopter Manoeu- 

vers, 2003. 

[XO06] S.N. Xin, M. Balakrishnan and E.J. Ohlmeyer. Integrated Guidance and Control of Missiles With 

θ-D Method. IEEE Transactions On Control Systems Technology, 14(6), November 2006. 

5 The Communication part 

[BR05] E. Taghi M. Bredin, J. Demaine and D. Rus. Deploying Sensor Networks with Guaranteed Capacity 

and Fault Tolerance. MobiHoc’05, May 25–27 2005. 

[CB04] M. Chandrashekar, K. Raissi and J. Baras. Providing Full Connectivity in Large Ad-Hoc Networks 

by Dynamic Placement of Aerial Platforms. MILCOM 2004 - 2004 IEEE Military Communications 

Conference, October 31 – November 3 2004. 

[HS02] M.J. Howard, A. Matari´c and G.S. Sukhatme. Mobile Sensor Network Deployment using Potential 

Fields: A Distributed, Scalable Solution to the Area Coverage Problem. In Proceedings of the 

6th International Symposium on Distributed Autonomous Robotics Systems (DARS02), June 25–27 

2002. 

[KP05] A. Krishnamurthy and R. Preis. Satellite Formation, a Mobile Sensor Network in Space. Proceedings 

of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS’05), April 

3–8 2005. 

[MA07] R. Martijn and B. Andreas. Multi–robot exploration under the constraints of wireless networking. 

Control Engineering Practice, 15(4):435–445, 2007. 

[MCT02] H. Murrieta-Cid, R. Gonzglez-Bafiost and B. Tovar. A Reactive Motion Planner to Maintain Visi- 

bility of Unpredictable Targets. Proceedings of the 2002 IEEE lnternational Conference on Robotics 

and Automation, May 2002. 

[MS01] F. Potkonjak M. Meguerdichian, S. Koushanfar and M. Srivastava. Coverage Problems in Wireless 

Ad-hoc Sensor Networks. Proceedings IEEE INFOCOM 2001. Twentieth Annual Joint Conference 

of the IEEE Computer and Communications Societies, April 22–26 2001. 

[Par02] L.E. Parker. Distributed Algorithms for Multi-Robot Observation of Multiple Moving Targets. 

Autonomous Robots, 12:231–255, 2002.

REFERENCES 85 

[SL02] J. Savarese, C. Rabaey and K. Langendoen. Robust Positioning Algorithms for Distributed Ad-Hoc 

Wireless Sensor Networks. USENIX Technical Annual Conference, June 2002. 

[SO07] P. Scerri and S. Owens. A Decentralized Approach to Space Deconfliction. IEEE Fusion Conference, 

2007. 

[SS08] A. Simonetto, P. Scerri and K. Sycara. A Mobile Network for Mobile Sensors. IEEE Fusion 

Conference, June 2008. 

[VS07] P. Velagapudi and P. Scerri. Maintaining Shared Belief in a Large Multiagent Team. IEEE Fusion 

Conference, 2007. 

[XS05] Y. Xu and P. Scerri. An Integrated Token-Based Algorithm for Scalable Coordination. AAMAS, 

July 25 – 29 2005. 

[YS06] B. Yu and P. Scerri. Scalable and Reliable Data Delivery in Mobile Ad Hoc Sensor Networks. 

AAMAS, 2006. 

6 Perturbation models 

[Ber99] M. Berz. Academic Press. Modern map methods in particle beam physics, 1999. 

[Cho02] V. Chobotov. Orbital Mechanics. AIAA – Education Series, 2002. 

[Fer08] D. Ferraro. Taylor Series Methods. Master’s thesis, Politecnico di Milano, 2008. to be submitted. 

[PS06] G. Parissenti and A. Simonetto. Perturbed Orbits. Report, Orbital Mechanics, 2006. 

7 Results 

[GM07] S. Gill, E. D’Amico and O. Montenbruck. Autonomous Formation Flying for the PRISMA Mission. 

Journal of Guidance, Control and Dynamics, 44(3):671 – 681, May – June 2007. 

[SS08] A. Simonetto, P. Scerri and K. Sycara. A Mobile Network for Mobile Sensors. IEEE Fusion Confer- 

Web sites 

ence, June 2008. 

[1] http://ants.gsfc.nasa.gov (2007) 

[2] http://www.esa.it (2007) 

8 Final remarks 

[BL08] A. Brambilla and M. Lavagna. A Decentralized Approach to Cooperative Situation Assessment in 

Multi-Robot Systems. 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), 

May 12–16 2008. 

[MY07] M. McCamish, S. Romano and X. Yun. Autonomous distributed control algorithm for multiple 

spacecraft in close proximity operations. AIAA Guidance, Navigation, and Control Conference and 

Exhibit, 20 – 23 August 2007.

86 REFERENCES

APPENDIX 

A 

SISTEMI MULTI AGENTE PER APPLICAZIONI SPAZIALI 

“Buongiorno Mastro Antonio” disse Geppetto “Che fai lì sul pavimento? 

“Insegno l’alfabeto alle formiche” 

Carlo Collodi (1826 - 1890) 

Questa tesi è un lavoro di ricerca sui sistemi multi agente in campo spaziale, in partico- 

lare per missioni di volo di formazione ed esplorative. Per prima cosa si analizza lo stato 

del’arte, in secondo luogo si propone un controllo distribuito robusto sia di alto livello sia di 

basso livello. Il secondo include un controllo sulla traiettoria e la formazione di una rete di 

comunicazioni. 

A.1 Introduzione 

I sistemi multi agente sono studiati da discreto tempo in campo informatico, dove i con- 

testi applicativi sono diversi, dai web services, ad applicazioni multi robot, come scenari 

di salvataggio o militari. Da qualche anno la loro versatilità e affidabilità ha focalizzato 

l’attenzione del settore spaziale, in particolare perché tali sistemi sono relativamente poco 

costosi e possono permettere alti tassi di perdite tra i satelliti che compongono la formazione 

senza compromettere l’adempimento della missione. Sia ESA che NASA stanno proponendo 

missioni, come grandi formazioni di satelliti per telescopi interferometrici o esplorazione della 

cintura di asteroidi, dove il paradigma multi agente è l’aspetto fondamentale. Per questo 

motivo i metodi classici di controllo centralizzato devono essere rivisti in ottica distribuita e 

multi agente, questa è anche l’idea fondamentale della tesi: studiare come gli algoritmi multi 

agente possano entrare nell’ambiente spaziale in modo verificabile e robusto. 

i

ii SISTEMI MULTI AGENTE 

A.1.1 Principali Contributi 

I principali contributi innovativi alla tesi sono diversi: in primo luogo la formulazione del 

controllo di alto livello – Goal Manager – è una estensione di una tecnica formale conosciuta 

come Fisica Artificiale. In secondo luogo, il controllo della traiettoria sviluppa un controllo 

non lineare alla Lyapunov per inquadrarlo in un contesto robusto, H∞, risolvendo con una 

opportuna semplificazione una particolare equazione di Riccati. Infine, per la parte di assicu- 

razione di una rete di comunicazione, le prestazioni di un metodo a campi a potenziale sono 

migliorate introducendo pacchetti di informazione random, chiamati token. 

A.2 Formulazione del Problema 

Il problema viene viene formulato come un problema di controllo robusto e distribuito: dato 

uno scenario, scrivere un algoritmo di controllo che consenta ad ogni agente di decidere 

autonomamente o con un minimo utilizzo di comunicazione, cosa fare e come fare a farlo, 

cioè come muoversi e come comunicare. 

Gli scenari studiati sono due: un volo di formazione di due satelliti, missione ESA - 

Prisma, dove le parte fondamentale è assicurare una distanza relativa in un certo intervallo 

a fronte di diverse perturbazioni, quali atmosferica e gravitazionale; poi, l’esplorazione della 

cintura degli asteroidi, dove uno sciame di satelliti è spedito per raccogliere informazioni. In 

questo caso la non conoscenza dell’ambiente impone che il sistema di controllo sia robusto 

per far fronte ad incertezze di modello e perturbazioni esterne. 

Il controllo di ogni agente è diviso in due parti, la parte di alto livello, che deve decidere 

che cosa fare, e la parte di basso livello, che invece decide come farlo. Per il controllo di alto 

livello si usa una approccio simile ai campi a potenziale, chiamato Fisica Artificiale (Artificial 

Physics). Tale metodo è selezionato perché, in primo luogo, è inquadrabile in un contesto 

formale, cosa importante per la verifica dell’algoritmo; in secondo luogo, richiede un minimo 

uso di comunicazione, cosa fondamentale in campo spaziale. La parte di basso livello è divisa 

ulteriormente in una parte di controllo della traiettoria e una parte di controllo della creazione 

di una rete di comunicazioni. 

A.3 Controllo di Alto Livello 

L’idea alla base dell’approaccio a Fisica Artificiale consiste nella formazione di un campo a 

potenziale sulla basse delle informazioni che arrivano dal mondo esterno. La scelta su cui si 

basa il controllo di Alto Livello è poi ridotta al calcolo del gradiente di tale campo. 

Definendo un opportuno insieme di agenti A, una misura delle capacità che ogni agente 

ha, cij, un opportuno insieme di obiettivi G, le capacità che ogni obiettivo richiede per essere 

soddisfatto, tij e una metrica che misura quanto gli agenti siano vicini tra di loro o all’obiettivo 

dij, si riesce a introdurre un potenziale nella forma 

Φ w i = � 

n 

αnicnwciw Φa(din) + � 

m 

βnitnwciw Φg(dim) , w = 1, . . . , k (A.1)

SISTEMI MULTI AGENTE iii 

Il quale estende la formulazione tradizionale di Fisica Artificiale a contesti con più ca- 

pacità/obiettivi. Da un calcolo del gradiente, con eventualmente una risoluzione di conflitti, 

ogni agente riesce, in modo completamente autonomo a decidere quale obiettivo perseguire. 

Siccome di suppone che gli agenti non abbiamo una conoscenza completa dell’ambiente, 

è importante notare che le informazioni che servono ad ogni sonda per costruire il proprio 

campo a potenziale locale sono di due tipi, da un lato quelle ricavate dai sensori, dall’altro 

quelle inviate dalle altre sonde. Questo implica l’esistenza di una rete di comunicazioni e un 

protocollo attraverso il quale l’informazione è condivisa. Per questioni di velocità di risposta, 

si sceglie un approccio broadcast per il protocollo. Inoltre, l’informazione è opportunamente 

modificata viaggiando da agente ad agente per tener conto delle scelte personali ed eventuali 

conflitti. 

A.4 Controllo Robusto della Traiettoria 

L’acquisizione della traiettoria che porta ogni singolo agente nel luogo desiderato è formulata 

dividendo la dinamica in lungo e corto periodo. 

La parte di lungo periodo tiene in considerazione la traiettoria globale ed è sviluppata 

con variabili equinoziali non singolari nel contesto di un controllo non lineare alla Lyapunov. 

Inoltre per tenere presente le perturbazioni tale approccio è esteso usando una tecnica H∞ 

che rende robusto l’algoritmo. L’equazione di Hamilton – Jacobi – Isaacs risultante 

0 = min 

u max 

w [L(x, u, w) + ∇ V B (u + w)] (A.2) 

porta ad una particolare equazione di Riccati, che viene denominata equazione di anna, che 

può essere risolta con opportune semplificazioni e permette di derivare le condizioni necessarie 

e sufficienti per la stabilità. Il controllo viene scritto come 

u = − 1 

2 R−1 B ′ Q δx (A.3) 

Per la parte di corto periodo si propongono due possibili soluzioni, entrambe note in 

letteratura. La prima consiste in un accoppiamento controllo ottimo e campi a potenziale di 

velocità, la seconda in una ottimizzazione vincolata della dinamica discretizzata via matrici 

di convoluzione. Entrambe andrebbero adattate ed integrate al modello di lungo periodo. 

A.5 Architettura di Comunicazione 

L’architettura di comunicazione, o meglio come gli agenti designati a formare la rete di 

comunicazione tra gli esploratori e Terra si devono muovere, è formulata in un contesto di 

campi a potenziale. Questo si combina perfettamente con l’idea di Fisica Artificiale. Siccome 

i campi a potenziale non consentono, da un lato, di focalizzare gli agenti sul loro dovere 

ma, generalmente, li distribuiscono equamente nello spazio, e dall’altro, non minimizzano le 

comunicazioni in modo drastico, è sviluppato un approccio a potenziale dinamico basato su

iv SISTEMI MULTI AGENTE 

singoli pacchetti random di informazione, i token. Questo consente di migliorare le prestazioni 

sia per quanto riguarda la connettività globale sia per la tolleranza a rotture di singoli agenti. 

L’idea chiave dell’approccio dinamico è usare i singoli token per spedire attraverso la 

rete di comunicazione richieste di aiuto; mediante l’imposizione di una opportuna politica 

ogni agente sceglie quale agente è più in difficoltà e decide quale pacchetto di informazione 

seguire e quale rimbalzare ad altri. Le politiche analizzate sono diverse, ma includono tutte 

il concetto di connettività locale. 

A.6 Modelli per le Perturbazioni 

Due i modelli di perturbazione sviluppati, uno per le missioni di volo di formazione vicino 

Terra, che include sia la resistenza atmosferica sia effetti gravitazionali come J2 e J22, ed uno 

per le missioni nella cintura degli asteroidi. Quest’ultimo scrive il campo perturbativo come 

somma dei campi gravitazionali dovuti ai primi dieci più massivi asteroidi. 

A.7 Risultati 

I risultati si dividono in quattro gruppi, quelli del controllo di alto livello, applicato ad 

un esempio abbastanza generale; quelli del controllo della traiettoria, applicato a diverse 

simulazioni nel contesto della missione Prisma; quelli per la rete di comunicazione, applicati 

a scenari a due e tre dimensioni; infine, quelli per il controllo globale, applicato ad uno 

scenario di missione verso gli asteroidi. 

A.7.1 Controllo di alto livello 

Il controllo di alto livello e il meccanismo di condivisione delle informazioni, porta a buoni 

risultati, in particolare riesce a assegnare gli obiettivi con errori inferiori al 10% in uno scenario 

con 3 asteroidi e 3 capacità/obiettivi, Figura A.1. 

A.7.2 Controllo della traiettoria 

Il controllo della traiettoria è applicato alla missione Prima e, in particolare, si vuole garan- 

tire che il satellite che insegue riesca ad avere una distanza relativa rispetto a quello che 

precede, entro la decina di kilometri. Questo permetterebbe di sviluppare in sede successiva 

un controllo più fine di rendezvous. Diverse simulazioni e test sono stati condotti, il più sig- 

nificativo è una analisi statistica alla Montecarlo su incertezze delle variabili di stato iniziali 

dell’inseguitore, a simulare un’incertezza sulla posizione in cui il lanciatore lascerà il satellite. 

ā ē ī ¯ω ¯ Ω ¯ θ 

RE+ 700 km 0 98.2π/180 0 0 0 

Table A.1: Parametri orbitali del satellite Main.

SISTEMI MULTI AGENTE v 

Valore t ij /(t ij ) 0 

0.25 

0.2 

0.15 

0.1 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

Prestazioni del controllo di alto livello 

Primo Asteroide 

Secondo Asteroide 

Terzo Asteroide 

30 40 50 60 70 80 90 100 

Numero di agenti, N 

Figure A.1: Risultati per il Goal Manager; la larghezza delle linee rappresenta la deviazione 

standard. 

dove RE è il raggio terrestre. 


RE+ 700km ± 100 km ē + 0.1 ī ± 10 ◦ ¯ω ± 10 ◦ ¯ Ω ± 10 ◦ π/2 ± 10 ◦ 

Table A.2: Incertezza sui parametri del satellite Target per la simulazione Montecarlo. 

In Figura A.2 le distanze relative dopo 20h. I risultati mostrano un comportamento 

robusto dell’algoritmo di controllo consentendo un avvicinamento sotto i 5 km.

vi SISTEMI MULTI AGENTE 


Radial [km] 

2 

1.5 

1 

0.5 

0 

−0.5 

−1 

Distanza relativa per t ∈ [20, 21] h 

−1.5 

−40 −20 0 20 40 

In track [km] 

60 80 100 120 

2 

1.5 

1 

0.5 

0 

−0.5 

−1 

Distanza relativa per t ∈ [20, 21] h 

−1.5 

−40 −20 0 20 40 

In track [km] 

60 80 100 120 

Figure A.2: Posizioni relative raggiunte dopo 20 h. La linea nera è una delle traiettorie, le 

altre sono ad essa simili, e tutte passano per il punto 0 km in track.

SISTEMI MULTI AGENTE vii 

A.7.3 Dispiegamento della rete di comunicazione 

Il dispiegamento della rete di comunicazione viene testato su due scenari significativi, uno 

bidimensionale, l’altro tridimensionale, mostrando come gli algoritmi token portino ad un 

notevole aumento delle prestazioni, sia in termini di connettività globale media (nel tempo) 

< K >, che di efficienza media < E > del grafo. 

Nelle Figure A.3 - A.4, sono presentati i risultati per lo scenario tridimensionale. 

Connettività globale media 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 



Figure A.3: < K > per un differente numero di comunicatori, |C|: B sta per algoritmo Base, 

C per politica C, RC per politica RC. Le barre di errore nere sono la deviazione standard, le 

linee nere i valori finali di K. 

A.7.4 Cintura degli asteroidi 

Uno scenario all’interno della cintura degli asteroidi è sviluppato per mostrare come il con- 

trollo globale degli agenti funzioni assieme; in particolare si scelgono due asteroidi da rag- 

giungere, tre esploratori che devono raggiungerli, un agente hub che deve comunicare con 

Terra, e una serie di agenti di comunicazione tra esploratori e hub. I risultati, mostrati nelle 

Figure A.5 - A.7.4, portano ad un controllo robusto sulla traiettoria e ad un dispiegamento 

della rete di comunicazione tale da assicurare connettività almeno pari ad uno.

viii SISTEMI MULTI AGENTE 

Efficienza media 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 



Figure A.4: < E > per un differente numero di comunicatori, |C|: B sta per algoritmo Base, 

C per politica C, RC per politica RC. Le barre di errore nere sono la deviazione standard. 

y [AU] 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

−0.3 

−0.35 

Posizioni relative dall’Hub [AU] 

−0.1 0 0.1 0.2 

x [AU] 

0.3 0.4 

Figure A.5: Traiettorie per esploratori (nero), Hub (rosso) e asteroidi (blu) in un sistema di 

riferimento inerziale centrate nell’Hub.

SISTEMI MULTI AGENTE ix 

y [AU] 

0.1 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

−0.3 

−0.35 

−0.4 

−0.2 −0.1 0 0.1 0.2 

x [AU] 

0.3 0.4 0.5 0.6 

(a) Stato Iniziale 

y [AU] 

0.1 

0.05 

0 

−0.05 

−0.1 

−0.15 

−0.2 

−0.25 

−0.3 

−0.35 

−0.4 

−0.2 −0.1 0 0.1 0.2 

x [AU] 

0.3 0.4 0.5 0.6 

(b) Stato Finale 

Figure A.6: Scenario Asteroidi: i cerchi neri sono gli esploratori, quelli bianchi i comunicatori, 

le linee tratteggiate i collegamenti della rete di comunicazione, il triangolo, l’Hub. 

A.8 Sviluppi Futuri 

Tra gli sviluppi futuri si possono elencare 

� estensione del controllo di alto livello a piccole formazioni e quando la connettività 

iniziale è piccola; 

� dinamica di corto periodo per il controllo della traiettoria; 

� controllo di movimento linearizzato per l’assicurazione della rete di comunicazione.

that’s all folks

POLITECNICO DI MILANO - DCSC

Create successful ePaper yourself

Delete template?

Save as template?