System-of-Systems Engineering - Liophant Simulation

Developing Methods and Techniques forSystem of Systems EngineeringPresentedByDr. Dimitri MavrisBoeing Professor for Advanced Aerospace Systems AnalysisDirector, Aerospace Systems Design Laboratory (ASDL)Georgia Institute of Technologydimitri.mavris@ae.gatech.edu

Outline• Motivation• What is a SoS, and what is SoSE?• Challenges in SoSE• Enabling Techniques and Methods• Research Areas• ConclusionsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu2

The Aerospace Systems Design Laboratory• Founded in 1992, ASDL was created to bridge thegap between academia and industry’s researchperspectives– 175 MS and PhD Students, 50 Undergrads, 40 ResearchStaff Members and over $14M in sponsored research• School of AE was one of the seven originalGuggenheim Aeronautics schools, founded in 1930• GT consistently ranked 3rd or 4th best college ofengineering in the country based on US News &World Report* Aerospace, Transportation & Advanced Systems LaboratoryDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu3

Motivation - Increasing need for moreinteroperable, cost-effective systems• Military systems are becomingincreasingly complex as informationand communications technology surges• Distribution of operations is creating aneed for aerospace systems to becomeincreasingly interoperablehttps://nasea.faa.gov/architecture/main/display/2• Despite their complexity, aerospacesystems must be cost effectiveThese trends have resulted in an increasedfocus on System-of-Systems Engineering incivil, military, and space applicationsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu4http://www.ibm.com

Motivation- Emphasis on “Early-Phase” SE• Systems Engineering (SE)– An interdisciplinary approach to derive, evolve, and verify a life-cycle balancedsystem solution that satisfies customer expectations and meets publicacceptability (IEEE 1220-1994)• The most fundamental (and most difficult and costly to reverse) decisionsare made in the early phases of design and architecture definition– This is doubly true for System of SystemsVerification andValidation“Early-Phase” SEAdapted From INCOSE HandbookDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu5

Shift to a “Capability-based” Focus“In the past 15 years, the Department of Defense (DOD) has faced a constant streamof new challenges…the United States must be prepared both to deal with a largernumber of more diverse threats with varied attributes and to do so in circumstancesinvolving complex and uncertain risks.”-Naval Analytical Capabilities: Improving Capabilities-Based Planning, Committee on Naval Analytical Capabilities andImproving Capabilities-Based Planning, National Research Council• There is an overall desire to acquire capabilities, particularly in military applications– New acquisition paradigms attempt to be more “top-down” and avoid stove-piping• These capabilities are often not enabled by a single system, but rather by a systemof systems– The supporting systems (such as ships and aircraft) are typically multi-mission– Systems cannot be studied in isolation, but must be examined in the context ofoperational scenarios, environments, and interactions• High-quality System of Systems Engineering is a key component to successfulcapability-based designDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu6

What is a system?“A set of functional elements organized to satisfy user needs.” (IEEE P1220 p. 238)Developing Methods and Techniques for System of Systems Engineering7Dr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu7

What is a System of Systems?Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu8

System of Systems• “A set or arrangement of systems that results when independentand useful systems are integrated into a larger system thatdelivers unique capabilities.”– DoD Defense Acquisition Guidebook 2004• "System of systems applies to a system-of-interest whose systemelements are themselves systems; typically these entail largescale inter-disciplinary problems with multiple, heterogeneous,distributed systems."– INCOSE-TP-2003-002-03, Systems Engineering Handbook V3• “Groups of systems, each of which individually provides its ownmission capability, that can be operated collectively to achieve anindependent, and usually larger, common mission capability “– Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for FutureAir Force Systems Acquisition. National Academies Press, 2008Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu9

What makes Systems of Systems (SoS)Different?• Compared to a System, an SoS might:– Be larger in scope– Have more complex integration– Usually be subject to higher degree of uncertainty and risk– Evolve more continuously with elements of differing lifecycles– Lack a single management/acquisition entity and have abroader range of stakeholders– Have elements which are not designed to fit the whole, andwhich are integrated post-design and deployment– Exhibit emergent behaviors– Have more ambiguous requirements and fuzzy boundaries– Have continuous SE which is never finishedSystem-of-Systems Engineering is an emerging discipline in the aerospace communityand new methods and techniques are required to address SoS challenges, but theseshould be based on proven Systems Engineering methodsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.eduSources: INCOSE SE Handbook v3, "Architecting Principles for Systems-of-Systems“ by Mark W. Maier, DoD SE Guide for SoS10

What is SoSE?• System of Systems (SoS) Engineering is an emerginginterdisciplinary approach focusing on the effort required totransform capabilities into SoS solutions and shape therequirements for systems. SoS Engineering ensures that:– Individually developed, managed, and operated systems function asautonomous constituents of one or more SoS and provide appropriatefunctional capabilities to each of those SoS– Political, financial, legal, technical, social, operational, andorganizational factors, including the stakeholders’ perspectives andrelationships, are considered in SoS development, management, andoperations– A SoS can accommodate changes to its conceptual, functional,physical, and temporal boundaries without negative impacts on itsmanagement and operations– A SoS collective behavior, and its dynamic interactions with itsenvironment to adapt and respond, enables the SoS to meet orexceed the required capability.Source: System of Systems Engineering Center of Excellence, Sponsored by the Office of the Under Secretary of Defense for Acquisition,Technology, & Logistics, Defense Systems, Systems and Mission Integration, Joint Force Integration (USD-AT&L)Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu11

INCOSE Challenges in SoS Engineering1. Complexity is a major issue2. Management can overshadow engineering3. The initial requirements are likely to be ambiguous4. System elements operate independently5. Fuzzy boundaries cause confusion6. System elements have different life cycles7. SoS engineering is never finishedNew methods and techniques are required to address these challengesDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu12

The Evolution of New Ideas“All truth passes through three stages. First, it isridiculed. Second, it is violently opposed. Third, it isaccepted as being self-evident.”--Arthur Schopenhauer, German philosopher (1788 - 1860)This principle is observed for all new ideas as wellDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu13

Overcoming Organizational Barriers associatedwith any new Methodology• New methods go against the grain of established paradigms that are welldefined and accepted by the practicing community and thus are alwaysviewed with skepticism, criticism, or cynicism• Criteria to facilitate the introduction and acceptance of new methods :– The underlying theories, methods, mathematics, logic, algorithms, etc. uponwhich the new approaches are based must be well understood, accepted,scientifically sound and practical– Familiarity is needed with the underlying theories and the material needed forsomeone to understand the method itself must be readily available– Availability of training material written on the overarching method, tutorials,etc. with relevant examples– Tools automating the proposed method and making it practical for every dayuse to take the method beyond the academic level– Relevant examples and applications within a given field of study• Proposed methods which are grounded in or are complimentary toestablished practices have a better chance of succeedingDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu14

What is needed for this New Paradigm Shift to Occur?• Transition from single-discipline to multi-disciplinary analysis, design and optimizationthat can handle concurrently means and ways trades• Easy to use integrative environments that can handle the combinatorial nature of theSOS problems• Automation of the resultant integrated design process• Transition from a reliance on historical data to physics-based formulations, especiallytrue for unconventional concepts• Means to perform requirements exploration, technology infusion trade-offs andconcept down selections during the early design phases (conceptual design) usingphysics-based methods• Methods which will allow us to move from deterministic, serial, single-point solutionsto dynamic parametric trade environments• Incorporation of probabilistic methods to quantify, assess risk• Transition from single-objective to multi-objective optimization• Need to speed up computation to allow for the inclusion of variable fidelity tools so asto improve accuracy, from macro-level to meso- to micro-level representations• Means to facilitate data and knowledge creation, storage, versioning, retrieval andmining• An integrated knowledge based systems engineering and management framework• Means to perform dynamic visualization of the results in a team-centered, real-timeanalysis environmentDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu15

Synthesis of Established Techniques to Create New MethodsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu16

An Architecture-based Approach to SoSE• “The initial stages of architecture design are where the most fundamental design decisionsare made; these are the decisions which are most difficult to correct when they are in error”– Felix Bachmann, Software Engineering Institute (SEI) at Carnegie Melon• “To effectively acquire complex systems-of-systems in a capability-based acquisitionenvironment requires that we increase the use of integrated architectures to identify interrelationshipsand resolve issues with system integration and interoperability that impact theoperational effectiveness of warriors; platforms; command and control; networks; andweapons.”– Phillipp Charles, Chief Engineer of SPAWAR Systems Center• “Degradation in combat effectiveness can be caused by…poor or non-existent integration orinteroperability. Because integration and interoperability are so critical to combateffectiveness, the entire Family of Systems must be considered in the engineering andacquisition process if decision makers are to choose the most operationally sound, technicallyfeasible, and effective program investments”– Dickerson, Soules, Sabins, Charles in “Using Architectures for Research Development andAcquisition”Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu17

ArchitectureHistorically:• The art or practice of designing and constructing buildings (Oxford English Dictionary)• Formation or construction resulting from or as if from a conscious act resulting in a unifyingor coherent form or structure (Merriam-Webster Dictionary)• Many others• Common Themes: Structure, Utility, Beauty (or attractiveness)In Systems Engineering:• The fundamental organization of a system, embodied in its components, their relationshipsto each other and the environment, and the principles governing its design and evolution.(From ANSI/IEEE 1471-2000)At ASDL:• The fundamental organization of a system, embodied in its components, their relationshipsto each other and the environment, the principles governing its design and evolution, itspurpose (utility), and its attractiveness (e.g. functionality, cost, etc)Including the utility and the value into the architecture development phase requires the ability toestimate and evaluate these components of the architecture, thus driving a need for architectureframeworks to be integrated with systems engineering and modeling and simulationDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu18

The System Alternative Space• The Interactive Reconfigurable Matrix of Alternatives (IRMA) isused to explore the alternative space for a new system• The IRMA is demonstrated here using a notional exampleDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu19

SoS Architecture Alternative Space• Operational Alternatives (HOW and WHEN)– Changing the ways things are done (for example, thecommunication structure, or the order in which activities areperformed)• System Alternatives (WHAT and HOW MANY)– Changing the elements (physical systems, the means) of thearchitecture• Organizational Alternatives (WHO)– Changing who is responsible for certain elements, activities,facilities, etc• Network Alternatives (HOW)– Changing the network architecture that enables the informationflow required by the SoS• Combinations of the aboveDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu20

The Architecture-based Technology Evaluation andCapability Tradeoff Method (ARCHITECT)Capability: The ability to achieve an effect to astandard in a given environment using multiplecombinations of ways and means (DoD)WAYSMEANSDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu21

Modeling and Simulation Challenges in SoS Engineering• Physical experiments are typically infeasible or limited– Computer simulations are required, and are oftencomputationally intensive and time consuming– Verification and Validation is a challenge• SoS are complex– Limits available modeling techniques– Often results in high dimensionality• SoS have a large and diverse alternative space– Unfathomable number of combinations– Need to speed up modeling and simulation– Can be challenging to visualize results• SoS are stochastic in nature– For a given set of inputs, the results are a distribution– Behavior is often modal in natureDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu22

Extension to Systems-of-Systems Analysis• The DoD shift to capabilitybasedacquisition is mergingthe operations research andsystems design communities• The impact of systems andsub-systems is oftennegligible when compared totactics, doctrine, and strategy• Methods for efficientscenario construction,surrogate creation, andoptimization for SoS areneededInfinite FOR FixedTarget SensorGIT Battle ManagerIn-TheaterBasingTarget Latitude373635AirborneSensorLRS PlatformsHigh Speed Missile34Baghdad33323130Denied Basing39 40 41 42 43 44 45 46 47 48Target LongitudeDistribution of Successful Engagementsin Iraq (Desert Storm Scenario)Perfect CommunicationsJDAM-likeLow Speed MissileDefensesThrust/Weight Ratio1.51.41.31.21.110.90.80.70.60.50.40.3TCT Finder/ShooterTCT’sIADSSuccessful Engagements20 30 40 50 60 70 80 90 100 110 120 130 140 150Wing Loading (lb/ft 2 )Successful Engagements Favor HighWing Loading and Low Thrust/WeightRatio (Bomber-like)AreaDominance(Multi-Hit)High ValueTargets(Fixed)TargetBomberDead SAMsBattle ManagerMap Image c/o National Geospatial Intelligence Agency (www.nga.mil)Targets NeutralizedHigh SpeedLess SuccessfulPlatform SpeedBest Speed~ 750 knotsMunition RangeThe Complexity of the Systems-of-Systems Analysis Problem Often Confounds AnalystsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu23

What Is Meant By a Complex System?• Many contrasting views– Biology, Computer Science, Engineering, Economics, etc.• Complex System: two pertinent definitions– A system composed of interconnected parts that as a whole exhibit one or more properties(behavior among the possible properties) not obvious from the properties of the individualparts 1 (Reductionism vs. Holism)– A system having many interrelated, interconnected, or interwoven elements and interfaces 2Physical DomainSimple ComplexSimpleSystemVery Predictable.Traditionalengineeringmethods apply.ChaoticSystem(Non-Deterministic)Randomperturbations giveappearance ofcomplexity. Solvedusing Robust DesignComplicatedSystemSatisfies functionalrequirements, butcannot ensureunder all possibleconditions/ statesComplexSystemMust architectsystem to behavecorrectly bytailoring theemergentbehaviorsFigure adapted from Balestrini-Robinson, Santiago. “A Modeling Process toUnderstand Complex System Architectures,” 2009.Functional DomainSimple ComplexDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu Architecture 3 :Structure of componentsRelationships (Complexinformation exchanges, systeminterfaces, functionalinteroperation, etc.)Principles & guidelinesgoverning evolution over time1. Joslyn, C. and Rocha, L. (2000). Towards Semiotic Agent-Based Modelsof Socio-Technical Organizations, 2000.2. Crawley, Edward. System Architecture –course notes. MIT, 2005.243. IEEE Std 610.12

Fundamental Properties of a Complex System1. Self Organization2. Non-LinearInteractions3. Adaptation4. HeterogeneityEmergentBehaviorComplex system properties lead to emergent behaviorDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu25

Impact of Complexity on Modeling and Simulation• Self Organization– Feedback and interactions must be captured• Non-Linearity– Behavior is more than sum of the behavior of the components– Tiny change in a condition can eventually lead to a huge numberof different possible results• Adaptation– Complex systems continually adapt to their environment toimprove performance– Adaptive agents are more robust but more difficult to create• Heterogeneity– Chance of emulating emergent behavior increases with moreinteractions of diverse agentsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu26

Enablers for Complex SoS• Design of ComputerSimulations– Space Filling Designs– Adaptive DoE• Modeling and SimulationTechniques– Agent-based modeling andconstructive simulations– System Dynamics Modeling– Discrete Event Simulation– Mathematical Modeling Techniques• Non-linear SurrogateModeling– Neural Networks– Kriging/Gaussian– Stepwise RSE• Probabilistic Theory– Stochastic modeling– Surrogate modeling of stochasticprocesses– Monte Carlo Simulation• Visual AnalyticsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu27

Enablers for Complex SoS• Design of ComputerSimulations– Space Filling Designs– Adaptive DoE• Modeling and SimulationTechniques– Agent-based modeling andconstructive simulations– System Dynamics Modeling– Discrete Event Simulation– Mathematical Modeling Techniques• Non-linear SurrogateModeling– Neural Networks– Kriging/Gaussian– Stepwise RSE• Probabilistic Theory– Stochastic modeling– Surrogate modeling of stochasticprocesses– Monte Carlo Simulation• Visual AnalyticsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu28

Design of Experiments• Design of Experiments (DoE) was developedfor physical experiments– Effect of fertilizers on crops– Effect of food and environment on bacteriagrowth– Variations in weld strengths• Design of Experiments has historically focusedon how to devise screening techniques andsampling strategies for physical experimentsthat are aimed at mitigating the effects of therandom errorDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu29

Physical vs. Computational ExperimentsPhysical Experiments• Often a limited number offactors• Data collection must often bedone in “one shot” (forexample, one growing season)• Types of Error– Human Error: Experimentermakes a mistake– Systemic Error: Flaw inphilosophy of the experimentadds a consistent bias to result– Random Error: Measurementinaccuracies due to theinstruments being usedComputational Experiments•Often have a larger number offactors than real worldexperiments•Data collection is sequential innature•Types of Error– Human Error: Bugs in the code,incorrectly entered boundaryconditions, etc– Systemic Error: Consistenterrors due to approximations inthe code– Random Error: Does not exist incomputational experimentsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu30

Design of Computer Simulation• Computer simulation is a numerical technique forconducting experiments on certain types ofmathematical and logical models describing thebehavior of a system (or some componentthereof) on a digital computer over extendedperiods of real time. (Burdick & Naylor, 1966)• Design of Computer Simulation (DoCS) is gearedtoward developing sound experimental designpractices foe experiments performed oncomputational simulationsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu31

Enablers for Complex SoS• Design of ComputerSimulations– Space Filling Designs– Adaptive DoE• Modeling and SimulationTechniques– Agent-based modeling andconstructive simulations– System Dynamics Modeling– Discrete Event Simulation– Mathematical Modeling Techniques• Non-linear SurrogateModeling– Neural Networks– Kriging/Gaussian– Stepwise RSE• Probabilistic Theory– Stochastic modeling– Surrogate modeling of stochasticprocesses– Monte Carlo Simulation• Visual AnalyticsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu32

Agent-Based Behavioral Modeling• Agent based modeling (ABM) is amicro-level bottom-up approachto modeling phenomena at theentity level in position and time• Each agent interacts with itsenvironment and with each other• ABM answers “how properties ofthe whole emerge fromproperties of the constituentelements”– Behavior of overall stockmarket arises from the pricesof individual stocks boughtand sold– Scenes of battle in Lord ofthe Rings arise from thesimulated behavior of manyorcs• Methods for validation neededMassiveSoftwarewww.massivesoftware.comImage:http://www.vfxtalk.com/forum/massive-software-used-create-visualt5617.htmlNetlogo Agent-BasedSimulation Toolhttp://ccl.northwestern.edu/netlogo/Agent-Based Techniques Model Behaviors with a Bottom-Up Constructionist ViewDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu33

Agent-Based Behavioral Modeling• Agent Based Models are appropriate when [Bonabeau2002]:– Individual behavior is nonlinear and can be characterized bydiscrete decisions, thresholds, if-then rules, or nonlinearcoupling– Describing discontinuity in individual behavior is difficult withdifferential equations. For example, if a logistics officer ordersparts in batches, he may have a threshold for making partsrequests (rather than continuously demanding replacements forparts used)– History matters. Path-dependence, lagging responses, non-Markovian behavior, or temporal correlations including learningand adaptation are applicable to the system.– Averages are not good enough. Under certain conditions, smallfluctuations in a complex system can be amplified, so that thesystem is stable for incremental changes but unstable to largeperturbationsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu34

System Dynamics Modeling• System Dynamics (SD) is amacro-level top-downapproach to modelingphenomena at high levels ofabstraction and aggregation• SD models portray thestructure ofinterrelationships betweenvariables with flows andstocks– Stocks are the variables inthe system– Flows represent change• Methods for rapid modelconstruction and validationare neededWealthof TCCMoneyin terrorcentersMoneygiven outto TMCExcitementHumanagentTerrortargetsTCmoneyTCCmoneyTerrorCentersABM:humaninteractionswith TCs andtargetsSystemdynamics:TerroristfundingSystem Dynamics Handles Complexity with a Top-Down View of the WorldDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu35

Discrete Event Simulation• Discrete Event Simulation(DES) uses numerical analysisto analyze systems where thestate variable(s) changes onlyat discrete points in time• DES Paradigms– Activity Oriented– Event Oriented– Process Oriented• DES models have theadvantages of a relatively fastrun time, flexibility, andmodularity• Useful for modeling queuesand logisticsCases with both a low time to execute killchain and high probability of successCases with a low time to execute kill chainCases with high probability of successCases not on the Pareto FrontierDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu36

Discrete Event Simulations• Discrete event problems embody the following concepts[Fishman 2001]:– Work - items, jobs, or customers seeking service– Resources - provider of service– Routing - collection of required services– Buffers - waiting area for work awaiting service– Scheduling - pattern of resource availability– Sequencing - order resources provide service (e.g. first come firstserve)– Performance - overall system measure• DES has many types of applications and describes a broadclass of simulations• Queuing models are able to describe systems with resourceallocation and sequences of operations [Zimmermann 2008]Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu37

Mathematical Modeling Techniques• There are a variety ofmathematical modelingtechniques that areapplicable to SoS• Markov Chains– Used to model stochasticprocesses which adhere tothe Markov Property• Graph Theory– Basis of many networkmodels– Can also be used to studythe complexity andstructure of a SoSblog.purevisibility.comblog.purevisibility.comDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.eduEetimes.com38discovery.bits-pilani.ac.in

Enablers for Complex SoS• Design of ComputerSimulations– Space Filling Designs– Adaptive DoE• Modeling and SimulationTechniques– Agent-based modeling andconstructive simulations– System Dynamics Modeling– Discrete Event Simulation– Mathematical Modeling Techniques• Non-linear SurrogateModeling– Neural Networks– Kriging/Gaussian– Stepwise RSE• Probabilistic Theory– Stochastic modeling– Surrogate modeling of stochasticprocesses– Monte Carlo Simulation• Visual AnalyticsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu39

Surrogate Modeling Comes of Age• In the 1990s, surrogate models were seen as amathematical curiosity in the designcommunity• Response surface equations, Kriging models,and neural networks have been crossfertilizedand their use is now widespread• Automated tools for creation and validationare needed• A library of surrogates and the underlyingassumptions for their use must be constructedR 2Surrogate modeling for systems analysis is now a standard technique in the fieldDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu40

Surrogate ModelsWhy do we use surrogate models?• Statistical analysis of contributing factors, includinginteractions and high order effects• Closed-form mathematical characterization of “blackbox”• Dynamic / interactive visualization of model space• Computer evaluation time is multiple orders ofmagnitude less than legacy “black box” codes,enables Monte Carlo simulations and probabilisticanalysis• Share model of systemic behavior without sharingsensitive tools and models.Training data setV&V data setError checkingComputer model / Legacy code / Black boxWhat do we need to create surrogate models?• A carefully selected data set for regression or“training”, may imply a significant allocation ofresources (Design of Experiments)• A sufficiently broad data set for Validation &Verification (V&V) through and error checkingSecure sharingDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu41

Surrogate ModelsAre surrogate models the silver bullet? No,because:• They contain statistical error that maybe significant• Their applicability is limited by thedomain range of the DoE• The adequacy of a given surrogatemodel type depends on the nature ofbehavior it is meant to capture• They carry implicit assumptions thatcan be overlookedWhat kinds of surrogate models are there? Different surrogate models will capturevarying levels of complex behavior with agiven degree of accuracy/error using acorresponding regression/training data setResponse Surface Equations• …altogether, surrogates can bemisused or abused for analysis• Selecting the “right” type of surrogate(validation) is as important asconstructing the surrogate “right”(verification)Gaussian Process Regression,Or KrigingArtificial Neural NetworksDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu42

Challenges to Surrogate Modeling forStochastic Processes• For a given set of inputs, the outputs are a distribution, nota single point– Repetitions of each DoE case are required to capture thedistribution– If the variability is high, a large number of repetitions arerequired• The distribution may be non-normal, or be modal in nature– More difficult to accurately model a non-normal distribution– In the case of modal behavior, must determine the drivers forthe modes• Multiple surrogates will be required for each response– The number of surrogates required will be dependant on theassumed distributionDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu43

Surrogate Modeling for Stochastic Processes• Beta Distribution– Beta distributions areextremely flexiblefunctions– Beta pdf’s have fourparameters:• α : Shape parameter• β : Shape parameter• min : Range parameter• max : RangeparameterDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu44

Fitting Modal ResponsesDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu45

Creating Surrogates for Stochastic Models1. Run a small number of repetitions (10-20) to get anidea of the distribution of the outputs2. If it appears normal and non-modal, perform a t-testto determine the number of repetitions required forthe desired accuracy3. If it appears non-normal and non-modal, run a t-testto get a ballpark estimate for the number ofrepetitions, and assume at least a 20% increase in thisnumber will be required4. If it appears modal, perform testing to determinewhat factors are driving the modes, and follow theprocess for fitting surrogates of modal distributionsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu46

Enablers for Complex SoS• Design of ComputerSimulations– Space Filling Designs– Adaptive DoE• Modeling and SimulationTechniques– Agent-based modeling andconstructive simulations– System Dynamics Modeling– Discrete Event Simulation– Mathematical Modeling Techniques• Non-linear SurrogateModeling– Neural Networks– Kriging/Gaussian– Stepwise RSE• Probabilistic Theory– Stochastic modeling– Surrogate modeling of stochasticprocesses– Monte Carlo Simulation• Visual AnalyticsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu47

Probabilistic Analysis on Your Desktop• The use of probabilitydistributions in designapplications was oncecumbersome and timeconsuming• Desktop tools + faster computersmake probabilistic techniquesaffordable and effective fordesign• Probabilistic assessments (oncebeyond reach) provide decisionmakers with additionalinformation to quantifyuncertainty and riskf-102⎡1⎢1⎡⎛a − µ ⎞y⎢⎜1=exp⎟ −222πσ− ⎢2− 2 ⎢yσ 11 yρ ρ2⎣ ⎣⎝σy1⎠y21y2µσ-20µ CO2 = -16%σ CO2 = 4.3%Histogram ofDataAssumingNormalDistributionsµσ-90 -70 -50 -30 -10µ NOx = -61%σ NOx = 19.8%2⎛ a − µ ⎞⎛⎞ ⎛ ⎞ ⎤⎤yb − µ −⎜1yb µ⎟⎜2y⎟ + ⎜2ρ⎟ ⎥⎥⎝ σy ⎠⎝σ ⎠ ⎝ ⎠ ⎥⎥1yσ2y2⎦⎦Probabilistic techniques must be applied to all deterministic design toolsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu48

Enablers for Complex SoS• Design of ComputerSimulations– Space Filling Designs– Adaptive DoE• Modeling and SimulationTechniques– Agent-based modeling andconstructive simulations– System Dynamics Modeling– Discrete Event Simulation– Mathematical Modeling Techniques• Non-linear SurrogateModeling– Neural Networks– Kriging/Gaussian– Stepwise RSE• Probabilistic Theory– Stochastic modeling– Surrogate modeling of stochasticprocesses– Monte Carlo Simulation• Visual AnalyticsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu49

Introduction to Visual Analytics• Challenge: How to analyze overwhelming, disparate, dynamic information• Analytics is the “science of analysis” to discover and understand patterns– Uses statistical tools and methods– Primary goal is to understand the past to predict the future• Visual Analytics is “the science of analytical reasoning facilitated byinteractive visual interfaces”– Provides a mechanism for a user to see an understand large volumes of information atonce– The brain can best process information received through visual channels– Facilitates discovery of unexpected trends and highlights transparency of underlyingphysical phenomena• Applications include Homeland Security, marketing, design and optimization,disaster management, and othersVisualization aids decision making for otherwise insurmountable problems, such as thoseoften encountered when working with SoSDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu50

Pareto Optimal Solutions in Many Dimensions• The use of GeneticAlgorithms to find ParetoFrontiers has become astandard means for multiobjectiveoptimization• When extended into multipleresponses, the concept tendsto break down as allsolutionsappear Pareto Optimal• Methods are needed to slicethe design space andvisualizemultidimensional tradeoffsResponse 3Pareto FrontResponse 2MachResponse 154321Turbofan0 1000 2000 3000 4000SpThrustSpecific ThrustResponse 3RamjetTurbojetMach54321Turbofan1 2SFCThrust Specific Fuel Consumption (TSFC)Response 2RamjetTurbojetOn a 2-D plot, thePareto front maynot be easilyapparent for asystem with morethan 2 responses.Visualizing Multi-Dimensional Optimality is Difficult for Human Decision MakersDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu51

Multi-Attribute Decision MakingHow do we select among a plethora ofalternatives in the presence of uncertaintyor missing information?How do we support those making keydecisions?•Multi-Attribute Decision Making techniquesprovide a mechanism for:– Structuring information about thealternatives– Capturing value systems– Capturing decision-maker preferences– Scoring alternatives in a traceable andrepeatable fashion•MADM techniques can be readilyimplemented in interactive, dynamic, andvisual environments to support the predecisionalassessment phase of decisionmakingDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu52

Interactive Electronic Design Reviews• Interlinking data on alarge-format visualizationwall allows decisionmakers to interact withengineers• Surrogate models,probabilistic techniques,and defined interfacestandards are required• The speed of thedecision-making/designprocess is dramaticallyenhancedDesign ProcessInteractive design reviews are necessary for SoS because of the large amount of informationand dynamic nature of tradesDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu53

Collaboration and Integration in Design• Design is, by nature, a collaborativeendeavor• Facilities to support integrated designand visualization are becoming moreaffordable– ASDL has the Collaborative VisualizationEnvironment (CoVE) and the CollaborativeDesign Environment (CoDE)• Courses that encourage collaborationand the use of new web technologiesare needed• ASDL’s Grand Challenges are a usedto foster education in collaborativedesign methodsASDL’s CoVEASDL’s CoDEwww.combier.netwww.combier.netCollaborative Visualization and Design Facilities are Becoming Affordable and WidespreadDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu54

Collaboration and Integration in SoSE• Design is, by nature, acollaborative endeavor• Facilities to support integrateddesign and visualization arebecoming more affordable• Universities should encouragecollaboration and the use ofnew web technologies• Design competitions are a goodway to foster education incollaborative design methodsCollaborative Visualization Facilities are Becoming Affordable and WidespreadDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu55

No Shortage of Research Issues and Challenges• Limits of Predictability and Controllability• Time-Varying Dependencies• Robust vs. Optimal System Designs• Failure Modes: Graceful Degradation, SystemInstabilities, Controllability• Emergent Behaviors and UnpredictableEnvironments• Evolvability, Flexibility, and Responsiveness, andthe Role of Redundancy• Characteristics of Complex Systems (MultipleHeterogeneous Systems, Distributed, NotNecessarily Co-Located, Human-Social / TechnicalInteractions)• Difficult Decision Making Environments, and theImportance of Human/System Interactions• Time-Dependent Systems and the Importance ofUnderstanding Network Dynamics• Operational Independence of Elements• Managerial Independence of Elements• Impacts of Policy and Development/ AcquisitionDecisions on Readiness and Capability• Architectures (Distributed, Centralized,Heterogeneous, Hierarchical, Hybrids)• Experimentation (Component vs. System-Level,Interactions, Emergent Behaviors, Co-Evolutionof Technologies, Manned and UnmannedInteractions)• Modeling and Simulation (Static, Dynamic,Simulation Based Design, Agent-based)• Energy Integration with Emerging Technologies• Capability-Based Design Solutions for Robustnessto Changing Threat Scenarios• Metrics (Traditional, Non-Traditional,Quantification of Social Aspects)• Effective Decision Making (Repercussions,Coupling Techno-Policy- Infrastructure, System &Component Boundaries, Dependencies,Propagation of Information)Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu56

Concluding Remarks• System of Systems Engineering is a young field withmany exciting research opportunities– Systems-of-Systems Engineering requires new and novelapproaches to improve the traditional SE process• A wide range of methods is needed in order to addresskey SoSE challenges in both the military and the civildomain• Additionally, increased effort is needed in training thenext generation of engineers on how to approachproblems from an SoS perspective and on the use ofcollaborative visual analytics design approaches• Opportunities exist to collaborate under the presentONR study with the Italian Navy on Agent-basedMethods with University of GenoaDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu57

Questions?

Selecting a DesignDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu59

Engineers use many types of models...CAD Modelshttp://www.ansys.com/Drawingsmcgettrickengineering.comPhysics-Based ModelsSoftware ModelsConceptual ModelsStructural ModelsMathematical ModelsConstructive ModelsDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu60

A New Paradigm Shift: Simulation Based Engineering Science• Numerous expert-opinion-based techniques exist for resource allocation– Difficult to extrapolate technology impacts to system-of-systems level where interactionsdominate• Decisions that shape the future of the United States Armed Forces must be enabledby advances in modeling and simulation tools, methods, and techniques [Shelton,2001]• According to the NSF Simulation-BasedEngineering Science Report (Feb 2006),simulation:– “can be used to explore new theories and todesign new experiments to test these theories”– “also provides a powerful alternative to thetechniques of experimental science andobservation when phenomena are not observableor when measurements are impractical or tooexpensive”Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu61

Which of these are systems?Developing Methods and Techniques for System of Systems Engineering62Dr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu62

Which are simple? Which arecomplicated? Which are complex?PendulumAnt ColonyAircraftPenDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu63

Which of these are a SoS?Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu64

What is a Markov Chain• Markov Chains are named after AA Markov, who firstbegan studying them in 1907• A Markov Chain is a mathematical model consisting ofa set of states S={s 1 ,s 2 ,…s m } where the probability oftransitioning between s i and s j at any given time step isp ij , and p ij does not depend on what states have beenvisited prior to the current time step– In other words, the future is dependent only on thepresent and not on the past• Markov PropertyP X i | X = i , X = i ,..., X = i ) = P(X = i | X =(n+1=n+1 n n n−1n−10 0n+1 n+1 nin)Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu65

Markov Chains• A Markov Chain is a directed graph in which theedge weights represent transition probabilities,and the probabilities on the edges leaving avertex sum to 1• Markov chains are used to model stochasticprocesses– For example, a Markov chain can be used to create astochastic version of the SIR model developed usingsystem dynamics– Queuing models– Bacteria growthDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu66

Markov Chains• The transition probabilitybetween state i and j isdenoted p ij• They are typicallyrepresented in atransition matrix, wherethe rows represent thepresent state and thecolumns represent thestate at the next timestepS1p 21S2p 32p 12 p 23S3Developing Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu67

Types of Markov Chains• Time-homogenous– Transition probabilities do not vary with time• Discrete Time• Branching Process• Continuous Time• Poisson Process– Special case of CTMCDeveloping Methods and Techniques for System of Systems EngineeringDr. Dimitri Mavris, dimitri.mavris@ae.gatech.edu68