Formal Verification of Synchronous Models: An Industrial Application ...
Formal Verification of Synchronous Models: An Industrial Application ...
Formal Verification of Synchronous Models: An Industrial Application ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>Formal</strong> <strong>Verification</strong> <strong>of</strong> <strong>Synchronous</strong> <strong>Models</strong>:<strong>An</strong> <strong>Industrial</strong> <strong>Application</strong> <strong>of</strong> <strong>Formal</strong> MethodsMarch 31, 2007Dr. Steven P. Miller© 2006 Rockwell Collins, Inc. All rights reserved.
Acknowledgements• Dr. Michael Whalen, Rockwell Collins Inc• Ricky Butler, NASA Langley Research Center• Dr. Mats Heimdahl, University <strong>of</strong> Minnesota• <strong>An</strong>jali Joshi, University <strong>of</strong> Minnesota• Ajitha Rathan, University <strong>of</strong> Minnesota• Lucas Wagner, Rockwell Collins Inc© 2006 Rockwell Collins, Inc. All rights reserved.2
What Problem are We Solving?• Safety-Critical S<strong>of</strong>tware Is Too ExpensiveCut Development Costs/Cycle Time in Half• Safety-Critical S<strong>of</strong>tware Is Often WrongFind 10x More Errors than Current Methods• DO-178B/ED-12B Certification Is Too ExpensiveAlready Applying This to DO-178B Developments© 2006 Rockwell Collins, Inc. All rights reserved.3
Outline <strong>of</strong> PresentationIntroductionDevelopment <strong>of</strong> the ConceptsFrom Theory to <strong>Industrial</strong> UseFuture Directions and Conclusions© 2006 Rockwell Collins, Inc. All rights reserved.4
Who Are We?A World Leader In Aviation Electronics <strong>An</strong>d Airborne/ MobileCommunications Systems For Commercial <strong>An</strong>d Military <strong>Application</strong>sCommunicationsNavigationAutomated Flight ControlDisplays / SurveillanceAviation ServicesIn-Flight EntertainmentIntegrated Aviation ElectronicsInformation Management Systems© 2006 Rockwell Collins, Inc. All rights reserved.5
Rockwell CollinsHeadquartered in Cedar Rapids, Iowa18,000 Employees Worldwide2006 Sales <strong>of</strong> $3.86 Billion© 2006 Rockwell Collins, Inc. All rights reserved.6
RCI Advanced Technology CenterCommercial SystemsAdvanced Technology CenterGovernment Systems• The Advanced Technology Center (ATC) identifies, acquires,develops and transitions value-driven technologies to support thecontinued growth <strong>of</strong> Rockwell Collins.• The Automated <strong>An</strong>alysis section <strong>of</strong> ATC applies mathematicaltools and reasoning to the production <strong>of</strong> high assurance systems.© 2006 Rockwell Collins, Inc. All rights reserved.7
Automated <strong>An</strong>alysis Section199219941996AAMP5 Microcode <strong>Verification</strong> (PVS)AAMP-FV Microcode <strong>Verification</strong> (PVS)AAMP5 Partitioning (PVS)NASA LaRC FundedNSA FundedAFRL FundedTech Transfer199820002002NASAFGS Mode Confusion Study (PVS)FGS Safety<strong>An</strong>alysis(RSML -e )FGS ModeConfusion(RSML -e )AvSSPAFRLJEM Java Virtual Machine (PVS)FCP 2002 Microcode (ACL2)AAMP7 Separation Kernel (ACL2)NSA2004FCS 5000 FGSDisplays<strong>Verification</strong> (NuSMV)<strong>Verification</strong> (NuSMV)GreenHillsIntegrity RTOS(ACL2)SHADE(ACL2)vFaat(ACL2,PVS)2006CerTA FCS(NuSMV, Prover, ACL2)Turnstile(ACL2)2008© 2006 Rockwell Collins, Inc. All rights reserved.8
Methods and Tools for Flight Critical SystemsProject• Five Year Project Started in 2001• Part <strong>of</strong> NASA’s Aviation Safety Program(Contract NCC-01001) Ricky Butler - COTR• Funded by the NASA Langley ResearchCenter and Rockwell Collins• Practical <strong>Application</strong> <strong>of</strong> <strong>Formal</strong> Methods ToModern Avionics Systems© 2006 Rockwell Collins, Inc. All rights reserved.9
Convergence <strong>of</strong> Two TrendsModel-BasedDevelopmentAutomated<strong>An</strong>alysisLeading to a Revolutionary Change inHow We Design and BuildEmbedded Systems© 2006 Rockwell Collins, Inc. All rights reserved.10
Model-Based Development ExamplesCompany Product Tools Specified & Autocoded Benefits ClaimedAirbus A340 SCADEWith CodeGeneratorEurocopterGE &LockheedMartinSchneiderElectricUSSpacewarePSACSEETransportHoneywellCommercialAviationSystemsEC-155/135AutopilotFADEDC EngineControlsNuclear PowerPlant SafetyControlSCADEWith CodeGenerator• 70% Fly-by-wire Controls• 70% Automatic Flight Controls• 50% Display Computer• 40% Warning & Maint Computer• 90 % <strong>of</strong> Autopilot• 20X Reduction in Errors• Reduced Time to Market• 50% Reduction in Cycle TimeADI Beacon • Not Stated • Reduction in Errors• 50% Reduction in Cycle Time• Decreased CostSCADEWith CodeGenerator• 200,000 SLOC Auto Generatedfrom 1,200 Design Views• 8X Reduction in Errors whileComplexity Increased 4xDCX Rocket MATRIXx • Not Stated • 50-75% Reduction in Cost• Reduced Schedule & RiskElectrical SCADE • 50% SLOC Auto Generated • 60% Reduction in Cycle TimeManagement With Code• 5X Reduction in ErrorsSystemGeneratorSubwaySignaling SystemPrimus EpicFlight ControlSystemSCADEWith CodeGeneratorMATLABSimulink• 80,000 C SLOC Auto Generated• 60% Automatic Flight Controls• Improved Productivity from20 to 300 SLOC/day• 5X Increase in Productivity• No Coding Errors• Received FAA Certification© 2006 Rockwell Collins, Inc. All rights reserved.11
Does Model-Based Development Scale?Airbus A380Length239 ft 6 inWingspan261 ft 10 inMaximum Take<strong>of</strong>f Weight 1,235,000 lbsPassengers Up to 840Range9,383 milesSystems Developed Using MBD• Flight Control• Auto Pilot• Fight Warning• Cockpit Display• Fuel Management• Landing Gear• Braking• Steering• <strong>An</strong>ti-Icing• Electrical Load Management© 2006 Rockwell Collins, Inc. All rights reserved.12
Outline <strong>of</strong> PresentationIntroductionDevelopment <strong>of</strong> the ConceptsFrom Theory to <strong>Industrial</strong> UseFuture Directions and Conclusions© 2006 Rockwell Collins, Inc. All rights reserved.13
Flight Guidance System Mode LogicReuseRequirementsElicitationAutotestModelingAutocodeSimulationAutomated<strong>An</strong>alysis© 2006 Rockwell Collins, Inc. All rights reserved.14
Captured Requirements as Shalls© 2006 Rockwell Collins, Inc. All rights reserved.15
ModelingReuseRequirementsElicitationAutotestModelingAutocodeSimulationAutomated<strong>An</strong>alysis© 2006 Rockwell Collins, Inc. All rights reserved.16
Textual (Lustre, PVS, SAL, …)node Thrust_Required(FG_Mode : FG_Mode_Type ;Airborne : bool ;In_Flare : bool ;Emergency_Descent : bool;Windshear_Warning : bool ;In_Eng_Accel_Zone : bool ;On_Ground : bool)returns (IsTrue : bool) ;letModeling NotationsTabular (RSML -e , SCR)IsTrue =(FG_Thrust_Mode(FG_Mode) andAirborne)or(Airborne and Emergency_Descent)orWindshear_Warningor((FG_Mode = ThrottleRetard) andIn_Flare)or(In_Eng_Accel_Zone and On_Ground) ;tel ;Graphical (Simulink, SCADE)© 2006 Rockwell Collins, Inc. All rights reserved.17
ReuseSimulationRequirementsElicitationAutotestModelingAutocodeSimulationAutomated<strong>An</strong>alysis© 2006 Rockwell Collins, Inc. All rights reserved.18
Simulation© 2006 Rockwell Collins, Inc. All rights reserved.19
Automated <strong>An</strong>alysisReuseRequirementsElicitationAutotestModelingAutocodeSimulationModel Checkers© 2006 Rockwell Collins, Inc. All rights reserved.Automated<strong>An</strong>alysisTheorem Provers20
What Are Model Checkers?• Breakthrough Technology <strong>of</strong> the 1990’s• Widely Used in Hardware <strong>Verification</strong> (Intel, Motorola, IBM, …)• Several Different Types <strong>of</strong> Model Checkers– Explicit, Symbolic, Bounded, Infinite Bounded (SMT), …• Exhaustive Search <strong>of</strong> the Global State Space– Consider All Combinations <strong>of</strong> Inputs and States– Equivalent to Exhaustive Testing <strong>of</strong> the Model– Produces a Counter Example if a Property is Not True• Easy to Use– “Push Button” <strong>Formal</strong> Methods– Very Little Human Effort Unless You’re at the Tool’s Limits• Limitations– State Space Explosion (10 100 –10 200 States)© 2006 Rockwell Collins, Inc. All rights reserved.21
Advantage <strong>of</strong> Model CheckingTesting Checks Only the Values We SelectModel Checker Tries Every Possible Value!Even Small Systems Have Trillions(<strong>of</strong> Trillions) <strong>of</strong> Possible Tests!Finds every exception to theproperty being checked!© 2006 Rockwell Collins, Inc. All rights reserved.22
ModelModel Checking ProcessAutomatic TranslationSMVSpec.Does the systemhave property X?Yes!Counter ExampleAutomated CheckSMVEngineerProperties© 2006 Rockwell Collins, Inc. All rights reserved.Automatic TranslationSMV Properties23
Translated Shalls into SMV Properties© 2006 Rockwell Collins, Inc. All rights reserved.24
Validate Requirements through Model Checking• Proved Over 280 Properties in Less Than an Hour• Found Several Errors• Some Were Errors in the Model• Most Were Incorrect Shalls• Revised the Shalls to Improve the Requirements© 2006 Rockwell Collins, Inc. All rights reserved.25
What are Theorem Provers?• Available Since Late 1980’s– Widely Used on Security Systems• Use Rules <strong>of</strong> Inference to Prove New Properties– Also Consider All Combinations <strong>of</strong> Inputs and States– Also Equivalent to Testing with an Infinite Set <strong>of</strong> Test Cases– Generate <strong>An</strong> Unprovable Pro<strong>of</strong> Obligation if a Property is False• Not Limited by State Space– Applicable to Almost <strong>An</strong>y <strong>Formal</strong> Specification• Limitations– Require Experience - About Six Months to Become Pr<strong>of</strong>icient– Constructing Pro<strong>of</strong>s is Labor Intensive© 2006 Rockwell Collins, Inc. All rights reserved.26
ModelTheorem Proving Using PVSAutomatic TranslationPVSSpec.Why not?Does the systemhave property X?GuruAutomated Pro<strong>of</strong>PVSAutomatic TranslationEngineerPropertiesPVS Properties© 2006 Rockwell Collins, Inc. All rights reserved.27
Validate Requirements Using Theorem Proving• Proved Several HundredProperties Using PVS• More Time Consumingthat Model-Checking• Use When Model-CheckingWon’t Work© 2006 Rockwell Collins, Inc. All rights reserved.28
Outline <strong>of</strong> PresentationIntroductionDevelopment <strong>of</strong> the ConceptsFrom Theory to <strong>Industrial</strong> UseFuture Directions and Conclusions© 2006 Rockwell Collins, Inc. All rights reserved.29
Original Tool ChainRSML -e to NuSMVTranslatorNuSMV Model CheckerRSML -eRSML -e to PVSTranslatorPVS Theorem ProverRockwell Collins/U <strong>of</strong> MinnesotaSRI International© 2006 Rockwell Collins, Inc. All rights reserved.30
Conversion to SCADESCADENuSMVLustrePVSSafe StateMachinesDesignVerifierRockwell Collins/U <strong>of</strong> MinnesotaEsterel TechnologiesSRI International© 2006 Rockwell Collins, Inc. All rights reserved.31
Extension to MATLAB SimulinkSimulinkSimulinkGatewaySCADENuSMVLustrePVSStateFlowSimulinkGatewaySafe StateMachinesDesignVerifierRockwell Collins/U <strong>of</strong> MinnesotaEsterel TechnologiesSRI InternationalMathWorks© 2006 Rockwell Collins, Inc. All rights reserved.32
Current Tool ChainSimulinkSimulinkGatewaySCADENuSMVACL2ReactisLustrePVSStateFlowSimulinkGatewaySafe StateMachinesProverDesignVerifierICSRockwell Collins/U <strong>of</strong> MinnesotaEsterel TechnologiesSRI InternationalMathWorksReactive SystemsSALSymbolicModel CheckerBoundedModel CheckerInfiniteModel Checker© 2006 Rockwell Collins, Inc. All rights reserved.33
Translator OptimizationsModelCPU Time(To Compute ReachableStates)ImprovementBeforeAfterMode1> 2 hours11 sec∞Mode2> 6 hours169 sec∞Mode3> 2 hours14 sec∞Mode48 minutes< 1 sec480xArch34 sec< 1 sec34xWBS29+ hours1 sec105,240x© 2006 Rockwell Collins, Inc. All rights reserved.34
Example 1FCS 5000 Mode LogicMode Controller A6.8 x 10 21 Reachable StatesMode Controller BRequirementMode A1 => Mode B1Counterexample Found inLess than Two Minutes!Found 27 Errors to Date© 2006 Rockwell Collins, Inc. All rights reserved.35
Example 2ADGS-2100 Adaptive Display & Guidance System4,295 Subsystems16,117 Simulink BlocksOver 10 37 Reachable StatesExample Requirement:Drive the Maximum Number <strong>of</strong> Display UnitsGiven the Available Graphics ProcessorsCounterexample Found in 5 Seconds!Checking 573 Properties -Found 98 Errors!© 2006 Rockwell Collins, Inc. All rights reserved.36
Example 3 – LM Aero UAV Sensor VotingOFP Triplex Voter•96 Simulink Subsystems• 3 Stateflow Diagrams•6x10 13 Reachable States1syncsync2input_a3input_b4input_c5status_a6status_b7status_c8dst_index[trigger][A][B][C][status_a][status_b][status_c][DSTi][trigger][A][B][DSTi]DSTData StoreReadIndexVectorinput_ainput_bExtract Bits u16[0 3]f ailreportfailreport[MS][status_a][status_b][status_c][prev_sel][A][B][C]mon_f ailure_reportstatus_astatus_bstatus_cf ailure_reportprev_selinput_ainput_binput_c[DSTi]f ailure_reportdst_indexFailure_Processing<strong>Formal</strong> <strong>Verification</strong>• 25 Informal Requirements• 57 <strong>Formal</strong> Properties• 40 Minutes to <strong>An</strong>alyzeDOCText[C]trip_leveltrip_level1persist_limpersist_limpersistence limit[MS]totalizer_limpersistence limit1trip_levelinput_ctrip_levelpersist_limMStotalizer_limtriplex_input_monitorpctcpersistence_cnt3totalizer_cnttotalizer_cnt2persistence_cnt[trigger][A][B][C][DSTi]Failure_Isolationpctriggerinput_ainput_selinput_binput_cDST_indextriplex_input_selectorResulting In• 24 Counterexamples• 10 Design Modifications• Several Requirements Clarifications1failure_report4input_sel1zUnit Delay[prev_sel]<strong>Formal</strong> verification has found subtle errors thatwould likely be missed by traditional testing.© 2006 Rockwell Collins, Inc. All rights reserved.- Lockheed Martin37
Outline <strong>of</strong> PresentationIntroductionDevelopment <strong>of</strong> the ConceptsFrom Theory to <strong>Industrial</strong> UseFuture Directions and Conclusions© 2006 Rockwell Collins, Inc. All rights reserved.38
Extending the <strong>Verification</strong> Domain• Large Finite Systems (
Globally Asynchronous/Locally <strong>Synchronous</strong> ArchitecturesLeftFGSL►RChannelRightFGSR ►LChannel• Occurs Naturally in System <strong>Models</strong>• Model Asynchrony by Adding Enabling Clocks• Interleaving Leads to State Space Explosion• Need to Exploit the Constraints <strong>of</strong> Real Systems© 2006 Rockwell Collins, Inc. All rights reserved.40
System Architectural Modeling & <strong>An</strong>alysisSystemArchitectureModelLogicalAbstractsSecurity<strong>An</strong>alysisPerformance<strong>An</strong>alysisADLLevel BClassifiedImplementsAutoGenerateSafety<strong>An</strong>alysisSimulinkModelCCodeIMA CabinetVAPSModelCCodeCommon Computing Resource 3Common Computing Resource 2Common Computing Resource 1App A App B App CSys Specific Middleware(Schedule, Communication Routes)Reusable Trusted Middleware(RTOS, I/O , RT-CORBA)Separation KernelTarget HardwareLevel CUnclassifiedAdaCodeLevel ATop SecretS<strong>of</strong>tware Component DevelopmentPhysicalIMA BUSSystem Architecture Development© 2006 Rockwell Collins, Inc. All rights reserved.41
Model-Based Safety <strong>An</strong>alysisGreen PumpBlue PumpLoss AllBr a k i n gIsolation ValveIsolation ValvePower APedal 1PlantFeed backPedal 2Power BSystemASystemBFault TolerantBraking SystemControl Unit( BSCU )ShutNormalSystem<strong>An</strong>tiSkidCommandBraking +<strong>An</strong>tiSkidCommandNORMALSelector ValveMeterValveALTERNATEAccumulatorValveMeterValveMeterValveAccumulatorPumpMechanicalPedalPlantModelGreen PumpLos sNo r ma l Sy sLos sMet er Val veLos sPowerSuppliesFai lBSCU Lo s so f Comma n dBSCU Sel ectSi gnalInvertedAcc/ AS/ MechMet er Fai l sAlt SysLos sBlue FailsBo t h Pump sFa i lAcc FailsSel Val veSt uck• Model the Digital Controller Architecture and the Physical System• Add Fault Model for Physical Systemand Digital Controller Architecture• Integrates System and Safety Engineering About a Common Model• Automation Enables “What-If” Consideration <strong>of</strong> System Designs© 2006 Rockwell Collins, Inc. All rights reserved.42
Conclusions• Model-Based Development is the <strong>Industrial</strong> Use <strong>of</strong> <strong>Formal</strong> Methods– Providing the Modeling Language Has a <strong>Formal</strong> Semantics– Graphical Notations are Essential for Acceptance by Developers• Convergence <strong>of</strong> Model-Based Development and <strong>Formal</strong> <strong>Verification</strong>– Key is to Get Developers Producing Specifications that Can be <strong>An</strong>alyzed– More Powerful Tools Can Verify the <strong>Models</strong> the Developers Produce• <strong>Synchronous</strong> Languages Make <strong>Formal</strong> <strong>Verification</strong> Simpler– Constrain <strong>Models</strong> in Ways that Developers Will Accept– Fit Naturally With Model-Checking and Theorem Proving• Directions for the Future– Making <strong>Verification</strong> Tools More Powerful and Easier to Use– <strong>An</strong>alysis <strong>of</strong> Globally Asynchronous/Locally <strong>Synchronous</strong> Architectures– Modeling and <strong>An</strong>alysis <strong>of</strong> System Architectures© 2006 Rockwell Collins, Inc. All rights reserved.43
For More Informationhttp://shemesh.larc.nasa.gov/fm/fm-collins-intro.html• Steven P. Miller, Alan C. Tribble, Michael W. Whalen, Mats P. E. Heimdahl, Providing theshalls, International Journal on S<strong>of</strong>tware Tools for Technology Transfer (STTT), Feb 2006.• <strong>An</strong>jali Joshi, Mats P.E. Heimdahl, Steven P. Miller, and Mike W. Whalen, Model-BasedSafety <strong>An</strong>alysis, NASA Contractor Report NASA-2006-CR213953 , February 2006. Availableat http://hdl.handle.net/2002/16163.• Michael W. Whalen, John D. Innis, Steven P. Miller, and Lucas G. Wagner, ADGS-2100Adaptive Display & Guidance System, NASA Contractor Report NASA-2006-CR213952, Feb.2006. Available at http://hdl.handle.net/2002/16162.• Steven P. Miller, Mike W. Whalen, Dan O’Brien, Mats P.E. Heimdahl, and <strong>An</strong>jali Joshi, AMethodology for the Design and <strong>Verification</strong> <strong>of</strong> Globally Asynchronous/Locally <strong>Synchronous</strong>Architectures, NASA Contractor Report NASA/CR-2005-213912, Sept. 2005. Available athttp://hdl.handle.net/2002/15934.• Alan C. Tribble, Steven P. Miller, and David L. Lempia, S<strong>of</strong>tware Safety <strong>An</strong>alysis <strong>of</strong> a FlightGuidance System, NASA Contractor Report CR-2004-213004, March 2004, available athttp://techreports.larc.nasa.gov/ltrs/dublincore/2004/cr/NASA-2004-cr213004.html.© 2006 Rockwell Collins, Inc. All rights reserved.44
Change language