Evaluating costs and benefits of model-based development of ... - ESI

Evaluating costs and benefits of model-based development ofembedded software systems in the car industry – Results of aqualitative Case StudyCase Study results, Paris June 15 th 2010ECMFA 2010 Workshop: C2M:EEMDDSascha KirstanPublic voidInt i=0;While (i!=0)010010011101001101110

Agenda1 – Introduction2 – Main results of the case study3 – Identification of Best Practices in model-based design

Classification of the case study in the overall pictureThe conducted case study is part of an intensive investigation of the economics of modelbasedsoftware development (MBSD), which is conducted as a PhD thesis at theuniversity of technology in Munich1Setting up a theory Identify the differences inthe development processbetween classic andmodel-based development Analyze the impact ofthese differences on costs,time and quality2Case Studies Design questionnaires forthe Case study Case study interviews Evaluation of the resultsChosen approach3Design of cost models Development of an analyticalcost model Development of a qualitativecost model Validating the cost modelsvia talks with experts5Validating the cost model withthe data from the study Integration of the study datain the analytical cost model Validating the cost model ata car producer with realproject data4Global study Survey of car producers,suppliers and engineeringcompanies Conducting expert talks toget data for the qualitativecost model3

Covered topics in the case studyThe case study covers versatile aspects about model-based development. Thereforeexperts from the domain power train, chassis, energy management and comfort functionswere interviewed regarding the following questionsTool supportAre you satisfied with the actualtool support? Are there anyproblems when using the tools?Process redesignHow did your company redesignthe development process? Inwhich cost factors can theredesign costs be divided?Which accompanyingprocesses were affected?Used ToolsWhich tools do you use during thedevelopment process?Model-basedSW-Development(MBSD)Motivation to use MBSDWhy is your company developingsoftware model-based?Hand-coded developmentWhich kind of functions do youdevelop hand-coded in your company.What are the reasons therefore?Statements to the econometricsHow did costs, time and qualitychange because of the use of MBSD?Negative ExperienceWhat are your negative experienceswith MBSD in projects?Positive ExperienceWhat are your positive experienceswith MBSD in projects?Error detectionIs an earlier error detection possiblewith the use of MBSD?4

Agenda1 – Introduction2 – Main results of the case study3 – Identification of Best Practices in model-based design

Process redesign (1/2)Main cost drivers in the process redesign are not just tool costsCosts for the process redesignMain cost factorsInfluence on theCostsInfluence on theTimeLicence and support costs for tools High NoneDefinition and introduction of a new development process High MediumChanges in the organisational structure High MediumTraining of the employees Medium HighCosts for living the process Medium HighCosts for the improvement of the development process High HighDesign of already hand-coded developed projects High High6

Process redesign (2/2)Before model-based development is used for a series project, several pilot studies arebeing conductedUsed procedure for the process redesignEvaluation of toolsAt the beginning all available tools on the market are being analysed andbalanced. Therefore tool evaluations are being conductedDrafting of a newdevelopment processConduction of pilot studiesAdoption of the definedprocessConduction of further pilotstudiesFirst series projectWith the results of the tool evaluation the development of the newdevelopment process is being started. Depending on the used tool chainthe process has to be adapted to the companies needsThe developed process is validated via pilot studies. Pilot studies help theengineers to get used to the new development process without any pressureof a series project and to be able to judge the benefit of the new approachUsually process improvement potential is already identified in the first pilotstudy. The improvement potential is included in the iterative design of thedevelopment processThe case studies are iteratively continued until the process is mature for aseries developmentThe first series project shows, whether the defined process is suited for theseries development. If the first series project is conducted successfully thenew development process is being spread in the whole company7

Motivation to use MBSDDifferent reasons convince companies to use model-based designMotivation to use MBSDCompany internalmotivations Cost savings in the development The possibility to simulate in particular with Rapid Control Prototyping(RCP) allows the development of very innovative functions Companies are afraid to miss a next generation technologyAdvantages in thedevelopment Higher automation in the development process Possibility to detect errors earlier Easier communication with colleagues Light effort for reusing function models in different car linesCompetitive pressurebetweenthe suppliers Customer (OEM) demands that suppliers use model-baseddevelopment Supplier want to impress the customer (OEM) that they alreadyuse the newest development technology8

Hand coded developmentSome functions are still being developed hand coded. Not only technical aspects, butmainly company internal reasons are responsible for thatHand-coded developed functionsSaftey relevantfunctionsLow-Level-SoftwareFunctions with highspeed requirementsSafety relevant functions: Trust in code generatoris too lowFunctions with extremely high speed requirements:The experience shows that hand coded functionsare more efficient in RAM and ROM needsLow-Level-Software (Functions for diagnose, basissoftware)Company intern reasons for hand coded developmentModel-based development process is not established in all departments Costs for the processredesign are too high (especially the licence costs for the tools)On the supplier side the necessity to develop model-based is sometimes not seen. The reasontherefore is, that suppliers already use reuse concepts in hand-coded projects. This ensures that theycan reuse their hand-coded developed functions9

Negative experiences with MBSDThe negative experiences can be divided into four aspectsNegative experiences with MBSDToolsSimulationDependency on tool vendors: Tool restrictedinterfacesMaturity of toolsNo seamless tools over the whole developmentprocess availableDetailed knowledge necessary: Sometimes theKnow-How is not available in the companies.Following simulations are not conductedintensive enoughSimulations are time consuming especially if thefunction models are huge Simulations aredisregarded at high time pressureDevelopment of unstructured codeReusabilityFunction models are sometimes models whichhave gained in functionality over the years. Thegenerated code out of these function models isoften unstructured and inefficient Applicationof modelling rules and reuse concepts isimportantEven model-based developed functions can notbe reusable10

Positive experience with MBSDThe positive experiences can be divided into six aspectsPositive experiences with MBSDEasier communicationEasier development ofcomplex functionsSimulationThe graphical description offunction models allows aneasier communicationScreenshots of functionmodels can be used inpresentationsWith the use of RCPengineers can develop highinnovative functions via try anderror on a real carThe RCP-Model often servesas an executable specificationRCP allows to developcertain functionsCombination of RCP andModel in the Loop (MiL)allows an earlier errordetectionOverview over the wholesystemReusabilityPossibility to generate codeout of function modelsThe graphical descriptionallows to get an overview overthe whole system. It is easy tosee which functions interactamong each otherWith the use of a reuseconcept a function model canbe reused in different car lineswithout much additional effortAll case study participantsuse the automatic codegeneration out of the functionmodel11

Used toolsTools for model-based development are intensively used in the software design andthe implementationUsed tools in the development processSystem requirementsSystem architectureSoftware requirementsSoftware architectureSoftware designImplementationTestWord, DoorsPowerpoint, Visio, ExcelWord, Doors, Matlab/Simulink/Stateflow, AscetPowerpoint, Visio, Excel, Matlab/Simulink/Stateflow, AscetMatlab/Simulink/Stateflow, AscetTargetLink, RealTime WorkshopTPT, MTest, Polyspace, own developed tools• Tool support is seen as a big added value• The engineers need a lot of experience to exploit the full potential of the tool support (forexample simulations)• Companies moan about the dependency on the tool vendors, because the tools don't offeropen interfaces12

Error detection in the development processThe more intensive tests on models are conducted, the less errors are found in thefollowing development phasesChanges in the error detectionBeginning of theerror detectionUse of RCPUse of MiL73% start the error detection in the requirements analysis withRCP, 27% begin in the SW-Design73% use RCP and even 63% of them use it intensively100% use MiL-Tests and even 81% of them use it intensively Considerable fewer errors in the implementation and the whole test process after theimplementation phase Reasons: RCP, MiL-Tests, model reviews and the automatic generation of code Impact of MBSD on the maintenance:Effect of earlier error detection on the number of discovered errorsFewer errors in the series Fewer number of updates necessary63 % of the case study participants even report a higher product quality13

Statements about the econometricsCase study participants report about positive impacts on the econometricsStatements about changes in the econometrics because of MBSDCostsTimeProductqualityEarlier error detection because of the use of simulations, highautomation degree and easier communication. The highest cost savingpotential is seen in the reuse of function models over different carlinesEarlier error detection because of the use of simulations, highautomation degree and easier communication. The highest cost savingpotential is seen in the reuse of function models over different carlinesFunctions have a higher maturity level Impact on the maintenanceParticipants report improvements of the quality criteria reliability,usability, maintainability and portabilityInteresting cognition: The domain has no impact on cost, time and quality changes. Thishas even been confirmed by a case study participant, who has made this experience inseveral projects in different domains14

Agenda1 – Introduction2 – Main results of the case study3 – Identification of Best Practices in model-based design

From the case study interviews Best Practices can bederievedOnly when you develop model-based in the “right“ way, you will be able to achieve cost savingsBest Practice Approaches A reuse concept is also vital when developing model-based Defined change overs in the development process are more important in the praxis than to erasethe seamlessness of the development process, if you have already invested a large effort inerasing the seamlessness in your development process (usually more than 5 years usage) Using the potential of simulations over the whole development process. Particularly whendeveloping innovative functions an intensive use of RCP and MiL is recommended Training of the employees about best practices in software engineering concepts like e.g.modularisation. These concepts are very important to develop reusable and efficient code out ofthe function models. This knowledge can decide whether a project is conducted successfully orfails Size of the function model: It is advisable to use more small function models than one largefunction model Selection of the appropriate tool chain is a central aspect during the process redesign. Thedependency on tool vendors is very high and consequently a change in the tool chain iscomparable to a complete process redesign16

SummaryThe strengths and the potential of MBSD are prized by our case study participantsStrengths of MBSDWeaknesses of MBSD- Easier Communication- Higher automation degree- Reusability over different car lines- Possibility to simulate models leads to an earliererror detection- High complex functions can be developed easier- Costs savings and improvements in quality- Challenges with tools- High investments are needed for introducing andoptimizing the development process- Dependency on tool vendors- It is a key factor to know, how to develop modelbasedthe “right“ way. Otherwise there is a high errorpotential in developing model-based- Vision: Model-based development from requirementstill test with the aim to achieve synergy effects,because of more intensive model-based designProblem: Appropriate tools not available- Current challenges: Expanding the model-baseddevelopment on the architecture design and modelbasedtestingPotential of MBSD- Huge Know - How necessary to get the full potentialout of MBSD. Without this Know-How MBSD can becounterproductiveRisks of MBSD- Investments in the long run are necessary Forsmall companies maybe to expensive- Dependency on tool vendors17

www.altran.de

Backup slidesRCPFunction ModelBased on FloatingpointsCan be conducted on a real car or on a test benchMiLFunction ModelEnvironment Model19

Backup slidesSiLEnvironment ModelPiLEnvironment Model20