mapping between iso 9126 on software product quality metrics and ...

MAPPING BETWEEN ISO <strong>9126</strong> ON SOFTWARE PRODUCT QUALITYMETRICS AND ISO 12207 ON SOFTWARE LIFE CYCLE PROCESSESRAFA E. AL-QUTAISHÉcole de Technologie Supérieure (ÉTS)University of Québec1100 Notre-Dame St., Montréal, QC, H3C 1K3,Canadarafa.al-qutaish.1@ens.etsmtl.caHAMED J. AL-FAWAREHDept. of Computer ScienceZarqa Private UniversityP. O. Box 2000, Zarqa 13110,Jordanfawareh@zpu.edu.joABSTRACTNowadays, the International Organization for Standardization (ISO) is working on the next generation of the softwareproduct quality standards which will be referred to as Software Product Quality Requirements and Evaluation (SQuaRE– ISO 25000 series). However, this series of standards will replace the current version of ISO <strong>9126</strong> InternationalStandard which consists of inventories of proposed metrics to measure the quality of the internal, external, and in-usesoftware product. For each of these metrics there is a cross-reference on where they could be applied (measured) duringthe ISO 12207 Software Life Cycle Processes and activities (SLCP). This paper provides a <strong>mapping</strong> <strong>between</strong> those twostandards to highlights the weaknesses of these cross-references and proposes a number of suggestions to address them.Keywords: Software Measurement, Software Quality Metrics, Software Life Cycle Processes (SLCP), ISO <strong>9126</strong>, ISO12207.1. INTRODUCTIONMeasurements have a long tradition in natural sciences.At the end of the 19 th century the physicist, LordKelvin, formulated the following about measurement:“When you can measure what you are speaking about,and express it into numbers, you know some thing aboutit. But when you can not measure it, when you can notexpress it in numbers, your knowledge is of a meagerand unsatisfactory kind: It may be the beginning ofknowledge, but you have scarcely in your thoughtsadvanced to stage of science” [20].Moreover, Roberts [21] points out in his book aboutmeasurement theory that: “A major difference <strong>between</strong>well-developed sciences such as physics and some of theless well-developed sciences such as psychology orsociology is the degree to which things are measured”.In the area of software engineering, the concept ofsoftware measurement (or what is commonly calledsoftware “metrics”) is not new. Since 1972, a number ofso-called software “metrics 1 ”, or “measures”, have beendeveloped. From the wide range of software measures,four basic theories have been the source of the majorityof the research conducted on software measurement.Some of these measures have been defined by Halstead[7, 8], Albrecht [3], DeMarco [5], and McCabe [19].1 While the term “metrics” is used in ISO <strong>9126</strong>, the useof this term will be abandoned and replaced by“measures” in the upcoming new ISO 25000 as aninitial step towards harmonizing the softwareengineering measurement terminology with the ISO15939. [11]The definition of a “measure” is an empiricalobjective assignment of a number or a symbol to anentity to characterize a specific attribute [6]. Moreover,Ince et al. [11] have defined the software “metrics” asnumerical values of quality which can be used tocharacterized how good or bad that the product is interms of properties such as its proneness to error. InAddition, “metrics” defined in [10] as quantitativemeasures of the degree to which a system, component,or process possesses a given attribute, while within theISO 15939, it was defined as a variable to which a valueis assigned as a result of measurements [15].In order to standardize the software product qualitymeasurement process, in 1991, the ISO published itsfirst international consensus on the terminology for thequality characteristics for software product evaluation;this standard was called as Software Product Evaluation- Quality Characteristics and Guidelines for Their Use(ISO <strong>9126</strong>: 1991) [14].From 2001 to 2004, the ISO published an expandedversion, containing both the ISO quality models andinventories of proposed measures for these models (ISO<strong>9126</strong> parts 1, 2, 3, and 4) [12, 16-18].Recently, the ISO has recognized a need for furtherenhancement of ISO <strong>9126</strong> International Standard,primarily as a result of advances in the fields ofinformation technologies and changes in environment[4]. Therefore, the ISO is now working on the nextgeneration of software product quality standards [22],which will be referred to as Software Product QualityRequirements and Evaluation (SQuaRE – ISO 25000series). This series of standards will replace the current

ISO <strong>9126</strong> International Standard. However, manyresearches have focused on some weaknesses on thecurrent ISO <strong>9126</strong> and even on the draft versions of theupcoming new ISO 25000 series of standards (SQuaRE)[1, 2].The current version of the ISO <strong>9126</strong> consists ofinventories of proposed metrics to measure the qualityof the internal, external, and in-use software product.However, for each of these metrics there is a crossreferenceon where they could be applied (measured)during the ISO 12207 Software Life Cycle Processesand activities (SLCP). This paper provides a <strong>mapping</strong><strong>between</strong> these two standards to highlights theweaknesses of these cross-references and proposes away to address them.This paper is organized as follows: section 2 presentsan overview of the related software engineeringstandards, that is, ISO <strong>9126</strong> and ISO 12207. Section 3shows a detailed <strong>mapping</strong> of the ISO <strong>9126</strong> metrics towhere they could be measured during the Software LifeCycle Processes (SLCP) provided by ISO 12207.Section 4 discusses the results of this <strong>mapping</strong>. Finally,Section 5 concludes the paper with some comments andsuggestions.2. RELATED SOFTWAREENGINEERING ISO STANDARDS2.1 ISO <strong>9126</strong>The ISO <strong>9126</strong> series of standards now consists of oneInternational Standard [12] and three Technical Reports[16-18]:1. ISO <strong>9126</strong>-1: Quality Model [12].2. ISO TR <strong>9126</strong>-2: External Metrics [16].3. ISO TR <strong>9126</strong>-3: Internal Metrics [17].4. ISO TR <strong>9126</strong>-4: Quality in Use Metrics [18].The first document of the ISO <strong>9126</strong> series – QualityModel – contains two-parts quality model for softwareproduct quality [12]:1. Internal and external quality model.2. Quality in use model.The first part of the two-parts quality modeldetermines six characteristics in which they aresubdivided into twenty-seven subcharacteristics forinternal and external quality, as in Figure 1 [12]. Thesesubcharacteristics are a result of internal softwareattributes and are noticeable externally when thesoftware is used as a part of a computer system. Thesecond part of the two-part model indicates four qualityin use characteristics, as in Figure 2 [12].External and Internal Quality1. Functionality2. Reliability3. Usability4. Efficiency5. Maintainability6. Portability1.1 Suitability1.2 Accuracy1.3 Interoperability1.4 Security1.5 FunctionalityCompliance2.1 Maturity2.2 FaultTolerance2.3 Recoverabilit2.4 ReliabilityCompliance3.1 Understandability3.2 Learnability3.3 Operability3.4 Attractiveness3.5 UsabilityCompliance4.1 TimeBehavior4.2 ResourceUtilization4.3 EfficiencyCompliance5.1 Analyzability5.2 Changeability5.3 Stability5.4 Testability5.5 MaintainabilityCompliance6.1 Adaptability6.2 Installability6.3 Co-existence6.4 Replaceability6.5 PortabilityComplianceFigure 1: ISO <strong>9126</strong> Quality Model for External and Internal Quality (Characteristics and Subcharacteristics) [12].Quality in use1. Effectiveness 2. Productivity 3. Safety 4. SatisfactionFigure 2: ISO <strong>9126</strong> Quality Model for Quality in Use (characteristics) [12].The second, third, and fourth documents of the ISO<strong>9126</strong> series provide the following information [16]:1. Sets of metrics for each external quality subcharacteristic,internal quality subcharacteristic,and quality in usecharacteristic.2. Explanations of how to apply and use thesesets of metrics.3. Examples of how to apply these metricsduring the software product lifecycle.2.2 ISO 12207It consists of processes, activities for each process, andtasks for each activity [9, 13]. Figure 3 shows thesoftware life cycle processes, the number of activities ineach process, and the number of tasks in each process.The full list of the process, activities, and tasks can beseen in ISO 12207 and IEEE/EIA 12207 (the IEEE/EIA12207 is the IEEE version of the ISO 12207).

The ISO 12207 software life cycle processes aregrouped into three broad classes: primary; supporting;and organizational. Primary processes are the primemovers in the life cycle; they are acquisition, supply,development, operation, and maintenance. Supportingprocesses are documentation, configurationmanagement, quality assurance, joint review, audit,verification, validation, and problem resolution. Asupporting process supports another process inperforming a specialized function. Organizationalprocesses are management, infrastructure,improvement, and training. An organization mayemploy an organizational process to establish, control,and improve a life cycle process.Groups5.PrimaryProcesses6.SupportingProcesses7.OrganizationalProcessesProcesses5.1 Acquisition5.2 Supply5.3 Development5.4 Operation5.5 Maintenance6.1 Documentation6.2 Configuration Management6.3 Quality Assurance6.4 Verification6.5 Validation6.6 Joint Review6.7 Audit6.8 Problem Resolution7.1 Management7.2 Infrastructure7.3 Improvement7.4 TrainingNumber ofActivities5Number ofTasks237 2413 554 96 244 76 64 162 132 103 82 82 25 123 53 63 4Figure 3: ISO 12207 Software Life Cycle Processes, Activities, and Tasks.3. MAPPING BETWEEN ISO <strong>9126</strong>AND ISO 12207For each metric of the internal, external, and in-usemetrics, the ISO <strong>9126</strong> parts 2, 3, and 4 provides thefollowing information:• Metric name.• Purpose of the metric.• Method of application.• Measurement formula.• Interpretation of Measured value.• Metric scale type.• Measure type.• Input to measurement.• ISO 12207 SLCP Reference.• Target audience.Within the following subsections, detailed <strong>mapping</strong>s<strong>between</strong> the ISO <strong>9126</strong> quality metrics of the Internal,external, and in-use software product and the ISO12207 software life cycle processes and activities willbe provided. In more details, this <strong>mapping</strong> will focus onan investigation of the “ISO 12207 Software Life CycleProcesses (SLCP) References” provided by ISO <strong>9126</strong>for each of its metrics.3.1 INTERNAL QUALITY METRICSWithin the ISO <strong>9126</strong>-3 on software product internalquality metrics, there is 70 metrics. These metrics canbe applied during the software life cycle. Internalquality defined in ISO <strong>9126</strong>-1 as the totality ofcharacteristics of the software product from an internalview. Internal quality is measured and evaluated againstthe internal quality requirements. Details of softwareproduct quality can be improved during codeimplementation, reviewing and testing, but thefundamental nature of the software product qualityrepresented by internal quality remains unchangedunless redesigned [12].Figure 4 shows the number of internal qualitymetrics which can be applied (measured) during each of

the ISO 12207 software life cycle processes. Forexample, within the “verification process” (of the“supporting processes”) 59 metrics can be applied(measured). As an example, Appendix A shows adetailed structure of the software product internalquality metrics’ names and where they can be measuredduring the software life cycle processes or activitiesalong with the corresponding characteristic andsubcharacteristic for each of those metrics. In thisappendix, only the software life processes/activitieswhich have internal quality metrics are mentioned.However, from Figure 4 we can note that there is nometrics which could be measured during 4 out of 5primary life cycle processes. This means that there is noany metric from ISO <strong>9126</strong> external quality metricscould be useful during the acquisition, supply,operation, and maintenance primary life cycleprocesses. Moreover, there is no metrics which could bemeasured during 3 out of 8 of the supporting life cycleprocesses; that is, documentation, configurationmanagement, and audit processes.5. Primary Life Cycle Processes 6. Supporting Life Cycle Processes5.1 Acquisition (0 Metric)6.1 Documentation (0 Metric)5.2 Supply (0 Metric)6.2 Configuration Management (0 Metric)5.3 Development6.3 Quality Assurance (2 Metrics)(1 Metric in Software Qualification6.4 Verification (59 Metrics)Testing activity)6.5 Validation (13 Metrics)5.4 Operation (0 Metric)6.6 Joint review (59 Metrics)5.5 Maintenance (0 Metric)6.7 Audit (0 Metric)6.8 Problem Resolution (4 Metrics)Figure 4: The ISO <strong>9126</strong>-3 Internal Quality Metrics and where they could be measured in the SLCP.3.2 EXTERNAL QUALITY METRICSWithin the ISO <strong>9126</strong>-2 on software product externalquality metrics, there is 110 metrics. These metrics canbe applied during the software life cycle. Externalquality defined in ISO <strong>9126</strong>-1 as the totality ofcharacteristics of the software product from an externalview. It is the quality when the software is executed,which is typically measured and evaluated while testingin a simulated environment with simulated data usingexternal metrics. During testing, most faults should bediscovered and eliminated. However, some faults maystill remain after testing. As it is difficult to correct thesoftware architecture or other fundamental designaspects of the software, the fundamental design usuallyremains unchanged throughout testing [12].Figure 5 shows the number of external qualitymetrics which can be applied (measured) during each ofthe ISO 12207 software life cycle processes. Forexample, within the “operation process” of the “primaryprocesses”, 93 metrics can be applied (measured).5. Primary Life Cycle Processes 6. Supporting Life Cycle Processes5.1 Acquisition (0 Metric)6.1 Documentation (0 Metric)5.2 Supply (0 Metric)6.2 Configuration Management5.3 Development(0 Metric)(7 Metrics in Software Integration activity)6.3 Quality Assurance (14 Metrics)(100 Metric in Software Qualification Testing activity)6.4 Verification (0 Metrics)(7 Metrics in System Integration activity)6.5 Validation (47 Metrics)5.4 Operation (93 Metric)6.6 Joint review (0 Metric)5.5 Maintenance (48 Metric)6.7 Audit (0 Metric)6.8 Problem Resolution (1 Metric)Figure 5: The ISO <strong>9126</strong>-2 External Quality Metrics and where they could be measured in the SLCP.3.3 QUALITY IN USE METRICSWithin the ISO <strong>9126</strong>-2 on software product quality inuse metrics, there is 15 metrics. These the 15 metricscan be applied during the software life cycle. Quality inUse defined in ISO <strong>9126</strong>-1 as the user’s view of thequality of the software product when it is used in aspecific environment and a specific context of use. Itmeasures the extent to which users can achieve theirgoals in a particular environment, rather than measuringthe properties of the software itself. The term ‘user’

efers to any type of intended users, including bothoperators and maintainers, and their requirements canbe different. [12].Figure 6 shows the number of quality in use metricswhich can be applied (measured) during each of the ISO12207 software life cycle processes. For example,during the “software qualification testing” activity ofthe “development process” of the “primary processes”,12 metrics can be applied (measured).5. Primary Life Cycle Processes 6. Supporting Life Cycle Processes5.1 Acquisition (0 Metric)6.1 Documentation (0 Metric)5.2 Supply (0 Metric)5.3 Development(12 Metric in Software Qualification Testing activity)5.4 Operation (15 Metric)5.5 Maintenance (0 Metric)6.8 Problem Resolution (0 Metrics)6.2 Configuration Management(0 Metric)6.3 Quality Assurance (0 Metrics)6.4 Verification (0 Metrics)6.5 Validation (11 Metrics)6.6 Joint review (0 Metrics)6.7 Audit (0 Metric)Figure 6: The ISO <strong>9126</strong>-4 Quality in Use Metrics and where they could be measured in the SLCP.4. DISCUSSION OF THE MAPPINGIn ISO <strong>9126</strong>-3, there are some external quality metrics –as in Table 1 - which have been referred to be appliedduring the “integration” activity of the “developmentprocess” of the “primary processes”. But, within the“development” process, there are two activities relatedto the “integration”, that is, “system integration” and“software integration”. However, this document (ISO<strong>9126</strong>-3) did not specify during which “integration”activity those metrics can be applied (measured).Table 1: Some External Quality Metrics.1- Estimated latent fault 2- Incorrect operationdensityavoidance3- Failure density 4- Availabilityagainst test cases5- Failure resolution 6- Mean down time7- Fault density 8- Mean recovery time9- Fault removal 10- Restartability11- Mean time <strong>between</strong>failures (MTBF)12- User supportfunctional consistency13- Breakdown 14- Restore effectiveness15- Failure avoidance 16- RestorabilityIt is clearly noted that through the ISO 12207“organizational processes” none of the 195 qualitymetrics - found in ISO <strong>9126</strong> series of internationalstandards - can be applied (measured).As mentioned in ISO <strong>9126</strong>-1, the quality in usemetrics should be measured during the execution of thesoftware product in an actual working environment.However, in Figure 6 we can see that there is 12 metricswhich could be measured through the “softwarequalification testing” activity. But since ISO 12207mentioned that the “software qualification testing”activity is a part of the “development process” Thus, it’sstrange and make no sense to measure the 12 metricsThe “joint review process” of the “supportingprocesses” consists of three activities; one of theseactivities is the “technical reviews” activity. The“technical reviews” activity contains one task that is,“Technical reviews shall be held to evaluate thesoftware products or services under consideration andprovide evidence that: a) they are complete . . . ” [9].Now, if we go back to Figure 4, we will find that thereis 59 internal quality metrics that could be measuredduring the “joint review” process. Whereas, fromFigures 5 and 6, it is seen that there is no any externalquality or quality in use metrics that can be appliedduring “joint review” process.5. CONCLUSIONThe current edition of the ISO <strong>9126</strong> consists ofinventories of proposed metrics to measure the qualityof the internal, external, and in-use software product.However, for each of these metrics there is a crossreferenceon where it could be applied (measured)during the ISO 12207 software life cycle processes andactivities. This paper provided a <strong>mapping</strong> <strong>between</strong> thosetwo standards to investigate the cross-references<strong>between</strong> them. Based on this <strong>mapping</strong>, the followingcomments and suggestions for the upcoming new ISO25000 series of standards (SQuaRE) can be concluded:- There is no any metric can be measured duringthe “organizational processes”.- A number of external quality metrics wherementioned in ISO <strong>9126</strong>-2 to be measured duringthe “integration” activity. However, within theISO 12207 there are two activities labeled“system integration” and “software integration”.This will make the user of the ISO <strong>9126</strong>confused.

- Many of the ISO <strong>9126</strong> quality metrics referred toprocesses. However, as known, each process inISO 12207 contains a number of differentactivities. Thus, it is more usable for the ISO<strong>9126</strong> users to refer to the activities of the ISO12207. This can be done using cross-referencenumbers from ISO 12207. For example, thecross-reference number 5.3.9 is referring to“primary processes”, “development process”, and“software qualification testing” activity,respectively.In addition to the <strong>mapping</strong> in this paper, it is a goodidea to investigate where to collect the data for each ofthe ISO <strong>9126</strong> quality metrics in the ISO 12207 softwarelife cycle processes and activities. This will save timeand assure that the data have been completely collectedbefore the measurement of the metrics is performed.REFERENCES[1] Abran, A., Al-Qutaish, R. E., and Desharnais, J.M., "Harmonization Issues in the Updating of theISO Standards on Software Product Quality,"Metrics News Journal, Vol. 10, No. 2, pp. 35-44,2005.[2] Abran, A., Al-Qutaish, R. E., Desharnais, J. M.,and Habra, N., "An Information Model forSoftware Quality Measurement with ISOStandards," in Proceedings of the InternationalConference on Software Development (SWDC-REK'05), Reykjavik, Iceland, pp. 104-116, 2005.[3] Albrecht, A. J., "Measuring ApplicationDevelopment Productivity," in Proceedings of theIBM Application Development JointSHARE/GUIDE Symposium, Monetary,California, pp. 83-92, 1979.[4] Azuma, M., "SQuaRE: The next Generation ofISO/IEC <strong>9126</strong> and 14598 International StandardsSeries on Software Product Quality," inProceedings of the European Software Controland Metrics Conference (ESCOM), London, UK,pp. 337-346, 2001.[5] DeMarco, T., Controlling System Projects,McGraw-Hill, New York, USA, 1982.[6] Fenton, N. E. and Pfleeger, S. L., SoftwareMetrics: A Rigorous and Practical Approach, 2 nded., PWS Publishing Company, Boston, USA,1997.[7] Halstead, M. H., Elements of Software Science,Elsevier North-Holland, New York, 1977.[8] Halstead, M. H., "Natural Laws ControllingAlgorithm Structure," ACM SIGPLAN Notices,Vol. 7, No. 2, pp. 19-26, 1972.[9] IEEE, IIEEE/EIA-12207: Information Technology– Software Life Cycle Processes, the Institute ofElectrical and Electronics Engineers, New York,USA, 1996.[10] IEEE, Std. 610.12-1990: Standard Glossary ofSoftware Engineering Terminology, the Instituteof Electrical and Electronics Engineers, NewYork, USA, 1990.[11] Ince, D. S., Sharp, H., and Woodman, M.,Introduction to Software Project Management andQuality Assurance, McGraw-Hill, New York,USA, 1993.[12] ISO/IEC, ISO/IEC <strong>9126</strong>-1: Software Engineering- Product Quality - Part 1: Quality Model,International Organization for Standardization,Geneva, Switzerland, 2001.[13] ISO/IEC, ISO/IEC 12207: InformationTechnology - Software life cycle processes,International Organization for Standardization,Geneva, Switzerland, 1995.[14] ISO/IEC, ISO/IEC IS <strong>9126</strong>, Software ProductEvaluation - Quality Characteristics andGuidelines for Their Use, InternationalOrganization for Standardization, Geneva,Switzerland, 1991.[15] ISO/IEC, ISO/IEC IS 15939: SoftwareEngineering - Software Measurement Process,International Organization for Standardization,Geneva, Switzerland, 2002.[16] ISO/IEC, ISO/IEC TR <strong>9126</strong>-2: SoftwareEngineering - Product Quality - Part 2: ExternalMetrics, International Organization forStandardization, Geneva, Switzerland, 2003.[17] ISO/IEC, ISO/IEC TR <strong>9126</strong>-3: SoftwareEngineering - Product Quality - Part 3: InternalMetrics, International Organization forStandardization, Geneva, Switzerland, 2003.[18] ISO/IEC, ISO/IEC TR <strong>9126</strong>-4: SoftwareEngineering - Product Quality - Part 4: Quality inUse Metrics, International Organization forStandardization, Geneva, Switzerland, 2004.[19] McCabe, T. J., "A Complexity Measure," IEEETransaction on Software Engineering, vol. 2, No.4, pp. 308-320, 1976.[20] Pressman, R. S., Software Engineering: APractitioner's Approach, 3 rd ed., McGraw-Hill,New York, USA, 1992.[21] Roberts, F. S., Measurement Theory, withApplications to Decision Making, Utility, and theSocial Sciences, Add<strong>iso</strong>n Wesley, New York,USA, 1979.[22] Suryn, W., Abran, A., and April, A., "ISO/IECSQuaRE: The Second Generation of Standards forSoftware Product Quality," in Proceedings of the7 th IASTED International Conference on SoftwareEngineering and Applications, California, USA,2003.

APPENDIX AISO <strong>9126</strong>-3 INTERNAL QUALITY METRICS AND WHERE THEY COULD BE APPLIED(MEASURED) IN ISO 12207 PROCESSES AND ACTIVITIES5-Primary Processes:5.3 Development:9) Software qualification testing:1. Functional specification stability (1.1 2 )5.4 Operation:1. Functional specification stability (1.1)6-Supporting Processes:6.3 Quality Assurance:1. Functional specification stability (1.1)2. Test adequacy (2.1)6.4 Verification:1. Computational accuracy (1.2)2. Precision (1.2)3. Data exchangeability (data format based) (1.3)4. Interface consistency (protocol) (1.3)5. Functional Compliance (1.5)6. Intersystem standard compliance (1.4)7. Fault detection (2.1)8. Fault removal (2.1)9. Test adequacy (2.1)10. Failure avoidance (2.2)11. Incorrect operation avoidance (2.2)12. Restorability (2.3)13. Restoration Effectiveness (2.3)14. Reliability Compliance (2.4)15. Completeness of description (3.1)16. Demonstration capability (3.1)17. Evident functions (3.1)18. Function understandability (4.1)19. Completeness of user documentation and/orhelp facility (3.2)20. Input validity checking (3.3)21. User operation cancellability (3.3)22. User operation Undoability (3.3)23. Customizability (3.3)24. Physical accessibility (3.3)25. Operation status monitoring capability (3.3)26. Operational consistency (3.3)27. Message Clarity (3.3)28. Interface element clarity (3.3)29. Operational error recoverability (3.3)30. Attractive interaction (3.4)6.5 Validation:1. Functional adequacy (1.1)2. Functional implementation completeness (1.1)3. Functional implementation coverage (1.1)4. Functional specification stability (1.1)5. Access auditability (1.4)6. Access controllability (1.4)7. Data corruption prevention (1.4)31. User Interface appearance customizability (3.4)32. Usability Compliance (3.5)33. Response time (4.1)34. Throughput time (4.1)35. Turnaround time (4.1)36. I/O Utilization (4.2)37. I/O Utilization Message Density (4.2)38. Memory utilization (4.2)39. Memory utilization message density (4.2)40. Transmission Utilization (4.2)41. Efficiency Compliance (4.3)42. Activity recording (5.1)43. Readiness of diagnostic function (5.1)44. Change recordability (5.2)45. Change impact (5.3)46. Modification impact localization (5.3)47. Completeness of built-in test (5.4)48. Autonomy of testability (5.4)49. Test progress observability (5.4)50. Maintainability Compliance (5.5)51. Adaptability of data structures (6.1)52. Organizational Environment adaptability (6.1)53. Hardware Environmental Adaptability (H/W,network) (6.1)54. System software Environmental adaptability (OS,concurrent application) (6.1)55. Porting User Friendliness (6.1)56. Continued use of Data (6.3)57. Functional inclusiveness (6.3)58. Available co-existence (6.4)59. Portability Compliance (6.5)8. Data encryption (1.4)9. Failure avoidance (2.2)10. Incorrect operation avoidance (2.2)11. Ease of setup retry (6.2)12. Installation effort (6.2)13. Installation flexibility (6.2)2 This number refers to the characteristic-number.subcharacteristic-number which can be taken from Figure 1. Forexample, the number 1.1 refers to the “Functionality” characteristic and “Suitability” subcharacteristic which meansthat this metric (Functional specification stability) is used to measure the “Suitability” subcharacteristic in which it is apart of the “Functionality” of any software product, this metrics could be measured during the “Software qualificationtesting” activity of the “development” process. Throughout these Appendices, the Software Quality Metrics nameshave been written in italic fonts.

6.6 Joint Review:1. Functional adequacy (1.1)2. Functional implementation completeness (1.1)3. Functional implementation coverage (1.1)4. Computational accuracy (1.2)5. Precision (1.2)6. Data exchangeability (data format based) (1.3)7. Interface consistency (protocol) (1.3)8. Access auditability (1.4)9. Access controllability (1.4)10. Data corruption prevention (1.4)11. Functional Compliance (1.4)12. Intersystem standard compliance (1.4)13. Fault detection (2.1)14. Fault removal (2.1)15. Failure avoidance (2.2)16. Incorrect operation avoidance (2.2)17. Restorability (2.3)18. Restoration Effectiveness (2.3)19. Reliability Compliance (2.4)20. Completeness of description (3.1)21. Demonstration capability (3.1)22. Evident functions (3.1)23. Function understandability (3.1)24. Completeness of user documentation and/orhelp facility (3.2)25. Input validity checking (3.3)26. User operation cancellability (3.3)27. User operation Undoability (3.3)28. Customizability (3.3)29. Physical accessibility (3.3)30. Operation status monitoring capability (3.3)6.8 Problem Resolution:1. Functional specification stability (1.1)2. Test adequacy (2.1)31. Operational consistency (3.3)32. Message Clarity (3.3)33. Interface element clarity (3.3)34. Operational error recoverability (3.3)35. Attractive interaction (3.4)36. User Interface appearance customizability (3.4)37. Usability Compliance (3.5)38. Response time (4.1)39. Throughput time (4.1)40. Turnaround time (4.1)41. Efficiency Compliance (4.3)42. Activity recording (5.1)43. Readiness of diagnostic function (5.1)44. Change recordability (5.2)45. Change impact (5.3)46. Modification impact localization (5.3)47. Completeness of built-in test (5.4)48. Autonomy of testability (5.4)49. Test progress observability (5.4)50. Maintainability Compliance (5.5)51. Adaptability of data structures (6.1)52. Organizational Environment adaptability (6.1)53. Hardware Environmental Adaptability (H/W,network) (6.1)54. System software Environmental adaptability (OS,concurrent application) (6.1)55. Porting User Friendliness (6.1)56. Continued use of Data (6.3)57. Functional inclusiveness (6.3)58. Available co-existence (6.4)59. Portability Compliance (6.5)3. Failure avoidance (2.2)4. Incorrect operation avoidance (2.2)