Software Reliability Analysis and Evaluation - LAAS CNRS

Folder 3
Software Reliability
Analysis and Evaluation
Karama KANOUN
LAAS-CNRS
(karama.kanoun@laas.fr)
3 - 1
OUTLINE
☞ Motivations
☞ Methods for software reliability analysis and evaluation (Folder 3)
• Data collection and analysis
☞ Data collection and validation
☞ Descriptive statistics
☞ Trend analysis
• Software reliability evaluation (Folder 4)
☞ Case studies (Folder 5)
☞ Dependability benchmarking of operating systems (Folder 6)
3 - 2
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Why Software Reliability?
☞ Increasing role of software in real life systems
☞ System dependability is more and more synonymous of software
reliability
☞ Difficulties in mastering the software development process and in
reducing design faults for complex systems
☞ Increasing cost of system non-dependability
☞ Real needs for improving software reliability to improve system
dependability and reduce maintenance cost
☞ Dependability requirements are part of system requirements (as
important as functional requirements)
☞ Quantification is essential
3 - 3
Objectives of software reliability analysis and evaluation
☞ Manage and improve the reliability of software products
☞ Check the efficiency of development activities
☞ Estimate the “effort” needed to reach a high dependability level
☞ Evaluate the software reliability at the end of validation activities and in
operation
☞ Estimate the maintenance effort to “correct” faults activated during
development and residual faults in operation
☞ Needs for experimental & analytical methods and
techniques to reach these objectives
3 - 4
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Software vs hardware reliability
Hardware
• Physical faults
• Operational life
• Stable reliability (constant failure rate)
• White-box approach
• Markov models
• Database for components failures
Software
• Only design faults
• Development and operation
• Reliability growth (↓ failure rate)
• Usually black-box approach
• Specific models
• Based on data collection
Failure Rate
Hardware in operation
Software in operation
Time
3 - 5
Measures
Static Measures
of the product and
process
(quality oriented)
Usage profile
(environment)
Dynamic Measures
characterizing occurrence of
failures
(reliability oriented)
Number of faults
Fault density
Complexity measures
…
Failure intensity
Failure rate
MTTF
Reliability
…
3 - 6
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

☞ Supplier point of view
• During development:
☞development follow up
(failure intensity, fault density)
☞evaluation of software reliability before operation
(MTTF, pre-operational failure rate)
• During operation
☞product reliability follow up
(residual failure rate, MTTF)
☞maintenance planning
(cumulative number of failures)
☞ Users / customers, operational life
☞be confident in the reliability level of the product
(residual failure rate, MTTF)
3 - 7
Percentage of faults and corresponding MTTF (published by IBM)
Product
MTTF (years)
5000 1580 500 158 50 15.8 5 1.58
⇐
1
2
3
4
5
6
7
8
9
34,2
34,3
33,7
34,2
34,2
32,0
34,0
31,9
31,2
28,8
28,0
28,5
28,5
28,5
28,2
28,5
27,1
27,6
17,8
18,2
18,0
18,7
18,4
20,1
18,5
18,4
20,4
10,3
9,7
8,7
11,9
9,4
11,5
9,9
11,1
12,8
5,0
4,5
6,5
4,4
4,4
5,0
4,5
6,5
5,6
2,1
3,2
2,8
2,0
2,9
2,1
2,7
2,7
1,9
1,2
1,5
1,4
0,3
1,4
0,8
1,4
1,4
0,5
0,7
0,7
0,4
0,1
0,7
0,3
0,6
1,1
0,0
⇐
3 - 8
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Overview of a global reliability analysis method
Development
Validation &
Operation
1
Data related to
previous projects,
similar products
data collection
Objectives
of the study
Types
of faults
3
Collected data
2
Data Validation
Validated
data
Data set partition
Consequences Phase Components
of failures
• • •
4 Descriptive Analyses 5 Trend Analyses
6 Model Application
Descriptive Statistics
Reliability Evolution
Reliability Measures
Feedback to the development process
3 - 9
☞ Some rules
Setting up of a data collection process
• Define clearly the objectives and the data to be collected
• Motivate and imply people that will be involved
• Simplify the collection process and reduce the number of data items to
be collected
☞ Support tools
☞ Practical organization of people involved
• Record and analyze data in real-time
• Feedback
☞ Origin of collected data
• Internal: recorded during development and validation
• External: by the customers
3 - 10
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Data to be collected
☞ Background information
• Product itself: software size, language, functions, current version, workload
• Usage environment: verification and validation methods, tools, etc.
☞ Data relative to failures and corrections
• Date of occurrence, nature of failures, consequences
• Type of faults, fault location
☞ Usually, recorded through
• Failure Reports (FR)
• Correction Reports (CR)
☞ Well defined headings, well structured, easy to fill in
☞ Short tick-off questions
☞ Manually or automatically
3 - 11
☞ Failure Report (FR)
Required Information
• Serial number (for identification)
• Report editor
• Product reference, version affected (or prototype)
• Date and time of failure occurrence
Desirable Information
• Failure occurrence condition
• Failure criticality or consequences
• Affected function or task
• Action proposed (if any)
3 - 12
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

☞ Correction Report (CR)
Required information
• Serial number (for identification)
• Report editor
• Date of correction
• Correction nature
• Product reference
Desirable Information
• Identification of the modified components
☞ Integration with already existing data collection programs
☞ Importance of training
3 - 13
Data Validation
☞ Objectives
• check the validity and usability of the information recorded
• Keep only genuine software faults in the database
☞ Elimination of:
• Duplicated data (FR reporting of the same failure)
• FR proposing a correction related to an already existing FR (COR)
• False FR (signaling a false or non identified problem)
• FR proposing an improvement (IMPROVE)
• incomplete FRs or FRs containing inconsistent data (Unusable)
• FR related to a hardware failure
• …
3 - 14
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Example 1: a telecommunications equipment
(analyzed at LAAS)
☞ 2 146 Failure Reports
☞ Validation ⇒ 1 172 kept in the database
Duplicated FRs 816 38.0%
COR 53 2.5%
False FR 29 1.4%
IMPROVE 21 1.0%
Unusable 20 0.9%
Hardware 35 1.6%
Total 974 45.4%
.
3 - 15
☞ 3063 FRs
Example 2: a telephone switching system
(analyzed at LAAS)
☞ Validation ⇒ 1853 Software FRs:
Software 1853
Hardware 195
Documentation 165
Unusable, duplicated, … 716
Others 134
Unusable
24%
Others
4%
Documentation
5%
Hardware
6%
Software
61%
3 - 16
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Life cycle of an FR
Internal
site
Identification
of an
abnormal
behavior
External
sites
Interface
with users
Database
yes
FR
exists ?
No
creation
of an FR
Analysis &
validation
specialized
team
Already solved
or being solved
Proposition
correction ?
No
Report update
Implementation
of the corrections
FR resolved
Creation of a CR
Database
3 - 17
DESCRIPTIVE STATISTICS
☞ Aim: make syntheses of the observed phenomena
☞ Simple analyses
• Fault typology
• Fault density of components
• Failure / fault distribution among software components (new, modified, reused)
☞ Investigation of relationships
• Fault density / size
• Fault density / complexity
• Fault density / life cycle phase
• Nature of faults / life cycle phases
• Nature of faults / components
• Number of components affected by changes made to resolve an FR
☞ Analyses related to the development / debugging process
3 - 18
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Analyses related to the development process
☞ Factors affecting time to locate and solve problems
• The more FRs circulating, the more time it takes to handle each one
• Tendency to resolve the easier FRs first, the remaining ones take more time
• Loss of maintainability with continued changes to resolve faults
• Introduction of new faults while resolving the old
☞ Average time to resolve an FR
Modification request time =
Time when the FR is resolved - time when it is created
Measures
• Responsiveness of the field support system
• Complexity of maintenance
3 - 19
Case of the switching system of Example 2
# FRs recorded:
3063
# FRs with record
date: 3049
# FRs with record
date and resolve
date : 2446
250
200
150
100
50
# FRs recorded / month
# FRs resolved / month
0
1 10 20 30 40 50 60 68
7 / 12 months
22% (524)
>12 months
12% (307)
cumulative # of unresolved FRs
800
700
600
500
400
300
200
100
0
1 10 20 30 40 50 60 68
0 / 6 months
66% (1615)
Time to resolve an FR
3 - 20
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Data pre-processing for reliability analysis
☞ Two kinds of data sets can be extracted from FRs and CRs
• Time to failures (or between failures)
0
t 1
t 2
t k
failure
t k
= time between failure k-1 and k
• Grouped data
☞ Number of failures per unit of time, n(k)
☞ Cumulative number of failures N(k)
0
1 2
k
n(1)
n(k)
3 - 21
Time ?
☞ Time between failures
• Execution time
• Wall clock or Calendar time
• Number of executions
☞ Number of failures per unit of time
• The length of the unit time depends on:
☞ accuracy expected for the dependability measures
☞ number of observed failures
☞ objectives of the study
3 - 22
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Example A
Times between failures
☞ Real-time control system (Musa 1)
• 136 failures observed during system test (96 days)
# T i Dy # T i Dy
# T i Dy
# T i Dy
# T i Dy
# T i
Dy
# : number of
failures
T i : times
between
failures
(in seconds)
Dy: day of
observation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
3
30
113
81
115
9
2
91
112
15
138
50
77
24
108
88
670
120
26
114
325
55
242
68
1
2
9
10
11
11
17
20
20
20
20
20
20
20
20
20
30
30
30
30
30
30
31
31
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
422
180
10
1146
600
15
36
4
0
8
227
65
476
58
457
300
97
263
452
255
197
193
6
79
31
32
32
33
34
42
42
46
46
46
46
46
46
46
47
47
47
47
53
53
54
54
54
54
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
816
1351
148
21
233
134
357
193
236
31
369
748
0
232
330
365
1222
543
10
16
529
379
44
129
56
56
56
57
57
57
57
59
59
59
59
59
59
59
59
61
62
63
63
63
64
64
64
64
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
810
290
300
529
281
160
828
1011
445
296
1755
1064
1783
860
983
707
33
868
724
2323
2930
1461
843
12
64
64
64
65
65
65
66
66
66
66
67
67
68
68
68
69
69
69
69
70
71
72
72
72
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
261
1800
865
1435
30
143
108
0
3110
1247
943
700
875
245
729
4897
447
386
446
122
990
948
1082
22
72
73
73
74
74
74
74
74
75
76
76
76
77
77
77
78
79
79
79
79
79
80
80
80
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
75
482
5509
100
10
1071
371
790
6150
3321
1045
648
5485
1160
1864
4116
80
80
81
81
81
83
83
83
83
83
84
84
87
87
88
92
3 - 23
☞ Switching system
Example B
Number of failures per unit of time or Cumulative
• 52 failures in operation (15 months)
i n(i) NC(i) NS(i) i n(i) NC(i) NS(i) i n(i)
NC(i)
NS(i)
i: unit of time (week)
n(i): number of failures
per unit of time
NC(i): cumulative
number of failures
NS(i): number of systems
in operation at i
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
2
0
2
1
1
0
2
1
2
5
2
1
2
0
0
0
0
1
1
2
1
0
4
2
2
4
5
6
6
8
9
11
16
18
19
21
21
21
21
21
22
22
24
25
25
29
4
10
10
10
10
12
12
12
12
12
13
13
13
13
21
21
21
21
21
28
28
28
28
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
1
1
0
0
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
2
0
1
30
31
31
31
32
32
32
32
32
32
32
32
33
33
33
33
33
33
33
34
36
36
37
36
36
36
36
38
40
40
40
40
42
42
42
42
42
42
42
42
42
42
42
42
42
42
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
0
0
0
1
0
1
1
0
0
1
1
6
0
0
0
1
0
0
0
1
0
37
37
37
38
38
39
40
40
40
41
42
48
48
48
48
49
49
49
49
50
50
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
42
3 - 24
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Trend analysis
☞ Objectives:
• Analyze software reliability evolution
• Identify periods of reliability growth and decrease
corrections
Failure intensity
corrections
. . .
V i,1
corrections
+ spec./environment
changes
time
3 - 25
V i+1,4
V i+1,3
V i+1,2
V i,k
V i+1,1
V i,2
Reliability growth characterization
☞ Variable: time between failures
• T 1
, T 2
, … ,T n
: time between failure i and i-1
☞ Reliability growth: T i
≤ T k
∀ i < k
st
☞ Prob. {T i
< x } ≥ Prob. {T k
≤ x} i.e. F
Ti (x) ≥ F Tk (x) ∀ i < k ∀ x
☞ Variable: number of failures
• N(t 1
), N(t 2
), … , N(t n
) : cumulative number of failures between 0 and t i
• H(t i
) = E[N(t i
)] = expectation of N(t i
)
• If N(t i
) is a Non Homogeneous Poisson Process (NHPP):
☞ reliability growth if H(t 1
) + H(t 2
) ≥ H(t 1
+ t 2
) ∀ t 1 , t 2 ≥0 and 0≤ t 1 + t 2 ≤T
and inequality is strict for at least a pair t 1
, t 2
N(t) is a subadditive function
☞ reliability decrease if H(t 1
) + H(t 2
) ≤ H(t 1
+ t 2
) ∀ t 1 , t 2 ≥ 0 and 0≤ t 1 + t 2 ≤T
and inequality is strict for at least a pair t 1
, t 2
N(t) is a superadditive function
3 - 26
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Interpretation of Subadditivity & Superadditivity
☞ Subadditivity
H(t 1
) + H(t 2
) ≥ H(t 1
+ t 2
) ∀ t 1
, t 2
≥0 and 0≤ t 1
+ t 2
≤T
(The number of events in an interval of the form [0, t 2
] is larger than the
number of events taking place in an interval of the same length
beginning later (i.e. in the form of [T, T+t 2
])
☞ The number of failures is decreasing
☞ Superadditivity
H(t 1
) + H(t 2
) ≤ H(t 1
+ t 2
) ∀ t 1
, t 2
≥0 et 0≤ t 1
+ t 2
≤T
(The number of events in an interval of the form [0, t 2
] is smaller than
the number of events taking place in an interval of the same length
beginning later (i.e. in the form of [T, T+t 2
])
☞ The number of failures is increasing
3 - 27
Trend tests
☞ Means
• Raw data ⇒ graphical tests
• Analytical tests ⇒ quantitative indicators
☞ Raw data
• Times to successive failures
• Number of failures per unit of time
• Cumulative number of failures
☞ Trend indicators
• Empirical (arithmetical) means
• Subadditivity factor
• Laplace factor
3 - 28
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Graphical tests: times to failures (Example A)
Times to failures
7000
6000
5000
4000
3000
2000
1000
0
Failure #
Cumulative times
to failures
t 1 + …t k
100000
90000
80000
70000
60000
50000
40000
30000
20000
10000
0
Failure #
3 - 29
Graphical test: grouped data (Example B)
Failure
intensity
6
5
4
3
2
1
0
10
9
8
7
6
5
4
3
2
1
0
unit of time = one week unit of time = 4 weeks
Cumulative
number of
failures
60
50
40
30
20
10
0
t i
60
50
40
30
20
10
0
unit of time = one week unit of time = 4 weeks
3 - 30
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Empirical mean
Global trend
ξ k
: arithmetical mean of the times to failures (from failure 1 to k)
t
ξ k
= 1 + t 2 + … t k
k
ξ k
constitute a globally increasing series € reliability growth
ξ k constitute a globally decreasing series € reliability decrease
The trend is directly observed on
700
600
500
400
ξ k
Example A
300
200
100
0
k
3 - 31
Empirical mean
☞ Local trend
• The data items are grouped into subsets containing m successive data
• The average is evaluated for each subset
• The impact of old data items is eliminated
the evolution of ξ k
☞ Example A: m = 8 ⇒ 17 groups (136 failures)
3000
2500
2000
1500
1000
500
0
3 - 32
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Subadditivity factor
☞ Graphical interpretation of subadditivity
• H(t) = E[N(t)] is subadditive over [0,T] if :
t
a H
(t) = ∫ H(x) dx
t
- H(t) ≥ 0 ∀ t ≥0 and 0≤ t ≤T
0 2
a H
(t) =subadditivity factor
H(x)
a H
(x)
Chord
x
0
T
3 - 33
Subadditivity & monotonous growth / decrease
Monotonous growth:
a H
(x) > 0 increasing
H(x)
Monotonous decrease:
H(x)
a H
(x) < 0 decreasing
Cumulative number
of failures
x
x
t
T
t
T
Failure intensity
h(x)
h(x)
x
x
3 - 34
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Subadditivity & local trend fluctuations
Example: reliability growth with local fluctuations
H(x)
a H (x) ≥ 0 non decreasing
0
x
h(x)
0
x
3 - 35
Subadditivity & trend change
Decrease - Growth
H(x)
tangent
Growth - Decrease
H(x)
0
T L
x
T T
0
G
T L
T G
T
x
a H
(x)
a H
(x)
0
T L
T G
T
x
0
T L
T G
T
x
/ /
a H
(x) < 0 a H
(x) > 0 a H
(x) > 0 a H
(x) < 0
3 - 36
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Laplace factor
☞ Statistical Test of hypothesis ⇒ Laplace factor u
Random variable: times to failures T i
(realization of T i
= t i
)
u(T) =
N(T) i
1
Σ
N(T) i=1
Σ tj - T
j=1 2
T 1
√ 12 N(T)
N(T) = # failures in [0,T]
Random variable: # failures per unit of time
u(T) =
☞ In practice
k
Σ (i-1) n(i) -
i=1
√
k -1
k
Σ n(i)
i=1
u > 0 ⇒ global reliability decrease
u < 0 ⇒ global reliability growth
2
k 2 k
-1
Σ n(i)
12 i=1
n(i) = # failure during time unit i
3 - 37
Interpretation of the Laplace test
☞ Random Variable: time to failure
T
• = mid of the observation interval
2
N(T) i
1
• c =
Σ Σ tj
N(T) i=1 j=1
: statistical centre
T
2
> c ⇒ reliability growth
☞ Random Variable: # failures per unit of time
• The Laplace factor can be put in the form:
u(T) = -
T
√
a H
(T)
1
12 N(T)
Relationship between Laplace and subadditivity factors
3 - 38
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Link : graphical tests - Laplace - Subadditivity
10
8
6
4
2
Failure intensity
250
200
150
100
50
Subadditivity factor
0
0
50
Cumulative number of failures
50
40
30
20
10
0
0
2
4
6
8
10
12
14
16
18
20
3
2
1
0
-1
-2
-3
-4
-5
-6
Laplace factor
3 - 39
Laplace factor: local and global trend
u(k)
G
L
O
B
A
L
Reliability
decrease
Local trend changes
k
T
R
E
N
D
Reliability
growth
Reliability
decrease
T L1
T G
A B C D
Reliability growth
T L2
Reliability
decrease
LOCAL TREND
3 - 40
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Change of time origin
u(k)
k
T L1 T G
T L2
A B C D
u(k)
k
T L1
T L2
B - C
D
3 - 41
Link between trend indicators
Cumulative number
of failures
Failure intensity
Laplace factor
Monotonous
Growth
N(k)
k
n(k)
k
u(k)
k
N(k)
n(k)
u(k)
Monotonous
decrease
k
k
k
Decrease
followed
by growth
N(k)
n(k)
u(k)
k
T L
k
T L
k
T L
T G
Stability
N(k)
n(k)
u(k)
k
k
k
3 - 42
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

How to use trend test results
☞ Control of the efficiency of test activities
• Reliability decrease at the beginning of a new activity: OK
• Reliability decrease during a relatively long period of time: Pb ?
• Reliability growth after reliability decrease: OK
• Sudden reliability growth: caution!
• Stable reliability: saturation
☞ New tests
☞ Following phase
☞ End of test
☞ Application of reliability models
• Trend in accordance with model assumptions
3 - 43
Application to RADC data sets
Rome Air Development Center (USA)
System
Id.
#
instructions
#
programmers
#
failures
Type of
system
1 21 700
2 27 700
3 23 400
4 33 500
5 2 445 000
6 5 700
7 (14C) *****
8(17) 61 900
9(27) 126 100
10(40) 180 000
11A(SS1A) *****
11B (SS1B) *****
11C (SS1C) *****
12 (SS2) *****
13 (SS3) *****
14 (SS4) *****
9
5
3
6
7
275
8
110
8
8
8
unknown
unknown
unknown
unknown
unknown
136
54
38
54
831
73
36
38
41
101
112
375
277
192
278
196
RT.C
RT.C
RT.C
RT.C
RT.Com.
RT.C
military
military
military
OS
OS
OS
TS
WP
OS
RT: Real-time
C: control
Com. : commercial
WP: word Processing
TS : Time sharing
OS: Operating system
***** : not given
[John Musa], “Software Reliability Data”, Rome Air Development Center, NY, USA, 1979
3 - 44
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

Laplace factors
* : stable reliability
System
Laplace
factor
1 - 9,10
2 - 5,73
3 - 6,13
4 - 8,59
6 - 3,64
7 - 2,14*
8 - 4,65
9 - 5,16
10 - 9,60
11A - 1,36*
11B - 0,73*
11C - 5,15
12 + 0,74*
13 - 5,64
14 - 1,78*
3 - 45
System 2: times to failures
Time to failures, ti
10000
9000
8000
7000
6000
5000
4000
3000
2000
1000
# failures
0
1 6 11 16 21 26 31 36 41 46 51
3 - 46
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

☞ Variable: time to failure
System 2: Laplace factor
u(i)
# failures
0
-1 1 6 11 16 21 26 31 36 41 46 51
-2
-3
-4
-5
-6
u(i)
4
3
2
1
# failures
0
31 34 37 40 43 46 49 52
-1
-2
3 - 47
System 2: failure intensity
n(k)
12
10
8
6
4
2
k
0
1 2 3 4 5 6 7 8 9 10 11 12 1314 15 16 17 18 19 20 21 22
k: unit of tims = 5000 seconds of execution time
3 - 48
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

System 2: Laplace factor
☞ Variable: # failures
u(k)
failure # 31
failure # 41
k = unit of time
0 1 2 3 4 5 6 7
-1
8 9 10 111213 14 1516 17 1819 20 2122
-2
-3
-4
-5
-6
2 u(k)
1
k = unit of time
0
7 8 9 1011 1213 14 1516 17 1819 20 2122
-1
-2
Unit of time = 5000 seconds of execution time
3 - 49
System 4: arithmetical mean
1000
ξ i
800
600
400
200
0
10 15 20 25 30 35 40 45 50
# failures
3 - 50
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

System 9
120000
100000
Arithmetical
mean
80000
60000
40000
20000
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41
# failures
Laplace
Factor
u(i)
2
1
0
1
-1
3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41
-2
-3
-4
-5
-6
# failures
3 - 51
System 11A
200000
180000
Arithmetical
mean
160000
140000
120000
100000
11 21 31 41 51 61 71 81 91 101 111
# failures
3
Laplace
Factor
2
1
0
-1
111
# failures
11 21 31 41 51 61 71 81 91 101
3 - 52
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS

System 14
Arithmetical
mean
300000
250000
200000
150000
100000
1 21 41 61 81 101 121 141 161 181
# failures
Laplace
factor
u(i)
2
1,5
1
0,5
0
-0,5
1 21 41 61 81 101 121 141 161 181
-1
-1,5
-2
# failures
3 - 53
Conclusion
☞ Some systems can be modeled by an exponential distribution
• System for which -2 < u < 2
☞ Influence of the operational profile
• Systems 11 A, B, C are 3 copies of the same program
used in different environments
☞ Benefits from trend analysis
• Better understanding of the underlying processes
• Follow up of the development process in real-time, fast feedback
• Helpful for reliability model application
3 - 54
Software Reliability Evaluation - Karama KANOUN - LAAS-CNRS