Security Effectiveness Framework Study

Security Effectiveness Framework study
Figure 6 reports the combined average response to the
six budget questions shown in Table 2 against average
SES results. The line graph compares all four quartile
results. It clearly shows that respondents’ perceptions
about the adequacy of the security budget and SES
results are systematically related. That is, respondents
who perceive their budget for security as adequate enjoy
a higher SES result than those who do not.
Figure 6: Quartile comparison of respondents’ perceptions
about the adequacy of the security budget and
security effectiveness
0.80
0.60
0.40
0.20
0.00
-0.20
67%
0.60
41%
0.29
30%
21%
-0.40
(0.26)
(0.33)
Quartile 1 Quartile 2 Quartile 3 Quartile 4
Favorable on budget combined
SES average
Security technology, controls and governance
The Security Effectiveness Framework consists of six
security effectiveness metrics that are output variables
used to assess organisations (see Figure 1). These are
uptime, compliance, threat containment, cost management,
breach prevention and policy enforcement. In this
framework, a high value for each metric indicates that the
organisation is efficient in the use of these resources.
Respondents were asked to rank six security effectiveness
metrics from 6 = most important to 1 = least
important. Consistent with Figure 7, policy enforcement,
compliance and threat containment are viewed as the
most important security measures. Cost management
and uptime are viewed as less important measures for
effectiveness (Figure 8).
Respondents were asked to rate enabling security technologies,
control activities and governance practices
deployed with their organisations. A five-point scale
was used to rate a list of proven security technology in
terms of their impact on each one of the six effectiveness
metrics, from having no impact to having a very significant
impact on effectiveness. The same procedure was used
for control activities. Figure 7 presents the overall average
results for enabling technologies, control activities and
governance practices.
The number within each bar reflects the aggregate
performance rating for the effectiveness metrics. The
ratings range from 1 as a minimum value to 5 as a
maximum value. The highest ratings were achieved for
policy enforcement, compliance and threat containment.
This suggests that respondents attach a lower level of
importance to cost management and uptime. In addition,
governance is viewed as more important than enabling
technologies for policy enforcement and compliance
metrics. In contrast, enabling technologies are viewed as
most important for threat containment and data breach
prevention.
Figure 7: Security metrics according to technology, controls
and governance
12.00
10.00
12.00
8.00
10.00
6.00
8.00
4.00
6.00
2.00
4.00
0.00
2.00
0.00
5.00
4.50
4.00
3.50 5.00
3.00 4.50
2.50 4.00
2.00 3.50
1.50 3.00
1.00 2.50
0.50 2.00
1.50 -
1.00
0.50
-
2.52 2.80
3.87 3.89
2.15
2.60
2.60 2.52 2.80
3.87 3.89
2.50
2.54 3.22 3.19
1.93 2.15
2.60
2.46 2.95 3.53 2.60 3.45
2.50
2.54 2.70 3.22 2.81 3.19
1.93
Cost 2.46 Uptime 2.95 Breach 3.53 Threat 3.45 Compliance 2.70 Policy 2.81
management
prevention containment
enforcement
Cost
management
1.55
Cost 1.55
management
Cost
management
Enabling Uptime technology Breach ControlsThreat
Governance Compliance
prevention containment
Enabling technology Controls Governance
2.11
Uptime 2.11
Uptime
Policy
enforcement
Figure 8: Average rank of six security effectiveness metrics
3.75 3.96 4.30 4.55
3.75 3.96 4.30 4.55
Breach
prevention
Breach
prevention
Threat
containment
Threat
containment
Compliance
Compliance
Policy
enforcement
Policy
enforcement
12
Sponsored by: HP Information Security and Check Point Software Technologies Ltd.
Ponemon Institute © Research Report & Instrument