Evaluating A Selection of Tools for Extraction of Forensic Data: Disk ...

esulting acceptance spectrum can be divided into four levels, namely exceeds
requirements, target range, minimally acceptable and unacceptable, according to ISO
software evaluation standard 14598.1-2000. Tests will be executed and checked
against requirements defined in stage 1. Test results are collected and assessed against
the acceptance spectrum defined in stage 4. The complete evaluation result is released
to the community for review in stage 6.
The framework developed by Wilsdon & Slay (2006) is a refined approach of
NIST and SWDGE. Development of an organised reference set of different functions
of the tested software can make the testing in the proposed research more efficient.
Because multiple test subjects will frequently access different test cases for the same
function, unified reference sets will save time for retrieving the same data again and
again.
3.1.4 Applying Systematic Method For Commercial Off-the-shelf (COTS)
Selection
Kontio (1996) presented a case study applying a systematic method for reusable COTS
selection. The aim of the study is to prove that a more thorough definition of
evaluation criteria will result in a more effective and reliable evaluation process
(Kontio, 1996). A methodology called Off-The-Shelf Option (OTSO) has developed to
assist the process of search, evaluation and selection of COTS for decision makers.
The paper has placed the focus on how to define evaluation criteria and analyse data.
In addition, two data analysis methods, weighted scoring method (WSM) and Analytic
Hierarchy Process (AHP), were applied in the case study and compared and analysed
against various standards such as efficiency. The process of OTSO method is
illustrated in Figure 3.4, which demonstrates how the research was done. Evaluation
criteria definition is gathered from five different resources, requirement specification,
design specification, project plan, organisational characteristics and criteria feedback
from the process of software searching. Then from the selection criteria a set of formal
evaluation criteria is formed and these criteria will assist the software screening
process to narrow down the candidate software for in-depth evaluation.
50