Advanced Building Simulation

an experiment involving structured elicitation of expert judgment, so they were
unacquainted with the motions and underlying concepts of such an experiment.
Moreover, acting as an expert entails the assessment of subjective quantile values and
subjective probabilities, a task the experts are not familiar with. Extensive psychological
research (Kahneman et al. 1982; Cooke 1991) has revealed that untrained
assessors of subjective probabilities often display severe systematic errors or biases in
their assessments.
Hence, a concise training program for the experts was developed (de Wit 1997a),
which the experts had to complete before they gave their assessments in the elicitation
session.
ELICITATION
In this stage, the core of the experiment, the experts make their judgments available
to the analyst. Individual meetings with each expert were arranged. Moreover, the
experts were specifically asked not to discuss the experiment among each other. In
this way, the diversity of viewpoints would be minimally suppressed.
The elicitation took place in three parts. Prior to the elicitation meeting, each
expert prepared his assessments, for example, by looking up relevant literature and
making calculations. During the meeting, these assessments were discussed with the
analyst, who avoided giving any comments regarding content, but merely pursued
clarity, consistency and probabilistic soundness in the expert’s reasoning. On the basis
of the discussion, the expert revised and completed his assessments if necessary.
Completion of the elicitation coincided with the writing of the rationale, a concise
report documenting the reasoning underlying the assessments of the expert. During
the writing of this rationale, which was done by the analyst to limit the time expenditure
of the expert to a minimum, issues that had not been identified in the meeting
were discussed with the expert by correspondence.
COMBINATION OF THE EXPERTS’ ASSESSMENTS
Uncertainty in building simulation 45
To obtain a single distribution for the decision-maker, DM for all pressure coefficients,
the experts’ assessments must be combined. This involves two steps:
1 Construction of a (marginal) probability distribution from the three elicited
quantile values for each variable and each expert.
2 Combination of the resulting experts’ distributions for each variable.
Step 1: Construction of probability distributions. For each variable, three values
were elicited from the experts. These values correspond to the 5%, 50%, and 95%
quantiles of their subjective probability distribution. Many probability distributions
can be constructed, which satisfy these quantiles. The selection of a suitable probability
distribution is a technical issue, which is well-covered in Cooke (1991), but falls
outside the scope of this chapter.
Step 2: Combination of the experts’ distributions. For each coefficient, a weighted
average of the experts’ distributions was calculated for use in the uncertainty analysis.
The experts’ weights were based on their performance, which was obtained from