Master Thesis - Fachbereich Informatik

5.1. EXPERIMENTAL DESIGN 103
Ground Truth Length [mm]
52.5
52
51.5
51
50.5
50
49.5
49
48.5
Intra Human Variance
48
0 2 4 6 8 10 12 14
Tube
(a)
Ground Truth Length [mm]
52.5
52
51.5
51
50.5
50
49.5
49
48.5
Inter Human Variance
48
0 2 4 6 8 10 12 14
Tube
Figure 5.2: Intra and inter human variance for the test set in Table 5.3 under ideal laboratory
conditions. The error bars indicate the maximum and minimum length for each of the 12 tubes
as well as the mean value of the measurements once for one person (a) and once for 10 persons
(b). The average inter human variance is slightly larger compared to the intra human variance.
Theerrorbarsindicatethemaximumandminimumlengthaswellasthemeanvalueof
all measurements. The computed mean standard deviation is 0.078mm.
In a second experiment, the inter human variance is determined. Therefore, 10 persons
have been asked to measure the same test set again as precise as possible. The inter human
variance is slightly larger than the intra human variance (see Figure 5.2(b)). In this case,
the mean standard deviation was observed to be 0.083mm.
Furthermore, it is important to state that the manual measurements for the ground
truth data have been acquired very carefully with elevated concentration under laboratory
conditions and with the aim to be as precise as possible using the digital measuring slide
(see Figure 5.1). Less than 5 tubes can be measured within one minute at this precision. At
production, the sample measurements are performed with a standard sliding caliper and
at a much higher rate. There is a definitively tradeoff between accuracy and speed. The
expected individual measuring error at production is much larger. Furthermore, factors
like tiredness or distraction can significantly increase the inter and intra human measuring
variance.
The accuracy and precision of the visual inspection system, however, should be evaluated
with respect to the maximum possible accuracy humans can reach with the given
measuring slide under ideal conditions. Throughout this thesis, manual ground truth measurements
always refer to the ideal, laboratory condition measurements. One has to keep
in mind that there is still a certain unsureness in these measurements. The real absolute
length of a tube can not be determined exactly.
For the following experiments, all tubes have been measured three times to reduce the
influence of the human variance. The mean of the three measurements is taken as ground
truth reference. All measurements are stored in a database and each measured tube is
labeled by hand with a four digit ID using a white touch-up pen.
(b)