Master Thesis - Fachbereich Informatik

5.1. EXPERIMENTAL DESIGN 99
between light efficiency and motion blur effects. This shutter time requires a small Fnumber
of 1.4 to yield sufficient bright images.
In all experiments it is assumed that the system is calibrated correctly, the radial distortioncoefficientsareknownandateach-instephasbeenperformedtolearnfpix2mm.
In
addition, the perspective correction function has been determined before each experiment
to compensate for perspective distortions.
5.1.2. Evaluation Criteria
There are several criteria that can be used to compare and evaluate the results of different
experiments. These can be classified into quantitative and qualitative criteria.
Quantitative Criteria
Total Detection Ratio The system must exactly detect the number of tubes that pass
the visual field of the camera. Formally, this can be expressed in the following score Ωtotal:
Ωtotal = Ndetected
(5.1)
Ntotal
where Ndetected indicates the number of detected tubes and Ntotal the total number
of tubes respectively. Ωtotal = 1 is a necessary but not sufficient criterion for a correct
working inspection system.
Per Tube Measurements The average number of single measurements for each tube
depends mainly on the velocity of the conveyor and the camera frame rate. If N tubes
have been measured, the mean number of per tube measurements can be computed as:
ΩPTM = 1
N
where mi isthenumberofsinglemeasurementsoftheith tube.
N�
i=1
mi
(5.2)
False Positives/ False Negatives Each tube T can be classified into one of the three
groups G0 (good ),G− (too short), and G+ (too long) ifmeasuredmanually. G0 is defined
by the target length and the allowed tolerance for this length. It contains all tubes that
meet the tolerance in the real world. G− and G+ include all tubes of a real world length
that lie below the lower or above the upper tolerance threshold respectively.
In the same way, each tube can be categorized into one of the three groups G ′ 0 , G′ −,or
G ′ + based on the measured length by the visual inspection system. In the ideal case, this
three groups are equal to the corresponding ground truth classifications, i.e. G ′ 0 = G0,
G ′ − = G−, andG ′ + = G+ 1 .
In practice, however, the measurements are biased by many factors like perspective
errors, curved tubes, skew tube edges, noise, motion blur, or failures in measuring point
detection. In addition, as will be introduced in Section 5.1.3, the manually acquired ground
1 Theoretically, a fourth group U for unsure can be defined including all tubes that could not be detected
at all. These tubes have to be handled by different mechanisms as will be discussed in later sections