Master Thesis - Fachbereich Informatik

5.2. TEST SCENARIOS 105
enough, the time until the next tube in the supply tube is gripped by the belt is sufficient to
produce a spacing. Experiments have shown that the supply tube works only for velocities
> 30m/min. Otherwise it is possible that two consecutive tubes are not separated.
Thus, one has two disjunctive methods to fit a conveyor with tubes. One is working well
for lower velocities, the other for faster ones. In both cases the maximum number of tubes
is limited. Therefore, larger experiments have to be partitioned over several sequences.
Test Data Since it is not worthwhile to manually measure thousands of tubes as ground
truth reference, the number of tubes that can be compared to such reference lengths is
limited. However, it is possible to increase the number of ground truth comparisons if one
repeats the automated visual measurement of a manually measured tube. For example, one
can manually measure 20 tubes of each particular type (that number can be placed onto
the conveyor or into the supply tube at one time) and repeat the automated inspection
several times. From the algorithmic perspective the system is confronted with a new
situation every time, independent if there are 100 different tubes to be inspected or 5×20.
In the following it is distinguished between tubes of a length that meet the given target
length within the allowed tolerance and tubes of manipulated length falling outside this
tolerance. The system must be able to separate the manipulated tubes from the proper
ones.
5.2. Test Scenarios
Eight test scenarios have been developed to evaluate the system. In each scenario only
one parameter is varied, while the others are kept constant. The different scenarios are
introduced in the following.
Noise Before the system is tested with respect to real data, the accuracy and precision
of the measuring approach is evaluated on synthetic images. A rectangle of known pixel
size simulates the projection of an ideal tube that is not deformed by perspective. The
‘tube edges’ as well as the measuring points are detected with subpixel precision like at
real images. The resulting length in pixels must equal the rectangle width. To evaluate
the accuracy under the presence of noise, Gaussian noise of different standard deviation
is added systematically to the sequences.
Minimum Tube Spacing In this scenario the minimum spacing between tubes is investigated
both for black and transparent tubes on real images. The test objects have a size
of about 50mm within the allowed tolerance and a diameter of 8mm. The velocity of the
belt is 30m/min. Starting at sequences that allow for only one tube in the visual field,
e.g. the spacing is larger than the tube length, the spacing is decreased until the detection
rate Ωtotal fallsbelow1,i.e.atleastonetubecouldnotbedetected.
Conveyor Velocity The goal in this scenario is to investigate how accuracy and precision
of the measurements depend on the velocity of the conveyor. The focus is on four different
velocities: slow (10m/min), medium (20m/min), fast (30m/min), and very fast (40m/min).
This is the maximum velocity that can be reached at production. Currently, the production
line runs at approximately 20m/min. To test the limits of the system, even higher velocities