The mechanical assembly dimensional measurements with the ...

10666 J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675specific software for image acquisition and dimensional measurementsof the mechanical assembly and established measuringaccuracy, repeatability and suitability of the system.2. ProtectorFig. 1. Basic structure of the protector.than AVI systems. In many cases the human measurement resultsare subjective and depend on workers’s experience. Before the AVIsystem can replace the human worker it must be tested for accuracyand measurement errors. Measurement errors can occur becauseof the object translations regarding to the video camera,inappropriate optics, badly captured images and quantization.The object translations towards the camera can result in ablurred image. Because the image sensor (e.g. CCD) is very small,a small translation of the object can result in a huge image blurring.This can be prevented with object stabilization during imagecapturing, as in our system. In case of system vibrations (Ho, 1983),the vibration isolators or the stroboscopic lightening can be used.The errors that are a consequence of inappropriate optics can bemainly found as a zone error (spherical aberration). In this effect,the light rays close and parallel to the optical axis are intersectedtoo far away from the lens. Contrary, if the rays run through theedge of the lens, they are intersected closer. In lens systems, the effectcan be minimized using special combinations of convex andconcave lenses, as well as using aspheric lenses. Beside this effect,astigmatism must also be mentioned. In astigmatism the set ofrays, that emerge from one point out of optical axis, are copied intothe set with elliptical shape. Also this effect can be minimized bythe use of special lens systems. Literature (Yi-Chin et al., 2006) alsorefers to the color or chromatic error aberration caused by dispersion,because the lens for purple light has a shorter focal lengththan for the red color. The system of lenses also minimizes thiserror.The image capturing means that the outer continuous world isdigitalized and this process causes the quantization error. This isdirectly related to the image sensor resolution. Higher resolutionmeans lower error, but higher image processing time. For this reasona compromise must be found to satisfy both criteria.Using the thresholds to separate the objects from the backgroundis extremely important, meaning that the image qualityis high and the edges well expressed. For this reason it is necessaryto choose the correct lighting. Literature states that a proper lightingdesign in machine vision is a complex process and depends onmany factors. The rules can not be given, only the guidance (Tarabanis,Allen, & Tsai, 1995; Murase & Nayar, 1994). When the lightingis planned, the constructor should consider the use of varioustypes of illumination (monochromatic, polarized, focused, diffuse)and various properties of observed objects: diffusion, reflectivity,translucency and transparency. In addition, it is possible to lightthe object in many ways ( Miller & Friedman, 2003) and each hasits advantages and disadvantages. Also here the universal rule doesnot exist: from the front, rear, with the polarized or non-polarizedlight, direct light, with structured and stroboscopic light.In the work described in this article, we have exploited the poolof published knowledge, utilized low price hardware, developedThe basic components of the protector (Fig. 1) are electrical contactsfor connecting wires, electrical switch, bimetal with the setscrew, limit and toggle element. In protector, the bimetal bendsand over the set screw exerts force on the limit element. Whenthe limit element is pressed enough, it triggers the toggle element,which represents half of the electrical switch.The AVI system was developed for a particular protector type. Ithas nominal switch-off temperature at 400 °C with tolerances of±30 °C. In order to achieve switching within the prescribed temperatures,the most important is the distance between the toggle elementand the limit element, marked with the letter A (Fig. 7). Fineadjustment of this distance is performed with the configurationmachines, where first the set screw is set in a way that the protectorswitches-off at room temperature. Switching event is checkedthrough electrical contacts. Since it is practically impossible andeconomically completely unjustified to adjust the set screw atthe rated temperature (400 °C), the set screw is spined at roomtemperature for a certain angle in reverse and fixated with glue.This new distance A should be an appropriate distance for protectorto switch-off at the rated temperature.The described procedure is carried out on all manufactured protectors.Verification of correctness of the screw-set distance is donestatistically in the production in special ovens, where the switchofftemperatures are verified at high temperatures. These measuredswitch-off temperatures are used to set the proper numberof reverse revolutions of the set screw during production. An identicaloven is also used in laboratory and is described later.The control, whether the protector system is correctly assembled,is now visually performed by the worker, which causes a possibilityof subjective evaluation. Therefore, there may be some falseassembled protectors compound in the cooking plate. As a consequence,there is a need for introducing the AVI system, whichwould eliminate the false protectors and possibly set the appropriatedistance A. In addition, the measurement of other dimensions,prescribed by tolerances, can be done. These measurement resultscan be statistically observed in a long time period.3. Automated visual inspection systemThe decision to develop a system for non-contact visual systemwas only acceptable, due to the structure and composition of individualprotector parts (Sato, Ishikawa, Hiraki, & Takamasu, 2002).In the class of non-contact measuring sensors are widely used triangulationlaser systems (Nguyen, 1995), that project laser light asdot, line or other curve. Test trials of measurement by laser triangulationline system Keyence LJ-G030 have shown, that the desireddimensions of components and the distances between them cannot be measured, due to too small distances between the componentsof the protector.Also in video systems many problems are found. From the hardware,video camera types, lens choice and particularly the correctlighting. In measuring systems it is also very important to havethe proper camera field of view, which is inversely related to themeasurement accuracy, that is exhaustively discussed by Hsua,Lina, and Lee (2005).The developed AVI system for measuring the dimensions anddistances between certain components of the protector is shownin Fig. 2. It consists of a monochromatic video camera, a lens withappropriate extension rings, an adequate lighting and a mechanical

J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675 10667Fig. 2. Prototype of AVI system with close-up showing lighting solution and protector bed.clamping system with a light shield and a protector positioningsystem.3.1. Video cameraWhen the video camera needs to be selected, the following factorshave to be accounted (Davies, 2005): what kind of image wewant to capture, line or matrix; do we need black and white or colorsensor; which image resolution is the most suitable for desiredproperties and how the object is illuminated.Our customer defined a measurement error within the range of±0.01 mm and field of view to be about 5 3mm (W H). Toachieve the desired measurement error the recommended resolutionof the measuring system is at least ten times lower, in our case1 lm. The customer required in the first stage a prototype system.For this reason a compromise in the initial financial investmentwas needed. Selected and installed monochrome camera was DMK41AU02 type, produced by The Imaging Source. It has a Sony CCD arrayimage sensor size 1/2’’ with a resolution of 1280 960 pixels. Inthis way we achieved a horizontal resolution of approximately 4 lmon the image element and vertically approximately 3 lm on the imageelement. The camera is connected to a PC via USB 2.0 connection.3.2. Mechanical structure with a light shieldThe base of the prototype system is a mechanical structure, asshown in Fig. 2. The figure presents the system without a lightshield, which is necessary to maintain constant lighting conditions.The framework structure is designed for a rigid attachment of thevideo camera. The camera attachment system allows positioning ofthe camera in all three axes of the Cartesian coordinate system.This is done by special screws. The system is designed in a way,that the camera is mounted on a special groove to set camera fieldof view perpendicularly to the protector bed. This is extremelyimportant for the image depth of field.In addition, the mechanical structure has also a function toproperly position the protector (2) in camera field of view, as canbe seen on Fig. 3. The user can roughly position the protector inthe prototype system. Then, the protector is pushed (1) with apneumatic cylinder to predetermined limit position (4), all this enablesa narrower camera field of view. The design of the guides (5)was very careful. It was necessary to lean the guides on the protectorswitch system to position protectors the same way according tothe camera field of view. This is a must due to large tolerances ofthe ceramic housing. In the direction where the camera’s field ofview is larger, a support (3) was made directly on the ceramic bodyof the protector, which has proved quite appropriate. This supportFig. 3. Protector positioning in AVI system.was made with a screw, the other two lean points are usingsprings.3.3. The lens and the extension ringsThe most important component of each video system is an adequatelens. Similarly to the camera, cheaper optics were chosen,from the company Fujinon, type HF50HA-1B, the C mount adapter.In order to minimize the distortion, the lens has a focal length of50 mm, the minimal working distance is 0.5 m. The lens allowsmanual focusing and shutter adjustment from F2.3 to F22.Since the selected lens has a large focal length, the depth of fieldwas lost, which decreases with the square of focal length (Eq. (1)).In this equation abbreviation WD stands for working distance, BS isa blur spot, I represents the iris, while F is the focal length. Duringthe later trials, the reduction of depth of field, has not been a problem.Otherwise, the lens with a smaller focal length should havebeen mounted. However the use of such lens would result in greaterimage distortion.WDDOF ¼: ð1Þ1 BS I WD FF 2The desired field of view on the protector was very small andthe lens minimum working distance was too large. These reasonsrequired installation of the extension rings, between the cameramount and the lens. With these rings, the lens was moved away

J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675 10669StartProtectorProtectorImageacquisition19B 14F42Imagesatisfactory?E110 D68117E352EA 13E3124EE5Fig. 7. Search areas and measured dimensions.6E A ¼ 1; x; x 2 ; ...; x n ; ð2ÞE7h i 1p ¼ A T A A T y; ð3Þ^y ¼ p 0 þ p 1 x þ p 2 x 2 þþp n x n :ð4Þ8101214EEEdimension AE9E11E13EThe same sequence is repeated in the case of the bimetal edge,marked with number 2. Searching procedure is in the extremeupper right area of the image in the rectangle whose position isalso predefined by the user. In this area another user predefinedrectangle is present and used for finding the upper edge of the limitelement, marked with the number 3. The next event mathematicallydescribes both, the edge of the bimetal and the upper edgeof the limit element (numbers 2 and 3). With this information adistance between these two lines is calculated, on the Fig. 7marked with a letter F and on the Fig. 6 with a step number 4.The calculation of the distance F is needed to recognize possibledeviations that suggest poorly configured protectors. In this caseno further analysis is required and the program terminates.All the required distances are calculated by defining the intersectionpoints between mathematically approximated lines andthen by Eq. (5) the distances between these points are calculated:EqffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffidðA; BÞ ¼ ðx 2 x 1 Þ 2 þ ðy 2 y 1 Þ 2 ; ð5ÞEndEFig. 6. Software flow diagram.checks if all left area in the square is black. If not, an error is returnedand the measurement procedure is stopped. Determinationof the left vertical edge proceeds by finding the edge on the binaryimage. Then the method of the least squares (Eqs. (2) and (3)) isused, through the stored coordinates of the edge points. The polynomialof 1st order that is equal to the linear line is approximatedthrough the binary edge points (Eq. (4)). This method is used for alledge approximations, both for linear and higher order polynomialapproximations. The procedure filters the edges, which is importantdue to small dust particles that can appear on edges. Whenusing mathematical equations also the distance calculations aremore accurate. In the case that error occurs during the mathematicaledge approximation, the program informs the user and procedureis terminated. On the diagram such an event is marked withthe letter E.In Fig. 6 number 5 shows a step in which the lower edge of thelimit element is found and mathematically approximated by thelinear line. This procedure is performed on the right side of the image,still within the specified rectangle. By setting this line, the limitelement thickness is calculated, marked with the letter E andnumber 7 in Fig. 7. The tests showed that measurements of dimensionE are not as accurate as we wanted, as a consequence of thelimit element material reflectivity. This reason can cause the thicknessof the limit element to be too small, up to 0.02 mm comparingto the manufactured dimension. The limit element material ismanufactured by the EN ISO 9445: 2006 (International StandardISO, 2006) standard, class P. This standard defines limit elementthickness of 0.200 ± 0.008 mm. Due to the standard’s very narrowtolerances, in the further calculations a value of 0.20 mm is used.In the next step, based on the defined edges, the left edge or ahook of the limit element is defined. In the utmost point of thehook the vertical line from step 1 is applied again. This line ismarked with the number 6. In the middle of these two parallellines, the perpendicular distance D is calculated, on Figs. 6 and 7marked with line number 10. In the design drawings for protector

10670 J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675this distance is defined as 1.6 ± 0.05 mm. It directly influences theforce of bimetal pushing the toggle element.Due to the force exerted on the limit element, it slightly bendsand linear approximation of the upper edge is not suitable. Fig. 7demonstrates that the upper edge of the limit element lies onthe boundary where dimensions A and B connect. Therefore thedefinition of the boundary point is not possible. That is why thebottom limit element edge is mathematically defined, by using2nd order polynomial, marked with the line number 8. As next, thismathematically approximated polynomial is shifted upwards forthe fixed dimension E. This line represents the upper edge of thelimit element.Tests uncovered that the setting of the bimetal edge, markedwith the line number 2, is in most cases satisfactory only whenthe field of search rectangle is positioned upper-right. Therefore,we were forced to define the bimetal edge also on the left side ofthe set-screw, which is represented as line number 9. This searchprocedure is relative to the vertical line that defines the left edgeof the limit element hook and carried out in preset width. By combiningdata of the bimetal edge from Section 2 and 9, the bimetalbottom edge is most precisely defined.The calculations of the dimensions A and B are carried out in thecontact point of the set screw and the limit element. Since thispoint depends on the position of the set screw, it is impossible todetermine this point precisely. Most reasonable is that the pointof contact between the set screw and the limit element is takenin the middle of the set screw, as is defined in section 11. The middlepoint definition procedure uses the area from nearby right sideof the limit element hook vertical line and left rectangle side forlimit element edge search, letter E or line number 7. At this stepthe polynomial approximation of the upper limit element edge isshifted upwards for 0.04 mm. The algorithm first finds the blackarea of the set screw, the middle region represents the middle ofthe set-screw. Using this procedure the middle line of the set screwcan not be defined highly precise, but the tests demonstrated thatthe approach gives satisfactory results.A huge challenge was to define the toggle element edge, in Fig. 7numbered with line 12. Finding that edge was not trivial due to theshadows that can occasionally appear in the left bottom area of theimage. These shadows can appear from the limit element hookblocking the light. For successful definition of the toggle elementedge, a special algorithm was created. The algorithm is based on settingthe dynamic threshold of the image between the two presetthresholds. These two limits are set by the user, usually between10 and 100, with the lower and upper limits being 0 and 255. The dynamicadjustment step is 5. This dynamic approach was chosen toselect optimal value, i.e. when the threshold is increased the shadowslowly appears. The tests showed that there is a small white area betweenthe shadow and the toggle element edge, which is gettingsmaller with higher threshold values. When the threshold is high enoughthe white area disappears. This threshold is too high, thus previousvalue is used. Next, the mathematical approximation of theedge is carried out. The search is started from the short vertical linenumber 12 to the extreme right edge of the image, from bottom up.The last black pixel represents the eventual edge point. In this areadirt, shadows and part of the limit element hook can occur. That iswhy a moving average algorithm over the found edge points is used.This means that over the edge points a virtual window shifts fromleft to right, the step size is a pixel value. If the edge point in a windowis too far away from the last local mean value, then this pointgets the mean value. This algorithm does not catch the influence offar away points as in the case of direct use of the least mean squaremethod. Over the pool of edge binary points a linear approximationis defined, describing the toggle element edge.The last approximation line enables the calculation of dimensionsA and B, inFigs. 6 in Fig. 7 marked as lines number 13 and14. The calculation is performed by setting a line through the middleof the set screw perpendicular to the toggle element edge line.The intersections of approximated lines represent the dimensionsA and B.When the analysis is finished, the user is informed about themeasurement results and the program window displays dimensionsA, B, D, E and F. The user is also informed whether the dimensionsare in the predefined boundaries or not and the data is savedin the database.4.3. Calibration of the AVI systemEvery measuring instrument has to be calibrated before the use.For this purpose, our software offers also the calibration of themeasuring AVI system. For calibration purposes the user uses threecalibration protectors, with known dimensions A, B and E, in ourcase previously measured with the certified profile projector measuringsystem, type Mitutoyo PV500.The user has to manually insert the pre-measured dimensionvalues in the empty software fields, before the first calibration.The calibration procedure is the same as the measuring procedure,described in previous section. The main difference in calibrationprocedure is that dimensions are not calculated, but rather thenumber of pixels between intersection lines is defined. The numberof pixels between these edges is a decimal value number.By using the decimal pixel values the transformation betweennumber of pixels to metric units is performed. Because the opticsnonlinearities, the relationship between the pixels is not completelylinear. For this reason the user can select which approximationpolynomial order best transforms the number of pixels intometric units. The polynomial order can be selected from 1 to 6. Thispolynomial transformation is calculated by using the residuals andthe factor R 2 , called the coefficient of determination. This coefficienthas a value between 0 and 1, where higher value reflectsbetter approximation. An example of the transformation approximationcan be seen on Fig. 8, where polynomial of 4th order isused.R 2 was calculated using Eq. (6), where y is the measured value, yis the mean and ^y is the current calculated polynomial value.ResidualSS ¼ X ðy ^yÞ 2 ;TotalSS ¼ X ðy yÞ 2 ; ð6ÞR 2 ¼ 1ResidualSSTotalSS ;Fig. 8. User window for polynomial transformation from number of pixels intometric units.

J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675 10671For calibration purposes three normal protectors were chosenwith dimensions A, B and E being the most spread over the AVI systemmeasuring range. At this point it must be stated that also duringthe calibration procedure, the dimension E was fixed at value0.20 mm. This value is the same as measured by using calibratedprofile projector measuring system.5. Methodology5.1. Dimensional measurements of the calibration protectorsFor the calibration of the presented AVI system three objects areneeded. Their known dimensions A, B and E are pre-measured withan optical profile projector. It was also used to measure anotherfive protectors used as control samples for defining the measurementerror.On these eight protectors dimensions A, B and E were measured.Measurements were performed by the use of the profile projector,type Mitutoyo PV500. The resolution of this system is 1 lm withthe repeatability of 3 lm. The magnifying factor during the measurementswas set to 20. For more accurate protector positioningon the measurement board also a special magnifying glass with amagnifying factor of 10 was applied. Measurement uncertaintyof the profile projector is described by the publication EA-4/02(European Co-operation for Accreditation, 1999) and Eq. (7). Theresult and the measured distance L is expressed in lm.u ¼ 4 þ 5 10 6 L ; ð7ÞCalibration of the profile projector measuring system is performedevery 2 years. For the calibration special calibrating blocksand glass rulers are used. The procedure for the calibration of theprofile projector measuring system is not defined in detail by anyinternational standard. Therefore the calibration is performed byinternal calibration procedures, which are accredited by an externaljudge. In our case, by the Dutch independent, internationallyrecognized testing and certification institute, called NMi (NederlandsMeetinstituut). Every calibration protector was measuredonly once at room temperature (23 ± 1)°C.The measurement procedure is as following. The calibrationprotector is laid on the glass surface of the profile projector. Theprofile projector measuring system, by the use of lenses and mirrors,projects the image of the measured object on the glass surfaceequipped with coordinate system and ruler marks. By rotating themicrometers the coordinate system is moved to the preferred positionand orientation. In our case the X axis was levelled to thetoggle element and the Y axis to the middle of the set screw. Afterlevelling, the coordinate system was positioned at the bottom ofthe limit element still in the middle of the set screw. This wasthe position where the micrometers values were reset to zero. Thiscenter point was chosen as the most exposed part of the protectorwhen measured by a profile projector. The position of this centerpoint is not appropriate for measuring distances A and B, becausethe distance A is too short and distance B too long for 0.20 mm,as is the thickness of limit element material.This measurement procedure has a great subjective influence ofthe person who performs the measurements, because many edgesare difficult to define with great precision, although the use ofmagnification helps to achieve more accurate measurements.From all 8 reference protectors, three were defined as calibrationobjects to define the transformation of the number of pixelsinto metric units, in our case in millimeters. To describe the transformationa polynomial transformation of the 4th order was usedalongside the linear transformation function. The 4th order polynomialhad the highest R 2 value. The calibration protectors with sequencenumbers 2, 6 and 7 had their measured values A, B and Ethe most spread over the field of view of the AVI system. Other protectors,numbers 1, 3, 4, 5 and 8, had been used as control objectsfor defining measurement accuracy and repeatability of the AVIsystem. The measurements with the AVI system were performedat room temperature of 23 °C. Every protector was manually positionedinto the system bed. After a measurement was run, everyprotector was manually removed from the system bed. Numberof repeated measurements on each protector was 25.5.2. Dimensional measurements of the protectorsTo ensure that the manufactured protectors switch-off at theright temperatures, it is necessary to statistically test protectors ina laboratory. 72 protectors randomly taken from already sealedpackages presents a measurement set. Several sets are tested daily.Each protectors set is put into a special oven. Every protector in theoven is electrically connected to observe the state of the switch. Theoven is gradually heated. The inner temperature is measured with aPT-100 class A temperature sensor. Its measurement error is expressedby Eq. (8). The calculation temperature (T x ) is at 400 °C.DT ¼0:15 ð þ 0:002 T x Þ ¼ 0:95 C: ð8ÞThe heating of the oven proceeds until the last protector reachesits switch-off temperature. For safety reasons the maximum innertemperature is set to 480 °C, where the heaters are also switchedoff. After heating, the oven is passively cooled to the temperaturewhere the last protector switches back on. The switch-off andswitch-on temperature value of each protector are saved into acomputer database. The procedure is repeated three times in thelaboratory. As a measurement result, the average value of the secondand the third cycle is used. This measurement procedure in alaboratory is very slow and every set takes 6–9 h to complete. Inproduction facilities this slow procedure is performed only once.This is the main reason why a faster measurement would be verywelcome. The fastest method would be if the switch-off temperatureis in correlation to the dimension A. The presented AVI system needsless then a second to assess the dimensions A, B, D and F. Includingthe protector manipulation, the overall time is less than 2 s, whichis much less than manufacturing time cycle. Fast measurementsusing such a system would result in testing of each manufacturedprotector, as opposed to today’s statistical sampling methods.Until today no comparative measurement on a large set of protectorswas performed, where the dimension A could be directlycompared to the switch-off temperature. For measurement purposesa set of 220 protectors was chosen. All were fine calibratedduring their production. Furthermore, a second set of 220 protectorswas also chosen. They were only roughly calibrated at the set screwassembling. These are meant as non-calibrated. The reason to includealso non-calibrated protectors, was to verify if only rough calibrationwas enough for production. All protectors were numberedand than measured by the presented AVI system and also with thereference heat and cool oven procedure. During measurements withthe AVI system each protector was measured only once. In the oventhe standard procedure was repeated. In the set of calibrated protectors218 of them were successfully measured by the AVI system and217 of the non-calibrated set. The reason why some of the protectorswere not successfully measured originates from mechanical problemsof the pneumatic positioning system of the AVI system. Someof the ceramic protector housings were out of tolerances.6. Results6.1. Dimensional measurements of the calibration protectorsThe measurement results of dimensions A, B and E obtainedwhen using a certified profile projector are given in Table 1. All

10672 J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675Table 1Reference dimensions A and B.Protector dimension 1 (mm) 2 (mm) 3 (mm) 4 (mm)A 1.87 1.56 1.80 1.57B 0.71 0.95 0.70 1.09Protector 5 6 7 8A 1.76 1.76 2.27 1.32B 0.87 0.71 0.35 1.29measurements are rounded to two decimal places, the third doesnot represent measurement accuracy. The dimension E is0.20 mm. The other two dimensions are widely spread over themeasuring range. The range of dimension A ranges from 1.32 to2.27 mm, resulting in 0.95 mm dispersion. The spread of thedimension B is 0.94 mm wide, from 0.35 to 1.29 mm. The sum ofboth dimensions is 2.55 mm, representing almost whole verticalrange of the camera field of view, which is approximately 3 mm.The results gathered by AVI system with the linear transformationbetween number of pixels and metric units are displayed inTable 2. The first column lists the number of the measured protector,the second shows measured dimension, the third containsmean measured distances, the forth the error between currentmean and reference value, which can represent the accuracy andthe last column the error between current mean value and measuredvalue. The last can be considered as measurementrepeatability.The errors measured in the calibration protectors (numbered 2,6 and 7) are in the range of ±0.02 mm, meaning around ±5 pixels. Ifthe repeatability is observed the values range from 0.00 to0.01 mm, meaning around 2 pixels.Looking at the other five protectors, the errors are in the samerange of ±0.02 mm as in calibration protectors. The repeatabilityvalue of these protectors is ±0.01 mm and as such completelysatisfactory.The results measured by AVI system with the 4th order polynomialinterpolation between number of pixels and metric units, aredisplayed in Table 3. This polynomial order demonstrated the highestR 2 value. As before, the first column shows the number of themeasured protector, the second lists dimension abbreviation, thethird brings mean measured distances, the forth the error betweenmean and reference value, which can represent the accuracy andthe last column the error between mean value and measured value.The last column can be considered as a measurement repeatability.Table 2Measurement results when using linear transformation.No. Dimension Mean value (mm) Error (mm) Repeatability (mm)Min Max2 A 1.54 0.02 0.00 0.00B 0.97 0.02 0.00 0.006 A 1.75 0.01 0.00 0.00B 0.71 0.00 0.00 0.017 A 2.29 0.02 0.00 0.01B 0.34 0.01 0.00 0.011 A 1.87 0.00 0.00 0.00B 0.71 0.00 0.00 0.013 A 1.81 0.01 0.01 0.00B 0.70 0.00 0.00 0.004 A 1.55 0.02 0.01 0.01B 1.09 0.00 0.00 0.005 A 1.77 0.01 0.02 0.00B 0.86 0.01 0.00 0.008 A 1.30 0.02 0.00 0.00B 1.29 0.00 0.01 0.00Table 3Measuring results when using 4th order polynomial transformation.No. Dimension Mean value (mm) Error (mm) Repeatability (mm)Min Max2 A 1.56 0.00 0.01 0.00B 0.96 0.01 0.01 0.006 A 1.77 0.01 0.01 0.00B 0.70 0.01 0.01 0.007 A 2.28 0.01 0.01 0.00B 0.34 0.01 0.01 0.001 A 1.90 0.03 0.01 0.00B 0.69 0.02 0.00 0.013 A 1.84 0.04 0.01 0.00B 0.68 0.00 0.00 0.004 A 1.57 0.00 0.01 0.01B 1.09 0.00 0.01 0.005 A 1.80 0.04 0.01 0.00B 0.84 0.03 0.00 0.018 A 1.30 0.02 0.00 0.01B 1.29 0.00 0.00 0.00The error measured in the calibration protectors (numbered 2, 6and 7) is in the range of ±0.01 mm, meaning around ±2 pixels. Therepeatability values range from 0.01 to 0.00 mm.When checking the other five protectors, the encountered errorsare much higher. In protectors 3 and 5 the error for dimensionA is 0.04 mm, in dimension B the error is smaller having a value of0.02 mm. The repeatability is in range from 0.02 to 0.01 mm.Fig. 9 shows errors between results measured by the AVI systemand the reference values measured with the profile projector. Thebars for dimensions A and B are displayed for 1st (linear) and 4thorder polynomials used to transform number of pixels to metricunits. The upper diagram shows comparison for dimension A andthe bottom for dimension B. The horizontal axis represents theprotector number and the vertical axis the error in millimeters.Again it can be seen that error is more equally distributed overthe measuring range in the case of linear transformation.6.2. Dimensional measurements of the protectorsIn Fig. 10, a histogram shows switch-off temperatures of 220calibrated protectors from the production, without using AVIsystem. The measurement tests were made in a laboratory oven.Error (mm)Error (mm)0.020−0.02−0.040.040.020−0.02Dimension A1 2 3 4 5 6 7 8Protector numberDimension B1 2 3 4 5 6 7 8Protector number1st order 4th orderFig. 9. The error according to the polynomial order; the profile projector values aretaken as standard.

J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675 10673No. of samples6050403020100340 360 380 400 420 440Switch−off temperature (°C)Fig. 10. Sample of 220 calibrated protectors; switch-off temperature.The horizontal axis shows switch-off temperatures in degreesCelsius, steps of 20 °C. Two vertical dashed lines represent thetolerance area of ±30 °C, according to predefined switch-offtemperature of 400 °C. The mean value of measured sample is398 °C and standard deviation 9.2 °C. These results show that themajority of protectors switch-off in the tolerance range. Only threeprotectors switched-off outside the tolerance area, with switch-offtemperatures: 352, 369 and 441 °C.The measurements of the switch-off temperatures of protectorscalibrated in production demonstrated that they are in most casesproperly calibrated. For this reason the mean value of the dimensionA can represent the mean switch-off temperature. The distributionof dimension A, measured with AVI system in observedsample is shown in Fig. 11. The mean value is 1.93 mm with standarddeviation 0.05 mm. Based on these values the tolerance rangewas set to embrace equal number of protectors as seen in the tolerancearea of switch-off temperatures. This way the set tolerancearea has a width of ±0.15 mm according to the mean value, on thefigure being represented by two vertical dashed lines.To replace the very slow measurements of the switch-off temperaturein the oven with the AVI system approach, the relationbetween dimension A and switch-off temperature should be linear(Fig. 12). The horizontal axis represents dimension A, the verticalaxis stands for switch-off temperature in degrees Celsius. Each circlerepresents one numbered protector, the dashed lines define toleranceareas. If the relation would be linear, then the circles wouldappear on the line between the intersection of dashed lines. In thediagram it is clearly seen that the relation is not linear and toospread around the line. Furthermore, one protector with dimensionA within the tolerance area switched-off at to high temperature.The oven measurement results of switch-off temperatures for220 non-calibrated protectors are shown in Fig. 13. The horizontalaxis shows switch-off temperatures in degrees Celsius, step is10 °C. Two vertical dashed lines represent previously used tolerancearea of ±30 °C, with desired switch-off temperature of400 °C. The mean value for the sample is much higher, 425 °C withstandard deviation 23.1 °C. According to these figures most protectorsswitch-off outside the tolerance range. 96 protectors had theswitch-off temperatures higher than the maximal temperature inthe oven that is 480 °C and are not included in Fig. 13.The measurement results of dimension A by using AVI systemfor non-calibrated protectors are shown as histogram on Fig. 14.Switch−off temperature (°C)4504404304204104003903803703603501.7 1.75 1.8 1.85 1.9 1.95 2 2.05 2.1 2.15Dimension A (mm)Fig. 12. Sample of 218 calibrated protectors: measured dimension A vs. switch-offtemperature.504540352520No. of samples302520No. of samples151015105501.7 1.75 1.8 1.85 1.9 1.95 2 2.05 2.1Dimension A (mm)0350 360 370 380 390 400 410 420 430 440 450 460 470 480 490Switch−off temperature (°C)Fig. 11. Sample of 218 calibrated protectors; measurements of dimension A.Fig. 13. Sample of 220 non-calibrated protectors; switch-off temperature.

10674 J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675No. of samples40353025201510501.8 1.9 2 2.1 2.2 2.3Dimension A (mm)Fig. 14. Sample of 217 non-calibrated protectors; measurements of dimension A.The mean value of distance A is 2.03, 0.1 mm more then in sampleof calibrated protectors. The standard deviation is higher as well,having a value of 0.08 mm.In Fig. 15 is shown a correlation between the dimension A andthe switch-off temperature of the toggled protectors. The horizontalaxis stands for dimension A and the vertical axis for switch-offtemperature in degrees Celsius. The dashed lines mark thetolerance area, horizontal axis defines switch-off temperaturetolerances and vertical current tolerances for dimension A. Themajority, if not all samples, are away from the line between thetwo tolerance points, although the tendency is more linear. Manyprotectors have the dimension A inside the tolerance area, howeverthe switch-off temperature is too high and vice versa.7. DiscussionSwitch−off temperature (°C)4604404204003803601.8 1.85 1.9 1.95 2 2.05 2.1 2.15 2.2 2.25Dimension A (mm)Fig. 15. Sample of 217 non-calibrated protectors: measured dimension A vs.switch-off temperature.For finding the edges by using the AVI system, we have used linearand square polynomial approximation. The superiority of ourapproach is based on the least mean square approximation method(Eqs. (2)–(4)) that filter the edges. This approach proved to be veryuseful in our investigation, due to dust particles that are frequentlycovering the edges. Also, the distances were calculated from mathematicalequations of the edges and not directly from number ofpixels, which increases the resolution into subpixel region.The measurements made on five reference protectors with acertified profile projector have proved that measurement accuracydepends on the polynomial order used to transform the number ofpixels into metric units. When the linear function is applied, thefactor R 2 shows the lowest error on all eight test protectors anddoes not exceed ±0.02 mm. With 4th order polynomial, the factorR 2 was maximal, while the error was larger from 0.04 to 0.02 mm.The detailed analysis of the group of three calibrated protectorsshowed larger error if linear pixel to metric unit transformation isused. The range of the error is ±0.02 mm. With 4th order transformationpolynomial, the results are much better, having an error of±0.01 mm. However, 4th order polynomial is less appropriate fordescribing other distances. For this reason, the linear transformationis selected to be used in the future, this way distributing theerror more equally across the whole measurement area.Small part of the measurement error can originate from the referenceprofile projector. Following this analysis it can be specifiedthat measurement uncertainty is within ±0.01 mm. The parts of theprotector having the edges little rounded and also the material thatis very shiny are limiting factors for even higher accuracy.The measurement procedure with the profile projector is identicalto the AVI system. The biggest difference is in the lighting conditions.Namely, the AVI system illuminates the background andthe profile projector illuminates the foreground, which is causinga difference in determining the edges. However, using the profileprojector is the best possible measurement solution, in fact, it isthe only possible. The contact distance measurement system isnot usable to acquire desired dimensions.Despite the measurement uncertainty (Eq. (7)) of the profileprojector the main source of the measurement error is the AVI systemitself or an edge detection algorithm. In some cases the shadowsemerge as a consequence of the limit element position. Theseshadows influence on the offset of the mathematically describededge for a few pixels, with the pixel having a dimension of approximately4 lm. The shadows are most frequently present in the areawhere the edge of the toggle element is determined and also in thearea where exact bimetal edge is approximated. These two edgesdetermine the most important distances A and B. Several measurementsshowed that the error originating from the shadows is in therange of ±0.02 mm.The measurement results can be to some order further improvedby using the camera with a larger resolution. Someimprovement could be also achieved with special and more expensivemacro lenses. These would have smaller blurring and higherdepth of field as it is presented in this article.The development originated from need that measured distance Awould be obtained contactless as a replacement for slow referencemeasurements of the switch-off temperatures. Comparative measurementson larger number of protectors did not confirm a directcorrelation between distance A and switch-off temperature on bothcalibrated and non-calibrated groups of protectors (Figs. 12 and 15).When observing the non-calibrated protectors quite linear correlationwas recognized. However, the characteristic seem to be steep.Further analysis and tests using other measurement methodsshowed that unloaded bimetal has linear bending in the temperaturerange up to 425 °C. In the protector the bimetal is loaded,meaning that linear characteristics end at approximately 300 °C.For this reason it is impossible to completely replace slow referencemeasuring procedure in the oven with dimensional measurementsof the protector. The bimetal has too much non-repeatableand non-linear characteristics.Today the presented AVI system is used in parallel with theslow reference temperature measurement procedure in thelaboratory. Before a set of protectors is placed in the oven, all

J. Rejc et al. / Expert Systems with Applications 38 (2011) 10665–10675 10675dimensions are measured with an AVI system. This is important insearching for other correlations, as well as to quickly find false calibratedprotectors. In the future, the AVI system use is planned forthe production line with the task of eliminating badly calibratedprotectors as well as a protector assembly control system.8. ConclusionWe have developed, calibrated and tested an AVI system fornon-contact dimensional measurements of the mechanical system,called the protector. The system consists of a standard monochromaticvideo camera, standard optics with extension rings and twostandard white light emitting diodes (LED). The software enablessystem calibration, where the user can flexibly transform a numberof pixels into metric units. The software also enables flexible measurements,with various tolerance ranges and saving all necessarymeasured data into a database.In comparison with the reference profile projector measurementsystem, the AVI system is much quicker, but also less accurate.The main benefit of presented measurement system is themeasurement speed, where all five dimensions are determined inless than a second. With the use of the profile projector measurementsystem the measurement of these dimensions takes severalminutes.The tests and measurements have shown that the developedmeasurement system can not be used as replacement of the currenttemperature measuring method, because the characteristicsof the bimetal are not linear and repeatable at higher temperatures,resulting in a wide gap between dimension A and theswitch-off temperature of the protector.The AVI system can be used as a control system in the productionfor elimination of protectors that have dimensions out of toleranceranges. Also, the option of saving all measured data can beused in the future for statistic analysis.AcknowledgmentThis work was financially supported by ETA Cerkno Company.ReferencesBatchelor, B. G., & Whelan, P. F. (1996). Machine vision systems: Proverbs,principles, prejudices & priorities. In Proceedings of the SPIE conference onapplied machine vision (Vol. 96, pp. 7–19).Chin, R. T., & Harlow, C. A. (1982). Automated visual inspection: A survey. IEEETransactions on Pattern Analysis and Machine Intelligence, 4(6), 557–573.Davies, E. R. (2005). Machine vision: Theory, algorithms, practicalities (3rd ed.). SanFrancisco: Elsevier.Dom, B. E., & Brecher, V. (1995). Recent advances in the automatic inspection ofintegrated circuits for pattern defects, applications. Springer-Verlag.Don Braggins, Industrial vision application sectors. .European Co-operation for Accreditation (1999). Expressions of the uncertainty ofmeasurements in calibration, EA-4/02.Fang, Y.-C., Liu, T.-K., Macdonald, J., Chou, J.-H., Wu, B.-W., Tsai, H.-L., et al. (2006).Optimizing chromatic aberration calibration using a novel genetic algorithm.Journal of Modern Optics, 53(10), 1411–1427.Griffin, P. M., & Rene Villalobs, J. (1992). Process of computer vision. PatternRecognition, 24(1), 69–93.Haralick, R. M., & Shapiro, L. H. (1991). Glossary of computer vision. PatternRecognition, 24(1), 69–93.Ho, C.-S. (1983). Precision of digital vision systems. IEEE Transactions on PatternAnalysis and Machine Intelligence, PAMI-5(6), 593–601.Hsua, H.-Y., Lina, G. C. I., & Lee, S.-H. (2005). Enhanced image-based coordinatemeasurement using a super-resolution method. Robotics and Computer-Integrated Manufacturing, 21, 579–588.International Standard ISO 9445 (2006). Continuously cold-rolled stainless steelnarrow strip, wide strip, plate/sheet and cut lengths. Tolerances on dimensions andform. Geneva: International Organization for Standardization.Miller, J. L., & Friedman, E. (2003). Photonics rules of thumb, eBook. McGraw-HillProfessional. p. 418.Murase, H., & Nayar, S. K. (1994). Illumination planning for object recognition usingparametric eigenspaces. IEEE Transactions on Pattern Analysis and MachineIntelligence, 16(12), 1219–1227.Newman, T. S., & Jain, A. K. (1995). A survey of automated visual inspection.Computer Vision and Image Understanding, 61(2), 231–262.Nguyen, H. G. (1995). A simple method for range finding via laser triangulation,Technical document 2734. San Diego, CA: Naval Command, Control and OceanSurveillance Center, RDT&E Division.Ravishankar Rao, A. (1996). Future directions in industrial machine vision: Casestudy of semiconductor manufacturing applications. Image and VisionComputing, 14, 3–19.Sato, O., Ishikawa, H., Hiraki, M., & Takamasu, K. (2002). The calibration of parallel-CMM: Parallel-coordinate measuring machine. In Proceeding of the 3rd Euspeninternational conference (pp. 573–576). The Netherlands: Eindhoven.Tarabanis, K. A., Allen, P. K., & Tsai, R. Y. (1995). A survey of sensor planning incomputer vision. IEEE Transactions on Robotics and Automation, 11(1), 86–104.