An Automatic Warping Points Extraction Method for Calibration of ...

An Automatic Warping Points Extraction Method for
Calibration of Wide Angle Camera
Dae-Hyeon Kim, 1 Byung-Ik Kim, 1 Su-Young Ha, 1 Tae-Eun Shim, 2
Young-Choon Kim, 3 and Kyu-Ik Sohng 1
1 School of Electrical Engineering and Computer Science, Kyungpook National University, Korea
2 Division of IT Cooperative Systems, Gyeongbuk Provincial College, Korea
3 Dept. of Information and Communication Engineering, Youngdong University, Korea
Abstract - Radial or pincushion distortion problem of the
wide angle charge-coupled device (CCD) camera is an
important issue in the image calibration in case of close-up
photography of real scene. We designed the auto-extraction
method of the warping points in the calibration algorithm of
the image acquired by the wide angle CCD camera. Proposed
method extracts the distortion point by the wide angle CCD
camera, using the threshold value of the histogram of the
horizontal and vertical pixel lines. This processing step can be
directly applied to the original image of the wide angle CCD
camera output. Proposed method results are compared with
hand-worked result image using the two wide angle CCD
cameras having different angles (90° and 120°) with the
difference value of the result images respectively.
Experimental results show that proposed method can allocate
the distortion-calibration constant of the wide angle CCD
camera regardless of lens type, distortion shape and image
type.
When the picture is taken in occasion of camera with
radial distortion, straight line in picture becomes form of
curved line. The image pictured by the wide angle CCD
camera seriously has such distortions. The representative
algorithm of the image calibration is warping method.
But warping algorithm has the several faults. Method has
troublesome to find the points of the Fig. 4. (x i , y i ) and (x', y').
We designed the auto-extraction algorithm of the extraction of
the warping points in the calibration algorithm of the CCD
lens. Proposed method extract the warping points (x i , y i ) and
(x', y') briefly and exactly. And that is applied to camera
lenses with various angles. [3]
Keywords: image processing, image warping, auto
calibration, lens distortion.
1 Introduction
The image generation of various cameras – like film or
CCD cameras – are modeled with the pinhole camera
model. [1] However, the images of real cameras suffer from
more or less lens distortion, which is a nonlinear and
generally radial distortion. [1] The first is due to the fact that
many wide angle lenses have higher magnification in the
image center than at the periphery. This causes the image
edges to shrink around the center and form a shape of a radial.
The pincushion distortion is the inverse effect, when the
edges are magnified stronger. [2] The original image, radial
distortion image and pincushion distortion image are shown
as Fig. 1.
(a) (b) (c)
Fig. 1. (a) The original grid, (b) radial distorted grid image
and (c) pincushion distorted grid image.
The organization of the paper is as follows. Section 2
briefly introduces the basic warping algorithm for the image
calibration as the previous work. In section 3, the proposed
auto-extraction algorithm for the extraction of the warping
point is described and the image calibration with the
resolution preservation is explained. Section 4 gives the
experimental result using proposed algorithm and the analysis.
Conclusions are given in section 5.

2 A calibration method
The warping algorithm for improvement of image
distortion has been an essential algorithm in camera
calibration until a recent date. This algorithm is a basic
transformation that makes the spatial change of an image. The
conventional warping algorithm for camera calibration is
shown in Fig. 2. [4]
Distorted
image
Distorted
pattern image
Extract crosspoints
Calculate warping
parameter
Warping
Standard
pattern image
Calibrated
image
Fig. 2. The flow chart of the warping algorithm in camera
calibration.
At first, calculates the warping parameters (a i and b i )
with the distorted pattern image and standard pattern image.
Next, we can get the calibrated image by using the warping
parameter. The warping equations can be expressed by the
following Eq. (1), (2), and (3): [4],[5]
The (X, Y) and (U, V) are the respective distortion points and
standard points in Eq. (4):
X =
Y =
U =
V =
T
[ x1,
x2,
x3,
L xn
]
T
[ y1,
y2,
y3,
L yn
T
[ u1,
u2,
u3,
L un
]
[ v , v , v , L v ] T
1
2
3
n
] (4)
The warping parameters (a i and b i ) are calculated for the
calibration of distorted image. It finds the (x i , y i ) and (u i , v i )
points using the distorted and standard pattern images for the
estimation of the warping parameters. The (x i , y i ) and (u i , v i )
points are sample point of the distorted pattern image and
standard pattern image respectively. Eq. (1), the warping
parameters can be expressed by the following Eq. (5) for X:
⎡ x1
⎤ ⎡1
⎢ ⎥ ⎢
⎢
x2
⎥ = ⎢
1
⎢ M ⎥ ⎢M
⎢ ⎥ ⎢
⎣xn
⎦ ⎣1
u
u
M
u
1
2
n
v1
⎤
⎥⎡a0
⎤
v2⎥⎢
⎥
a1
M ⎥
⎥⎢⎣
a ⎥
2
v
⎦
n ⎦
And then Eq. (5) can be expressed by the following the Eq.
(6).
Where
(5)
X = PH
(6)
a
1-order warping equation:
X = a + a U + a V
Y = b + bU + b V
0
0
1
1
2
2
(1)
⎡1
⎢
⎢
1
P =
⎢M
⎢
⎣1
u1
v1
⎤
u
⎥
2
v2
⎥
M M ⎥
⎥
u n
v n ⎦
(7)
2-order warping equation:
X
Y =
2 2
= a0
+ a1U
+ a2V
+ a3UV
+ a4U
+ a5V
(2)
2 2
b0
+ bU
1
+ b2V
+ b3UV
+ b4U
+ b5V
3-order warping equation:
⎡a0
⎤
⎢ ⎥
(8)
H a
=
⎢
a1
⎥
⎢⎣
a ⎥
2 ⎦
The transformation matrix H a is expressed as Eq. (9):
H
a
T −1
T
= ( P P)
P X
(9)
X = a + a U + a V + a UV + a U
0
0
6
6
1
1
2
+ a U V + a UV
Y = b + bU + b V + b UV + b U
2
+ b U V + b UV
2
7
2
7
2
2
3
3
+ a U
8
8
+ b U
3
3
4
4
2
9
9
2
+ a V
+ b V
+ a V
3
3
5
5
+ b V
2
2
(3)
By the same way, the transformation matrix H b for Y is
expressed as Eq. (10):
T −1
T
H = ( P P)
P Y
(10)
b

However this algorithm has the several faults. That has
problem needing to find the upper two points (x i , y i ) and (u i ,
v i ). This means that it needs the pre-processing step finding
the several points directly in standard pattern image. And it
also requires high computation time. Therefore, we designed
the auto-extraction method of the warping points in the
warping algorithm.
3 Proposed method
In this paper, it extracts the distortion point by the wide
angle CCD camera, using the threshold value of the histogram
of the horizontal and vertical pixel lines. This processing step
can be directly applied to the original image of the wide angle
CCD camera output. The flow chart of the auto-extraction
method of warping points is as following Fig. 3.
It supposed that the lens and CCD is orthogonal. At first,
it eliminates image’s noise and detects the edge of the image
using the sobel filter in the acquired grid (pattern) image.
Then, it performs the histogram algorithm for the all pixel line
of the x-axis and y-axis. After the histogram process, it finds
the pixel position (center point) having the maximum value of
the histogram. This center point represents the green point (x 0 ,
y 0 ) in Fig. 4. Based on center point, the standard point is
generated by using the threshold value of the respective
histogram of the horizontal and vertical pixel line. The red
triangle in Fig. 4 represents the standard points. The guide
line is generated by using the vertical lines by the horizontal
Acquisition of grid (pattern) image
Noise elimination
Edge detection
Extraction of center point
Extraction of cross-point in
x-axis and y-axis center
Extraction of all cross-point
Fig. 3. The flow chart of the point extraction.
Fig. 4. The detection result of the cross-point processed by
auto-extraction method.
standard points and horizontal lines by the vertical standard
points. The intersection points of the guide line becomes (x i ',
y i '). Next step is drawing the extraction area (with 11 × 11
pixel size) based on the distance from the center point. The all
cross-points are obtained by using Eq. (11) and (12). That is,
the blue rectangular in Fig. 4 represents the extracted all
cross-points (x i , y i ). The displacement value for the all crosspoints
is calculated as the following Eq. (11) and (12):
Δ x = S x − x + S y − y
(11)
i
a
'
i
'
i
0
0
b
'
i
'
i
0
Δ y = S x − x + S y − y
(12)
i
c
d
The Δ xi
and Δ yi
are the pixel displacement value between
(x i ', y i ') and (x i , y i ). These values are increased in proportion to
the distance from the center point. The S a , S b , S c and S d
represent the distortion constant. In Eq. (11) and (12), the
value of Δ xi
and Δ yi
depend mainly on y i ' and x i '
respectively. Therefore these equations are simplified as
following the Eq. (13) and (14):
'
i
0
0
Δ x = S y − y
(13)
i
b
Δ y = S x − x
(14)
i
c
'
i
0
The S b and S c are the dominant distortion constant related to
the angle of the lens. The large angle of lens has the small S c
and small angle of that has the large S c .

4 Experimental results
Proposed method is applied to the gird (pattern) image
for the experiment. And the camera used in experiment is the
two cameras with 90 and 120 degree. Fig. 5 and 6 shows the
result image by proposed method. The extraction area and
extraction points mentioned in section 3 are showed in Fig. 5.
The green rectangular in Fig. 5 is acquired by Eq. (13)
and (14) according to pixel distance from the center point (x 0 ,
y 0 ). Finally, it extracts the extraction point by the intersection
point of the maximum histogram value of the all horizontal
and vertical directions in the green rectangular. We analyze S b
and S c , the distortion constant. S b and S c have 334 and 226
respectively in the camera with 90 degree. In case of the
camera with 120 degree, S b and S d have 334 and 221
respectively.
A Fig. 6 and 7 show the (a) original grid (pattern) image
(90° and 120°), (b) hand-worked result (90° and 120°), and
(c) image by proposed method (90° and 120°) respectively.
(a) (b) (c)
Fig. 6. Experimental result images: (a) original grid (pattern)
image (90°), (b) hand-worked image (90°), and (c) image by
proposed method (90°).
(a) (b) (c)
Fig. 7. Experimental result images: (a) original grid (pattern)
image (120°), (b) hand-worked image (120°), and (c) image
by proposed method (120°).
(a)
(a) (b) (c)
Fig. 8. Experimental result images: (a) real scene (90°), (b)
hand-worked image (90°), and (c) image by proposed method
(90°).
(a) (b) (c)
(b)
Fig. 5. Extracted areas and points by proposed method: (a)
extracted areas and points by 90° lens and (b) extracted areas
and points by 120° lens.
Fig. 9. Experimental result images: (a) real scene image
(120°), (b) hand-worked image (120°), and (c) image by
proposed method (120°).

The Fig. 8 and 9 respectively show (a) real scene image
(90° and 120°), (b) hand-worked result (90° and 120°), and
(c) image by proposed method (90° and 120°). For the
application of real scene acquired by wide angle CCD camera,
warping parameter (a i and b i ) calculated by using S b and S c
are applied to real scene. Fig. 8 and 9 show that result image
by proposed method is similar to hand-worked image by the
naked eye.
5 Conclusion
We designed the auto-extraction algorithm of the
extraction of the warping points in the calibration algorithm
of the CCD lens. Proposed method extracts the warping point
briefly and exactly. And that is applied to camera lenses with
various angles. And then it extracts exactly the distorted point
by the CCD lens. Experimental results show that proposed
method can allocate the distortion-calibration constant of the
CCD camera regardless of lens type, distortion shape and
image type.
6 References
[1] C. Slama, "Manual of photogrammetry," American
Society of Photogrammetry, Falls Church, VA, USA, 4th
edition, 1980.
[2] G. Vass and T. Perlaki, "Applying and removing lens
distortion in post production," Second Hungrian
Conference on Computer Graphics and Geometry, 2003.
[3] B. Prescott and G. F. McLean, "Line-based correction of
radial lens distortion," Graphical Models and Image
Processing, vol. 59, no. 1, pp. 39-47, Jan. 1997.
[4] G. T. Han, K. J. Lee, Y. K. Kim, and W. Y. Kim,
"Simplified auto calibration method for wide angle CCD
lens," The Collection of 1998 Autumn Conference's
Paper of Korea Information Processing Society, October
1999.
[5] I. N. Junejo, X. Cao and H. Foroosh, "Autoconfiguration
of a dynamic nonoverlapping camera network," IEEE
Transactions on Systems, vol. 37, no. 4, pp. 803-816,
Aug. 2007.