04.08.2013 Views

A Learning Based Deformable Template Matching Method for ...

A Learning Based Deformable Template Matching Method for ...

A Learning Based Deformable Template Matching Method for ...

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

A <strong>Learning</strong> <strong>Based</strong> <strong>De<strong>for</strong>mable</strong> <strong>Template</strong> <strong>Matching</strong> <strong>Method</strong> <strong>for</strong> Automatic Rib<br />

Centerline Extraction and Labeling in CT Images<br />

Dijia Wu 1 , David Liu 1 , Zoltan Puskas 1 , Chao Lu 1 , Andreas Wimmer 2 , Christian Tietjen 2 , Grzegorz<br />

Soza 2 , and S. Kevin Zhou 1<br />

1 Siemens Corporate Research, Princeton NJ 08540, USA<br />

2 Siemens Healthcare, Siemensstr. 1, Forchheim 91301, Germany<br />

Abstract<br />

The automatic extraction and labeling of the rib centerlines<br />

is a useful yet challenging task in many clinical applications.<br />

In this paper, we propose a new approach integrating<br />

rib seed point detection and template matching to detect<br />

and identify each rib in chest CT scans. The bottom-up<br />

learning based detection exploits local image cues and topdown<br />

de<strong>for</strong>mable template matching imposes global shape<br />

constraints. To adapt to the shape de<strong>for</strong>mation of different<br />

rib cages whereas maintain high computational efficiency,<br />

we employ a Markov Random Field (MRF) based articulated<br />

rigid trans<strong>for</strong>mation method followed by Active Contour<br />

Model (ACM) de<strong>for</strong>mation. Compared with traditional<br />

methods that each rib is individually detected, traced and<br />

labeled, the new approach is not only much more robust<br />

due to prior shape constraints of the whole rib cage, but removes<br />

tedious post-processing such as rib pairing and ordering<br />

steps because each rib is automatically labeled during<br />

the template matching.<br />

For experimental validation, we create an annotated<br />

database of 112 challenging volumes with ribs of various<br />

sizes, shapes, and pathologies such as metastases and fractures.<br />

The proposed approach shows orders of magnitude<br />

higher detection and labeling accuracy than state-of-theart<br />

solutions and runs about 40 seconds <strong>for</strong> a complete rib<br />

cage on the average.<br />

1. Introduction<br />

To find rib metastases and fractures in chest CT scans,<br />

it typically involves reading hundreds of axial CT slices to<br />

visually track changes in rib cross-section area. However,<br />

manual reading is rather time consuming and rib anomalies<br />

are frequently missed in practice due to human error<br />

[17]. Automatic extraction of rib anatomical centerlines can<br />

1<br />

be used to enhance the visualization of unfolded rib cage,<br />

which will make routine bone reading tasks more efficient<br />

and effective <strong>for</strong> the radiologists [19]. The extracted and<br />

labeled rib centerlines can also serve as the reference to localize<br />

organs [25], register pathologies [22] and guide correspondence<br />

between serial thoracic CT scans <strong>for</strong> interval<br />

change analysis [13]. In addition, the derived rib geometry<br />

can assist with the rib cage fracture fixation surgeries [7].<br />

(a) (b)<br />

Figure 1. Example of different rib cross section appearances. (b)<br />

shows more clear dark rib marrow than (a).<br />

Despite its clinical importance, automatic detection and<br />

labeling of ribs in CT scans has not been extensively studied<br />

be<strong>for</strong>e and remains a challenging task. Most of the<br />

prior works model the ribs as elongated tubular structures<br />

and employ Hessian or structure tensor eigen-system analysis<br />

<strong>for</strong> ridge voxel detection [2, 21, 23]; however, these algorithms<br />

are usually computationally expensive and moreover,<br />

the rib marrow is typically darker than its boundary<br />

thus the rib center points are not exactly ridge voxels as<br />

shown in Fig.1(b). To construct the rib centerline, tracking<br />

based methods such as Kalman filter are usually used<br />

to trace detected rib center points from one slice to the next<br />

[19, 21, 9, 10]; however, some of them require manual ini-


tial seed points and more critically, these point to point<br />

tracking methods are highly sensitive to local ambiguities<br />

or discontinuities posed by rib pathologies like fractures as<br />

shown in Fig.2, which are nevertheless of the most interest<br />

to radiologists. For all of these algorithms, each rib is individually<br />

detected and traced, hence rib labeling requires a<br />

separate heuristic method [23, 18].<br />

In this paper, we propose a new rib centerline extraction<br />

approach that addresses all the above problems. First,<br />

a learning based rib center point detection method is introduced<br />

using computationally efficient Haar-like features<br />

which was proposed by Papageorgiou et al. <strong>for</strong> object detection<br />

[16]. To further speed up runtime detection, a coarseto-fine<br />

pyramid learning structure is used. Then the obtained<br />

probability response map is used to extract the rib<br />

centerlines via matching of a whole rib cage template. By<br />

extracting all rib centerlines simultaneously instead of tracing<br />

each of them individually, the proposed template matching<br />

not only imposes prior constraints between neighboring<br />

ribs and there<strong>for</strong>e improves the robustness significantly, but<br />

also provides rib labeling in the same time of the tracking.<br />

To adapt to the de<strong>for</strong>mation of the rib cage and maintain<br />

high computational efficiency, we break the long rib into<br />

short rib segments and carry out articulated rigid segment<br />

matching by searching <strong>for</strong> the optimal similarity trans<strong>for</strong>mation<br />

parameters in a manner similar to marginal space<br />

learning [28]. Compared with traditional point to point rib<br />

tracing algorithms, this new segment to segment matching<br />

approach is much more robust against local ambiguities or<br />

discontinuities. Finally, the piecewise rigidly matched rib<br />

centerlines are refined and smoothed with the active contour<br />

model [8].<br />

(a) (b)<br />

Figure 2. Example of different types of rib pathologies. (a) missing<br />

rib segments as a black gap along the rib (b) rib metastases where<br />

the rib appears much thicker than normal.<br />

Earlier works related to articulated registration within<br />

medical imaging were mainly applied to the problems of<br />

skeletal structures. Baiker et al. [3] proposed a hierarchical<br />

anatomical model of the mouse skeletal system with specified<br />

joint positions and realistic articulation degrees of freedom.<br />

The articulated registration is per<strong>for</strong>med by traversing<br />

the hierarchical anatomical tree in a top-down manner with<br />

iterative closest point algorithm on micro-CT data. In [1],<br />

du Bois d’Aische et al. employed an articulated registration<br />

<strong>for</strong> vertebral column by searching <strong>for</strong> the best parameters of<br />

the articulated trans<strong>for</strong>mation to maximize the mutual in<strong>for</strong>mation,<br />

constraining the vertebrae to move relative to their<br />

neighbors. More recently, Martín-Fernández et al. proposed<br />

a landmark based registration method <strong>for</strong> aligning hand radiographs<br />

[5]. This method models the bone skeleton with a<br />

wire model where wires are drawn by connecting landmarks<br />

located in the main joints of the skeletal structure.<br />

The remainder of the paper is organized as follows: In<br />

Section 2, we present in detail the new learning based de<strong>for</strong>mable<br />

template matching method <strong>for</strong> rib centerline extraction<br />

and labeling. Experimental results on 112 annotated<br />

volumes are given in Section 3 with discussion. Section<br />

4 offers with conclusions and outlines the future work.<br />

2. Rib Centerline Extraction and Labeling<br />

2.1. Rib Centerline Voxel Detection<br />

A<strong>for</strong>ementioned, the rib can not be simply modeled as<br />

a solid bright tubular structure due to its interior dark marrow,<br />

hence the maximum response solely obtained by Hessian<br />

eigen-system based filters does not always match the<br />

rib centerline voxels reliably. In addition, ribs across different<br />

volumes show a variety of size, shape and edge<br />

contrast as shown in Fig.1. There<strong>for</strong>e, we develop a robust<br />

learning-based object specific centerline detection algorithm,<br />

wherein the obtained probability map is used to<br />

track and label the rib centerlines. This supervised classification<br />

based two-level stratified learning system was successfully<br />

used in various computer vision problems such<br />

as semantic recognition [11], contextual analysis [26] and<br />

edge based segmentation [12].<br />

We manually annotated 12 pairs of rib centerlines from<br />

40 CT volumes <strong>for</strong> training. In this work, we select 3D<br />

Haar-like features due to their efficiency and effectiveness<br />

in object recognition [16]. A 3D Haar-like feature considers<br />

adjacent box regions at a specific location in a detection<br />

window and calculate the difference between the sum of the<br />

pixel intensities within the regions. These features can be<br />

rapidly computed using summed area tables first introduced<br />

in [6].<br />

The probabilistic boosting tree (PBT) classifier is trained<br />

with nodes composed of AdaBoost classifiers [24]. Instead<br />

of using only one PBT, we employ a coarse-to-fine pyramid<br />

of PBT classifiers which not only significantly speeds<br />

up the detection by image downsampling thus reducing the<br />

number of samples in earlier stage of the classifiers, but exploits<br />

longer range spatial context in lower resolution which<br />

was usually limited by Haar wavelets [14]. Finally given a<br />

volume, the learned classifiers will generate the probability<br />

response map P(x) that indicates the likelihood of each<br />

voxel being the rib centerline, as shown in Fig.3. The detec-


tion runs highly efficiently and it only takes about 2 seconds<br />

<strong>for</strong> a volume of size 200 × 200 × 200 voxels on the Intel R○<br />

2.27GHz Xeon R○ plat<strong>for</strong>m with 2.75GB RAM.<br />

Figure 3. Example of the rib centerline probability map. The left<br />

is the original volume and the right is the generated rib probability<br />

map with most of the other bones such as scapula and vertebrae<br />

significantly suppressed.<br />

2.2. Rib Cage <strong>De<strong>for</strong>mable</strong> <strong>Template</strong> <strong>Matching</strong><br />

The obtained probability map is not always reliable because<br />

of the distractions from neighboring similar bone<br />

structures such as clavicle and scapula, or local ambiguities<br />

and discontinuities caused by rib lesions. There<strong>for</strong>e, a<br />

robust rib tracking and labeling method is necessary. The<br />

traditional point to point tracing methods such as Kalman<br />

filter [19] or region growing process [23] that <strong>for</strong>ms line elements<br />

from ridge voxels are highly sensitive to these local<br />

rib anomalies and subject to error propagation.<br />

To address these problems, we propose a new template<br />

matching method <strong>for</strong> rib centerline extraction. The template<br />

is constructed by manually annotating and labeling 12 pairs<br />

of rib centerlines from one normal rib cage, and each rib<br />

Ri (1 ≤ i ≤ C, C = 24) is represented by a set of evenly<br />

sampled centerline voxels xn (1 ≤ n ≤ Ni) where Ni<br />

is proportional to the length of rib Ri. We intend to find<br />

the best trans<strong>for</strong>mation T that maximizes the sum of response<br />

of the trans<strong>for</strong>med template on the generated probability<br />

map:<br />

C Ni <br />

P (T (xn)) (1)<br />

i=1 n=1<br />

The use of a whole rib cage template makes the new<br />

method different from all previous rib detection methods in<br />

that all ribs are tracked or matched simultaneously. The advantages<br />

of this are twofold apparently: shape constraints<br />

can be imposed on neighboring ribs during tracking or<br />

matching to overcome the distractions from adjacent bone<br />

structures such as clavicle or adjacent ribs; all ribs are automatically<br />

labeled via matching.<br />

Due to considerable de<strong>for</strong>mation of the rib cage and<br />

spacing between adjacent ribs as shown in Fig.4, simple<br />

rigid trans<strong>for</strong>mation does not suffice. Traditional nonrigid<br />

(a) (b)<br />

Figure 4. Example of different rib cage shapes. The extracted rib<br />

centerlines are shown in green.<br />

trans<strong>for</strong>mation or registration approaches can be classified<br />

to the intensity based methods and feature based methods.<br />

The <strong>for</strong>mer methods compare intensity patterns in images or<br />

subimages such as the well known B-spline based free <strong>for</strong>m<br />

de<strong>for</strong>mations (FFD) [20] while the latter methods like the<br />

thin plate spline based robust point matching (TPS-RPM)<br />

[4], find correspondence between features such as points,<br />

lines or surfaces. Both methods <strong>for</strong>mulate the registration<br />

as an energy minimization problem which comprises<br />

a trans<strong>for</strong>mation smoothness term and a similarity measure<br />

such as normalized mutual in<strong>for</strong>mation between two images<br />

(FFD) or Euclidean distance between two sets of corresponding<br />

points (RPM). The objective functions are minimized<br />

via gradient descent and expectation-maximization<br />

(EM) like alternating optimization method respectively in<br />

FFD and RPM. However, these optimization methods are<br />

usually computationally expensive, furthermore, both energy<br />

functions are non-convex thus two methods are vulnerable<br />

to local minimum, especially in case of a large number<br />

of outliers and significant de<strong>for</strong>mation.<br />

In this work, we present a much simpler articulated template<br />

matching method by breaking each rib Ri into several<br />

short rib segments Ri,k (1 ≤ k ≤ K). Each rib segment<br />

centerline thus has much less degree of curvature and there<strong>for</strong>e<br />

can be approximately matched via rigid trans<strong>for</strong>mation<br />

by searching <strong>for</strong> the optimal similarity trans<strong>for</strong>m parameters<br />

Ti,k = (ti,k, oi,k, si,k), where (ti,k, oi,k, si,k) denote<br />

the translation, orientation and scale parameters, respectively<br />

as shown in Eq.(2):<br />

ˆTi,k = arg max<br />

Ti,k∈T<br />

Ni,k <br />

P (Ti,k(xn)) (2)<br />

n=1<br />

where T is the set of similarity trans<strong>for</strong>mations T . Instead<br />

of exhaustively searching the original nine-dimensional parameter<br />

space of (tx, ty, tz, ox, oy, oz, sx, sy sz), only<br />

low-dimensional marginal spaces are searched in a strategy<br />

similar to the marginal space learning (MSL) [28].<br />

To be specific, the trans<strong>for</strong>m estimation is split into three


steps: position estimation, position-orientation estimation<br />

and position-orientation-scale estimation. First, the positional<br />

marginal space is searched exhaustively and a small<br />

portion of the best position hypothesis is preserved. Second,<br />

the orientation marginal space is exhaustively searched<br />

<strong>for</strong> each position candidate and a limited number of best<br />

position-orientation candidates are kept after this step. Finally,<br />

the scale parameters are searched in the constrained<br />

space in a similar way. It is shown that this marginal space<br />

searching effectively reduces the number of testing hypothesis<br />

by six orders of magnitudes, compared with exhaustive<br />

full space search [28].<br />

In Eq.(2), each rib segment is searched individually and<br />

there<strong>for</strong>e likely matched to a wrong rib due to the similarity<br />

between adjacent ribs. To avoid this problem, we impose<br />

the pairwise smoothness constraints on the trans<strong>for</strong>m parameters<br />

of neighboring rib segments in a Markov random<br />

field (MRF) model:<br />

arg max<br />

(Ti,k,Tj,k)∈T<br />

i=1 n=1<br />

C<br />

Ni,k <br />

P (Ti,k(xn)) − λ <br />

i,j∈Nδ<br />

L(Ti,k, Tj,k)<br />

where λ is the regularization parameter, Nδ stands <strong>for</strong> the<br />

set of neighboring rib pairs (i, j) and L is the similarity<br />

function of two trans<strong>for</strong>m parameters defined as below:<br />

<br />

Ni,k<br />

L(Ti,k, Tjk) = ||Ti,k(xn) − Tj,k(xn)|| 2 /Ni,k<br />

n=1<br />

Nj,k <br />

+<br />

n=1<br />

(3)<br />

||Ti,k(xn) − Tj,k(xn)|| 2 /Nj,k (4)<br />

where L measures the average Euclidean distance of two<br />

neighboring rib segments Ri,k and Rj,k trans<strong>for</strong>med by different<br />

parameters Ti,k and Tj,k, respectively.<br />

The proposed articulated template matching method is<br />

illustrated in Fig.5. Each rib is split into four segments<br />

(K = 4) and starting from the segment (k = 1) connected<br />

to the central vertebrae, we search <strong>for</strong> the optimal<br />

rigid trans<strong>for</strong>mation parameters <strong>for</strong> short segments of all<br />

ribs Ri,k (1 ≤ i ≤ C) simultaneously by maximizing<br />

Eq.(3). The result is used to initialize the pose of the next<br />

rib segment (k = k + 1) and repeat the optimization until<br />

all rib segments are matched (k = K).<br />

The optimization of MRF energy function Eq.(3) is generally<br />

an NP-hard problem. To simplify the problem, we<br />

relax the neighborhood Nδ by only considering all vertical<br />

pairs of neighboring ribs from either side (e.g., the first and<br />

second rib on the left) and one horizontal pair of neighboring<br />

ribs from both sides (we choose the sixth ribs on the<br />

left and right side). The resulting H-shaped graph contains<br />

no loops of cliques and thus can be efficiently solved via<br />

dynamic programming. First, the trans<strong>for</strong>mation parameter<br />

Ti,k is searched separately that maximizes Eq.(2) in the<br />

manner of marginal space learning and a number of top candidates<br />

are kept <strong>for</strong> each rib segment Ri,k (1 ≤ i ≤ C),<br />

then the optimal combination of all rib segment trans<strong>for</strong>mations<br />

ˆ Ti,k can be found by adding smoothness constraints in<br />

Eq.(3).<br />

In practice, we find this approximation of imposing less<br />

neighborhood constraints already provides satisfactory results.<br />

As will be validated in Section 3, the new segment to<br />

segment articulated template matching approach is robust<br />

to local rib discontinuities and anomalies such as fractures<br />

or metastases.<br />

2.3. Active Contour Model Refinement<br />

Because each rib segment is trans<strong>for</strong>med rigidly, the articulated<br />

rib centerline is piecewise smooth and subject to<br />

small deviation due to slight de<strong>for</strong>mation of each rib segment<br />

as well as limited resolution of discrete trans<strong>for</strong>mation<br />

parameter search space. As a result, we employ the active<br />

contour model or snakes [8] to further refine the matching<br />

results:<br />

E =<br />

C Ni <br />

α<br />

2<br />

i=1 n=1<br />

|x′ n| 2 + β<br />

2 |x′′ n| 2 − P (xn) (5)<br />

where α and β control the elastic <strong>for</strong>ce and rigid <strong>for</strong>ce respectively,<br />

and xn stands <strong>for</strong> the rib centerline points. One<br />

of the notorious problems of applying snakes associates<br />

with the initialization. The starting contour must, in general,<br />

be close to the true boundary or else it will likely converge<br />

to the wrong result [27]. But this is not a problem in<br />

this system because the articulated matching in Section 2.2<br />

guarantees good initialization. The minimization of function<br />

Eq.(5) gives rise to solving corresponding Euler equations<br />

and the iterative numerical method in [27] is used. The<br />

result improvement with the active contour model is clearly<br />

shown in Fig.6.<br />

Figure 6. Rib centerline refinement using snakes.


Figure 5. Articulated rigid rib segment matching. Each rib is split into four segments and the registration is per<strong>for</strong>med on a total of twelve<br />

pairs of rib segments together starting from the segments connected to the central vertebrae.<br />

3. Experimental Results<br />

In the experiment, we collect 112 thoracic CT scans from<br />

different subjects and hospitals. These volumes show a significant<br />

variety of the size, shape and pathologies of the rib<br />

cages, with slice distances up to 5mm. All rib centerlines<br />

are fully annotated with a sampling distance of 0.5mm and<br />

labeled by two radiologists based on visual assessment and<br />

consensus review. 40 volumes are used to train the rib centerline<br />

detector and remaining 72 volumes are used <strong>for</strong> evaluation.<br />

With regard to the parameter setting, we select the<br />

smoothness regularization parameter λ = 10 in Eq.(3) and<br />

α = 50, β = 50 in snakes algorithm Eq.(5) with iterative<br />

step size γ = 1.0. The template is first shifted by aligning<br />

its centroid with the center of the mass of probability<br />

map P . Then we choose the translation search space as<br />

[−20mm, 20mm] <strong>for</strong> the first rib segments (k = 1) and<br />

[−4mm, 4mm] <strong>for</strong> the other rib segments (k > 1) with<br />

a sampling grid of 2mm in each direction; the orientation<br />

search space as [−π/3, π/3] <strong>for</strong> the first rib segments and<br />

[−π/6, π/6] <strong>for</strong> the other rib segments with a sampling grid<br />

of π/24; and [0.9, 1.1] with a grid of 0.05 <strong>for</strong> the scale<br />

search space. The obtained trans<strong>for</strong>mation parameters of<br />

previous rib segments Ri,k are used to initialize the next rib<br />

segments Ri,k+1, there<strong>for</strong>e the search space of the first rib<br />

segments is set larger than the following rib segments. In<br />

all experiments, each rib is evenly split into four segments.<br />

We first compare our approach with another fully automated<br />

rib centerline extraction method proposed recently<br />

[19]. It starts from a few automatically detected seed points,<br />

progressively traces along the rib centerlines slice to slice<br />

using a 3D Kalman filter, and employs the Random Walker<br />

algorithm to segment successive 2D rib cross sections and<br />

obtain the center voxels. Because each rib is traced individually,<br />

a separate rib pairing and ordering method is required<br />

to label the traced ribs [18].<br />

Table 1 lists the number of missed ribs, falsely detected<br />

ribs (either traced to clavicle or pelvis) and mislabeled ribs<br />

by two methods, with example extraction results shown in<br />

No. of Ribs Tracing <strong>Matching</strong><br />

Missed 138 (8.0%) /40 (55.6%) 0 / 0<br />

False 55 (3.2%) /19 (26.4%) 0 / 0<br />

Mislabeled 692 (40.0%) /46 (63.9%) 0 / 0<br />

Table 1. Result errors of two rib centerline extraction methods.<br />

Each entry gives the number (percentage) of incorrect ribs and the<br />

number (percentage) of volumes these incorrect ribs come from.<br />

Fig.7. It is not surprising that the new articulated template<br />

matching method is significantly more robust than the tracing<br />

method because the latter is highly sensitive to initially<br />

detected seed points; point to point tracing approach is more<br />

vulnerable to local ambiguities than the segment to segment<br />

matching; separate detection of individual rib centerline<br />

lacks prior constraints between neighboring ribs thus<br />

is more susceptible to labeling error; unsupervised random<br />

walker segmentation is more sensitive to the image noise<br />

or lower resolution compared with supervised learning with<br />

training images collected from a variety of resolutions and<br />

noise levels. In contrast, the new algorithm is highly robust<br />

against the de<strong>for</strong>mation of the rib cage and rib anomalies<br />

caused by lesions. Fig.10 shows some challenging cases<br />

where the proposed method still yields reasonably good results.<br />

To the best of our knowledge, pathological ribs were<br />

not discussed in previous literatures of automatic rib detection<br />

methods.<br />

In addition, we also compare the new MRF based articulated<br />

rigid matching method with the nonrigid robust point<br />

matching method (RPM) in [4]. We select the feature point<br />

based registration method <strong>for</strong> comparison, rather than intensity<br />

based ones, because the matching of a rib cage template<br />

is essentially the registration of a set of points in our problem.<br />

The target set of points is built by thresholding the<br />

derived probability map, wherein the outliers are generated<br />

inevitably. We use the RPM implementation developed at<br />

Yale University [15] in this experiment. As shown in Fig.8,<br />

the resulting rib centerlines are very likely attracted to adjacent<br />

ribs or trapped between two neighboring ribs due to<br />

the local minimum problem as mentioned above. The re-


Figure 7. Six example pairs of automatically extracted rib centerlines. The left is the random walker segmentation based tracing method;<br />

the right is the proposed learning based articulated template matching method.<br />

sults are not improved much by increasing the number of<br />

annealing iterations. For a volume of 200 × 200 × 200 voxels<br />

in size, it takes about 9 minutes on average <strong>for</strong> shorter<br />

annealing process (T0 = 4, Tfinal = 2, r = 0.9 where<br />

T0 and Tfinal is the initial and final temperature, r is the<br />

annealing rate), and nearly 25 minutes <strong>for</strong> longer annealing<br />

process (T0 = 20, Tfinal = 1 and r = 0.9). However,<br />

the proposed articulated rigid matching method only takes<br />

about 40 seconds <strong>for</strong> a volume of the same size.<br />

For quantitative comparison, we employ the modified<br />

Hausdorff distance between the extracted rib centerline<br />

points and ground truth as the measurement and results are<br />

given in Table 2. The proposed matching method outper<strong>for</strong>ms<br />

RPM by one order of magnitude. Notice that <strong>for</strong><br />

both methods, the top and bottom pair (especially the top)<br />

of ribs show larger errors compared with the middle ones,<br />

mostly because they are shorter and more curvy, with less<br />

constraints from adjacent ribs and more distractions from<br />

neighboring bones like the clavicles.<br />

RPM Articulated Rigid <strong>Matching</strong><br />

pair 1 16.7 4.1<br />

pair 2 12.4 1.5<br />

pair 3-10 11.6 1.2<br />

pair 11 11.2 1.6<br />

pair 12 14.5 3.6<br />

Table 2. The average modified Hausdorff distance (mm) between<br />

the ground truth and extracted rib centerlines by two matching<br />

methods.<br />

<br />

<br />

inf d(a, b) inf d(b, a)<br />

a∈A b∈B b∈B a∈A<br />

d(A, B) = max(<br />

,<br />

) (6)<br />

|A|<br />

|B|<br />

As mentioned in Section 1, one of the applications of<br />

the obtained rib centerlines is to enhance the visualization<br />

of the rib cage where the rib is digitally unfolded along the<br />

centerline. The unfolded two dimensional image makes it<br />

easier <strong>for</strong> the radiologists to find out the rib abnormalities


(a) (b)<br />

Figure 8. RPM registration results. (a) T0 = 4, Tfinal = 2 and<br />

r = 0.9; (b) T0 = 20, Tfinal = 1 and r = 0.9. The red points<br />

represent the registered rib cage template and the green points are<br />

obtained by probability map thresholding.<br />

such as fractures or varying severity of rib metastases. Two<br />

examples of rib unfolding images are shown in Fig.9 with<br />

multiple rib fractures.<br />

(a) (b)<br />

Figure 9. Two examples of rib unfolding images. (a) left rib cage;<br />

(b) right rib cage. In both cases, multiple rib fractures occur.<br />

4. Conclusion and Discussion<br />

In this paper, we presented a learning based de<strong>for</strong>mable<br />

template matching method to extract and label rib centerlines.<br />

Compared to previous rib detection and segmentation<br />

approaches, this method is highly robust because of multiple<br />

reasons. First, a supervised learning based detector<br />

is trained to locate the rib centerlines. It is more reliable<br />

with regard to the image noise or lower resolution given a<br />

reasonably large training dataset. Then, a template matching<br />

method is used to track all rib centerlines simultaneously<br />

with constraints between neighboring ribs imposed in<br />

a MRF model. This helps prevent individual rib tracked<br />

to adjacent ones and also makes the labeling much easier.<br />

Finally, we introduced an articulated rigid trans<strong>for</strong>mation<br />

method where the ribs are matched segment to segment. It<br />

is much more robust against local rib anomalies than traditional<br />

point to point rib tracing methods.<br />

Although this paper focuses on the problem of rib centerline<br />

extraction, the proposed articulate template matching<br />

method constrained by Markov random field in the trans<strong>for</strong>mation<br />

parameter space can be applied to other articulated<br />

structures such as the vertebrae. Ongoing work will include<br />

the extension of this algorithm to partial rib cage which not<br />

all twelve pairs of ribs are present in the volume, which requires<br />

a more flexible template. In addition, the precision<br />

of two rib endpoints can also be improved with some learning<br />

based algorithms. Currently, the search of the trans<strong>for</strong>m<br />

candidates takes most of the time in the whole system, and<br />

a coarse to fine pyramid may be used to speed up this component.<br />

References<br />

[1] U. Author. Estimation of the de<strong>for</strong>mations induced by articulated<br />

bodies: registration of the spinal column. Biomed.<br />

Signal Process. Control, 1(2):16–24, 2007.<br />

[2] S. R. Aylward. Initialization, noise, singularities, and scale in<br />

height ridge traversal <strong>for</strong> tubular object centerline extraction.<br />

IEEE Trans. Med. Imag., 21(2), 2002.<br />

[3] M. Baiker, J. Milles, A. M. Vossepoel, I. Que, E. L. Kaijzel,<br />

C. W. G. M. Lowik, J. H. C. Reiber, J. Dijkstra, and B. P. F.<br />

Lelieveldt. Fully automated whole body registration in mice<br />

using an articulated skeleton atlas. In ISBI, 2007.<br />

[4] H. Chui and A. Rangarajan. A new point maching algorithm<br />

<strong>for</strong> non-grid registration. Comput Vis Image Und, 89(2-<br />

3):114–141, 2003.<br />

[5] M. A. M. Fernandez, R. Cardenes, E. M. Moreno,<br />

R. de Luis Garcia, M. M. Fernandez, and C. A. Lopez. Automatic<br />

articulated registration of hand radiographs. Image<br />

and Vision Computing, 27:1207–1222, 2009.<br />

[6] C. Franklin. Summed-area tables <strong>for</strong> texture mapping. In<br />

SIGGRAPH, 1984.<br />

[7] I. Helzel, W. Long, D. Fitzpatrick, S. Madey, and M. Bottlang.<br />

Evaluation of intramedullary rib splints <strong>for</strong> lessinvasive<br />

stabilisation of rib fractures. Injury, 40(10):1104–<br />

1110, 2009.<br />

[8] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active<br />

contour model. IJCV, pages 321–331, 1988.<br />

[9] D. Kim, H. Kim, and H. S. Kang. An object-tracking segmentation<br />

method: vertebra and rib segmentation in CT images.<br />

In SPIE Medical Imaging, 2002.<br />

[10] A. P. Kiraly, S. Qing, and H. Shen. A novel visualization<br />

method <strong>for</strong> the ribs within chest volume data. In SPIE Medical<br />

Imaging, 2006.<br />

[11] L. Lu, K. Toyama, and G. D. Hager. A two level approach<br />

<strong>for</strong> scene recognition. In CVPR, 2005.<br />

[12] J. Ma, L. Lu, Y. Zhan, X. Zhou, M. Salganicoff, and<br />

A. Krishnan. Hierarchical segmentation and identification<br />

of thoracic vertebra using learning-based edge detection and<br />

coarse-to-fine de<strong>for</strong>mable model. In MICCAI, 2010.


(a) (b) (c) (d)<br />

Figure 10. Difficult cases: (a) missing rib segments (b) rib metastases (c) unusually large rib spacing (d) connected ribs. The top row shows<br />

3D volume rendering images whereas the bottom row shows 2D coronal planes.<br />

[13] M. Mohr, E. Abrams, C. Engel, W. B. Long, and M. Bottlang.<br />

Geometry of human ribs pertinent to orthopedic chest<br />

wall reconstruction. Journal of Biomechan, 40:1310–1317,<br />

2007.<br />

[14] A. Montillo, J. Shotton, J. Winn, J. E. Iglesias, D. Metaxas,<br />

and A. Criminisi. Entangled decision <strong>for</strong>ests and their application<br />

<strong>for</strong> semantic segmentation of CT images. In IPMI,<br />

2011.<br />

[15] X. Papademetris, M. Jackowski, N. Rajeevan, R. T. Constable,<br />

and L. H. Staib. Bioimage suite: An integrated medical<br />

image analysis suite, 2011.<br />

[16] C. P. Papageorgiou, M. Oren, and T. Poggio. A general<br />

framework <strong>for</strong> object detection. In ICCV, 1998.<br />

[17] M. Prokop and M. Galanski. Spiral and Multislice Computed<br />

Tomography of the Body. Thieme, 2003.<br />

[18] S. Ramakrishnan and C. Alvino. An automatic method <strong>for</strong><br />

rib ordering and pairing in 3D medical images. In ISBI, 2011.<br />

[19] S. Ramakrishnan, C. Alvino, L. Grady, and A. Kiraly. Automatic<br />

three-dimensional rib centerline extraction from CT<br />

scans <strong>for</strong> enhanced visualization and anatomical context. In<br />

SPIE Medical Imaging, 2011.<br />

[20] D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G.Hill, M. O.<br />

Leach, and D. J. Hawkes. Nongrid registration using free<strong>for</strong>m<br />

de<strong>for</strong>mations: application to breast MR images. IEEE<br />

Trans. Med. Imag., 1999.<br />

[21] H. Shen, L. Liang, M. Shao, and S. Qing. Tracing based<br />

segmentation <strong>for</strong> the labeling of individual rib structures in<br />

chest CT volume data. In MICCAI, 2004.<br />

[22] H. Shen and M. Shao. A thoracic cage coordinate system <strong>for</strong><br />

recording pathologies in lung CT volume data. In IEEE Nucl<br />

Sci Conf R, 2003.<br />

[23] J. Staal, B. van Ginneken, and M. A. Viergever. Automatic<br />

rib segmentation and labeling in computed tomography<br />

scans using a general framework <strong>for</strong> detection, recog-<br />

nition and segmentation of objects in volumetric data. Med<br />

Image Anal, 2007.<br />

[24] Z. Tu. Probabilistic boosting-tree: <strong>Learning</strong> discriminative<br />

models <strong>for</strong> classification, recognition, and clustering. In<br />

ICCV, 2005.<br />

[25] H. Wang, J. Bai, and Y. Zhang. A relative thoracic cage<br />

coordinate system <strong>for</strong> localizing the thoracic organs in chest<br />

CT volume data. In IEEE Eng Med Miol Soc., 2005.<br />

[26] D. Wu, L. Lu, J. Bi, Y. Shinagawa, K. L. Boyer, A. Krishnan,<br />

and M. Salganicoff. Stratified learning of local anatomical<br />

context <strong>for</strong> lung nodules in CT images. In CVPR, 2010.<br />

[27] C. Xu and J. L. Prince. Snakes, shapes, and gradient vector<br />

flow. IEEE Trans. Image Processing, 7(3), 1998.<br />

[28] Y. Zheng, A. Barbu, B. Georgescu, M. Scheuering, and<br />

D. Comaniciu. Four-chamber heart modeling and automatic<br />

segmentation <strong>for</strong> 3D cardiac ct volumes using marginal<br />

space learning and steerable features. IEEE Trans. Med.<br />

Imag., 27(11):1668–1681, 2008.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!