SATO ET AL.: TISSUE CLASSIFICATION BASED ON **3D** LOCAL INTENSITY STRUCTURES FOR VOLUME RENDERING 177 it suffers from the risk of fatal overlook, operatordependence, and operator's burden. Our approach, that is, a combinati**on** of enhancement filtering and volume rendering with opacity adjustment, provides more objective visualizati**on** and greatly reduces the operator's burden. The following future work is envisaged: First, semiautomated optimizati**on** of opacity functi**on**s should be investigated. As our method now stands, opacity functi**on** parameter tuning is manually performed and is not ong>basedong> **on** mathematical criteria. We c**on**sider that such criteria will be obtainable by analyzing 2D histograms of whole and sampled data (as shown in Fig. 10b). However, the sampled data should not be acquired by time-c**on**suming manual tracing. We are now building an interactive system for opacity functi**on** optimizati**on** ong>basedong> **on** whole and sampled data acquired by specifying a tissue of interest using a simple VOI shape. Sec**on**d, the utility of **3D** **local** structures should be evaluated from the clinical point of view. Each of the medical problems referred to in Secti**on** 5.2 is clinically important. We believe that clinical validati**on** of our method, using a large set of volume data for each specific medical area, is an important aspect of future work. 6.2 Effect of Anisotropic Voxels Our **3D** **local** structure filtering methods assume the isotropy of voxels in the input volume data. While we use sinc interpolati**on** preprocessing in the third (z-axis) directi**on** to make voxels isotropic, such preprocessed data are inherently blurrier in the third directi**on**. The effect of anisotropic blurring **on** **3D** filtering depends **on** the directi**on**s of **local** structures. For example, sheet (line) structures with narrow widths easily collapse when sheet normal (line) directi**on**s are close to (deviate from) the third directi**on**. Three-dimensi**on**al **local** structure filtering would not be so effective in these cases. In recent work related to this problem, the effect of anisotropic voxels **on** the width quantificati**on** of sheet structures and its dependence **on** sheet normal directi**on**s were investigated in [36]. The results showed that anisotropy has a c**on**siderable effect **on** quantificati**on** accuracy. In the near future, however, the acquisiti**on** of volume data with (quasi-)isotropic voxels is expected to become much more comm**on** because of recent advances in CT and MR scanner technology in terms of high speed and resoluti**on**, such as multislice CT scanners [37], which means the problem of anisotropic voxels will become much less troublesome. 6.3 Design of a Classificati**on** Strategy We have dem**on**strated ong>classificati**on**ong> strategies ong>basedong> **on** two guidelines: an edge filter for removing partial voluming by suppressing the edge structure and sec**on**d-order **local** structure filters for highlighting or suppressing sheet, line, or blob structures. These **local** structures were combined with the original **intensity** to define a multidimensi**on**al feature space. The ong>classificati**on**ong> strategies were designed ong>basedong> **on** an interactive analysis of the **local** **intensity** structure of each tissue class following the two suggested guidelines. Although the processes for strategy design described in Secti**on** 4.2 are relatively straightforward, in practice they often involve trial and error because the criteria for filter selecti**on** are somewhat intuitive and the resultant ong>classificati**on**ong> strategy is not guaranteed to be optimal. Future work should therefore include the development of an automated method of ascertaining the optimal strategy, which will involve optimal filter type selecti**on**, as well as parameter adjustment. One way to accomplish it is to use traditi**on**al pattern ong>classificati**on**ong> methods ong>basedong> **on** multidimensi**on**al feature space analysis. Another possible approach would be to use an expert system which c**on**structs an optimal strategy from examples of target (positive) and unwanted (negative) tissues [38], [39]. APPENDIX A DERIVATION OF WIDTH RESPONSE CURVES FOR SECOND-ORDER STRUCTURES The normalized resp**on**se of the sheet filter to the sheet model, S sheet fh sheet …x; r †; f g, is written as S sheet fh sheet …x; r †; f gˆ d2 dx 2 G…x; f† h sheet …x; r †; …39† where is the normalizati**on** factor. The width resp**on**se curve for the sheet structure is defined by S sheet fh sheet …0; r †; f g, where 0 ˆ…0; 0; 0†. The sheet structure with variable width r is modeled by h sheet …x; r †ˆexp x2 2 2 r p ˆ 2 r G…x; r †; …40† where G…x; r † is the 1D Gaussian functi**on**. By combining (39) and (40), we have: S sheet fh sheet …x; r †; f g p ˆ 2 r d2 dx 2 G…x; f†G…x; r † …41† p q ˆ 2 r @2 @x 2 G x; 2 f ‡ 2 r ; …42† in which we used the relati**on** q G…x; 1 †G…x; 2 †ˆG…x; 2 1 ‡ 2 2†: Since d 2 dx 2 G…x; † ˆ x 2 p 2 1 p exp x2 5 2 3 2 2 ; …43† the width resp**on**se curve for the sheet is given by r S sheet fh sheet …0; r †; f gˆq 3 2 f ‡ 2 r 2 ˆ f …44† r 2 r ! f 3 : 2‡1 f r

178 **IEEE** TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 6, NO. 2, APRIL-JUNE 2000 Thus, S sheet fh sheet …0; r †; f g is essentially a functi**on** of f r when ˆ 2 f ,anditsmaximumis p p 2 … 3 † 3 … 0:385†when f r ˆ 2 . Using similar derivati**on**s, when the normalizati**on** factor ˆ 2 f is multiplied, the width resp**on**se curve for the line is given by 2 f 2 r 2 S line fh line …0; r †; f gˆq 4 ˆ q 4 ; 2 f ‡ 2 r … f r † 2 ‡ 1 whose maximum is 1 4 …ˆ 0:25† when f r ˆ 1. The width resp**on**se curve for the blob is given by 2 f 3 r S blob fh blob …0; r †; f gˆq 5 ˆ 2 f ‡ 2 r q whose maximum is 2 3 … 3† 5 … 0:186† when f APPENDIX B 5 f r 2 f r r ! 5 ; 2‡1 f r r ˆ q 2. 3 …45† …46† SINC INTERPOLATION wITHOUT GIBBS RINGING The sinc interpolati**on** al**on**g the third (z-axis) directi**on** is performed by zero-filled expansi**on** in the frequency domain [22], [23]. Let f…i† (i ˆ 0; 1; ...;n 1) be the profile in the third directi**on**. In the discrete Fourier transform of f…i†, f…i† should be regarded as cyclic and then f…n 1† and f…0† are essentially adjacent. Unwanted Gibbs ringing occurs in the interpolated profile due to the disc**on**tinuity between f…n 1† and f…0†. Thus, Gaussian-shaped slopes were added at the beginning and end of f…i† to avoid the occurence of unwanted ringing before the sinc interpolati**on**. Let f 0 …i† (i ˆ3 ; ...; 0; 1; ...;n 1;n;...; 3 ‡ n) be the modified profile, which is given by 8 < G…i; †f…0†; i ˆ3 ; ...; 0 f 0 …i† ˆ f…i†; i ˆ 0; ...;n 1 …47† : G…i n ‡ 1; †f…n 1†; i ˆ n; ...; 3 ; where G…x; † is the Gaussian functi**on** and the variati**on** is sufficiently smooth everywhere, including between f…3 ‡ n† and f…3 †. The discrete Fourier transform of f 0 …i† was performed (we used ˆ 4). After the sinc interpolati**on** of f 0 …i†, the added Gaussian-shaped slopes were removed. ACKNOWLEDGMENTS This work was partly supported by U.S. Nati**on**al Insitutes of Health grant P01 CA67165-02 and JSPS Grant-in-Aid for Scientific Research (C)(2) 11680389. The authors would like to thank Dr. Gary Zientara and Dr. Petra Huppi of Harvard Medical School and Brigham and Women's Hospital for providing MR images of a baby's brain, Dr. Hir**on**obu Ohmatsu of the Nati**on**al Cancer Center, Japan, for providing CT images of a chest, and Dr. D**on** Johann of New York University Medical School for providing MR images of a brain for neurosurgical planning. Yoshinobu Sato would also like to thank Dr. Hideki Atsumi and Dr. Nobuhiko Hata for helping him adapt to the envir**on**ment of the Surgical Planning Laboratory at Harvard Medical School and Brigham and Women's Hospital and Prof. Ferenc Jolesz of Harvard Medical School for c**on**tinuous encouragement. The work described in this paper was partly c**on**ducted while Yoshinobu Sato was a visiting research at the Surgical Planning Laboratory, Department of Radiology, Harvard Medical School and Brigham and Women's Hospital. REFERENCES [1] M. Levoy, ªVolume Rendering: Display of Surfaces from Volume Data,º **IEEE** Computer Graphics and Applicati**on**s, pp. 29-36, 1988. [2] R.A. Drebin, L. Carpenter, and P. Hanrahan, ªVolume Rendering,º Computer Graphics (Proc. SIGGRAPH '88), vol. 22, pp. 65-73, 1988. [3] G.D. Rubin, C.F. Beullieu, V. Anngiro, H. Ringle, A.M. Norbash, J.F. Feller, M.D. Dake, R.B. Jefferey, and S. Naple, ªPerspective Volume Rendering of CT and MR Images: Applicati**on**s for Endoscopic Imaging,º Radiology, vol. 199, no. 2, pp. 321-330, 1996. [4] P.T. Johns**on**, D.G. Heath, B.S. Kuszyk, and E.K. Fishman, ªCT Angiography with Volume Rendering: Advantages and Applicati**on**s in Splanchnic Vascular Imaging,º Radiology, vol. 200, no. 2, pp. 564-568, 1996. [5] C. Barillot, ªSurface and Volume Rendering Techniques to Display 3-D Data,º **IEEE** Eng. in Medicine and Biology, vol. 12, pp. 111-119, 1993. [6] M.W. Vannier, R.L. Butterfield, D. Jord**on**, W.A. Murphy, R.G. Levitt, and M. Gado, ªMultispectral Analysis of Magnetic Res**on**ance Images,º Radiology, vol. 154, no. 1, pp. 221-224, 1985. [7] M.C. Clark, L.O. Hall, D.B. Goldgof, L.P. Clarke, R.P. Velthuizen, and M.S. Silbiger, ªMRI Segmentati**on** Using Fuzzy Clustering Techniques,º **IEEE** Eng. in Medicine and Biology, vol. 13, pp. 730- 742, 1994. [8] Y. Sato, C.-F. Westin, A. Bhalerao, S. Nakajima, N. Shiraga, S. Yoshida, and R. Kikinis, ªong>Tissueong> Classificati**on** Based **on** **3D** Local Intensity Structures for Volume Rendering,º Proc. Japanese Associ. Medical Imaging Technology Fr**on**tier '97, pp. 167-172. 1997. (Mpeg movies are available at http://www.image.med.osaka-u.ac.jp/ ~yoshi/vr_index.html). [9] Y. Sato, S. Nakajima, H. Atsumi, T. Koller, G. Gerig, S. Yoshida, and R. Kikinis, ª**3D** Multiscale Line Filter for Segmentati**on** and Visualizati**on** of Curvilinear Structures in Medical Images,º Proc. Joint C**on**f. Computer Visi**on**, Virtual Reality, and Robotics in Medicine and Medical Robotics and Computer-Assisted Surgery '97, pp. 213-222, 1997. [10] Y. Sato, S. Nakajima, N. Shiraga, H. Atsumi, S. Yoshida, T. Koller, G. Gerig, and R. Kikinis, ªThree-Dimensi**on**al Multiscale Line Filter for Segmentati**on** and Visualizati**on** of Curvilinear Structures in Medical Images,º Medical Image Analysis, vol. 2, no. 2, pp. 143-168, 1998. [11] H. Knutss**on**, ªRepresenting Local Structure Using Tensors,º Proc. Sixth Scandinavian C**on**f. Image Analysis, pp. 244-251, 1989. [12] C.-F. Westin, ªA Tensor Framework for Multidimensi**on**al Signal Processing,º thesis no. 348, LinkoÈping Univ., Sweden, 1994. [13] T.M. Koller, G. Gerig, G. Szekely, and D. Dettwiler, ªMultiscale Detecti**on** of Curvilinear Structures in 2-D and 3-D Image Data,º Proc. Fifth Int'l C**on**f. Computer Visi**on** (ICCV '95), pp. 864-869, 1995. [14] G. Kindlmann and J. Durkin, ªSemi-Automatic Generati**on** of Transfer Functi**on**s for Direct Volume Rendering,º Proc. **IEEE** Symp. Volume Visualizati**on**, pp. 79-86, 1998. [15] D.H. Laidlaw, K.W. Fleischer, and A.H. Barr, ªPartial-Volume Bayesian Classificati**on** of Material Mixtures in MR Volume Data Using Voxel Histograms,º **IEEE** Trans. Medical Imaging, vol. 17, no. 1, pp. 74-86, 1998. [16] R.M. Haralick, L.T. Wats**on**, and T.J. Laffey, ªThe Topographic Primal Sketch,º Int'l J. Robotic Research, vol. 2, pp. 50-72, 1983. [17] C.-F. Westin, S. Warfield, A. Bhalerao, L. Mui, J. Richolt, and R. Kikinis, ªTensor C**on**trolled Local Structure Enhancement of CT Images for B**on**e Segmentati**on**,º Proc. Int'l C**on**f. Medical Image Computing and Computer-Assisted Interventi**on** '98, pp. 1,205-1,212, 1998.