Elsevier

Medical Image Analysis

Volume 16, Issue 3, April 2012, Pages 642-661
Medical Image Analysis

A review of 3D/2D registration methods for image-guided interventions

https://doi.org/10.1016/j.media.2010.03.005Get rights and content

Abstract

Registration of pre- and intra-interventional data is one of the key technologies for image-guided radiation therapy, radiosurgery, minimally invasive surgery, endoscopy, and interventional radiology. In this paper, we survey those 3D/2D data registration methods that utilize 3D computer tomography or magnetic resonance images as the pre-interventional data and 2D X-ray projection images as the intra-interventional data. The 3D/2D registration methods are reviewed with respect to image modality, image dimensionality, registration basis, geometric transformation, user interaction, optimization procedure, subject, and object of registration.

Introduction

Image registration is one of the enabling technologies for image-guided radiation therapy (IGRT) (Jaffray et al., 2007, Jaffray et al., 2008), image-guided radiosurgery (IGRS) (Chang et al., 2003, Dieterich et al., 2008, Romanelli et al., 2006a) and image-guided minimally invasive therapy (IGMIT) which includes a wide variety of therapies in surgery (Peters, 2006, Germano, 2000, Peters and Cleary, 2008), endoscopy (Mayberg et al., 2005) and interventional radiology (Mauro et al., 2008). Registration is concerned with bringing the pre-intervention data (patient’s images or models of anatomical structures obtained from these images and treatment plan) and intra-intervention data (patient’s images, positions of tools, radiation fields, etc.) into the same coordinate frame (Peters, 2006, Sauer, 2005, Galloway, 2001, DiMaio et al., 2007, Yaniv and Cleary, 2006, Romanelli et al., 2006a, Romanelli et al., 2006b, Avanzo and Romanelli, 2009). Currently, the pre-interventional data are three-dimensional (3D) computed tomography (CT) and magnetic resonance (MR) images, while the intra-intervention data are either two-dimensional (2D) ultrasound (US), projective X-ray (fluoroscopy), CT-fluoroscopy, and optical images, or 3D images like cone-beam CT (CBCT) and US, or 3D digitized points or surfaces. MR images are still seldom used intra-interventionaly. With respect to intra-interventional data dimensionality, registration is thus either 3D/2D or 3D/3D.

All the above mentioned medical specialties benefit from image registration through easier and better guidance of an intervention leading to reduced invasiveness and/or increased accuracy. In image-guided minimally invasive surgery, the registration of pre- and intra-interventional data and instrument tracking provide surgeon with information about the current position of his instruments relative to the planned trajectory, nearby vulnerable structures, and the ultimate target. In image-guided endoscopy, 3D virtual images of the anatomy and pathology are generated from pre-interventional images and registered to real-time live endoscopic images to provide augmented reality which enables display of anatomical structures that are hidden from the direct view by currently exposed tissues. In interventional radiology, registration of the pre-interventional image to the X-ray fluoroscopic or US image allows visualization of tools, like catheters and needles, in 3D which can greatly improve guidance. In external beam radiotherapy, registration of planning CT images and daily pre-treatment images allow precise patient positioning, which is of utmost importance for exact dose delivery to the target and for avoiding irradiation of healthy critical tissue. Throughout this survey the term image-guided interventions (IGI) is used to describe IGRT, IGRS, and IGMIT because, as already Peters and Cleary (2008) noted, IGI covers the widest range of surgical and therapeutic procedures. In the literature the terms image-guided therapy (IGT), image-guided surgery (IGS) and image-guided procedures (IGP) are often used instead of IGI.

The aim of this paper is to survey those 3D/2D data registration methods which use a 3D CT or MR pre-interventional image and one or more intra-interventional 2D X-ray projection images as sources of data to be registered. The 3D/3D registration methods where the intra-interventional image is a CBCT, CT, MR, or US image (Jaffray et al., 2002, Pouliot et al., 2005, Jolesz, 2005, Penney et al., 2006) and 3D/2D volume-to-slice (Comeau et al., 2000, Wein et al., 2008, Micu et al., 2006, Birkfellner et al., 2007, Frühwald et al., 2009, Hummel et al., 2008, Fei et al., 2003) and (endoscopic) volume-to-video (Mori et al., 2002, Mori et al., 2005, Bricault et al., 1998, Deligianni et al., 2004, Deligianni et al., 2006, Burschka et al., 2005) registrations are beyond the scope of this review. Besides, registration methods that are based on intra-interventional data extracted from the patient’s skin surface or surfaces of exposed anatomical structures are also not reviewed. Nevertheless, other means of establishing 3D/2D registration that do not rely solely on X-ray images are mentioned where suitable. Although the term 2D/3D registration is more frequently used to describe registration in scope of this paper, we consistently use the term 3D/2D registration instead, since the 3D image is transformed to achieve the best possible correspondence with the 2D image(s), as shown in the following section.

Section snippets

Alignment of pre- and intra-interventional patient data

In IGI, registration is used to align the pre- and intra-interventional data just before and often also during an intervention in such a way that corresponding anatomical structures in the two data sets are aligned. The data sets to be registered are defined in distinct spaces or coordinate systems. The 3D pre-interventional data is defined in the data (image) coordinate system Spre. The intra-interventional 3D data is defined either in some world (patient, treatment room) coordinate system Sw

Survey of 3D/2D registration methods

Maintz and Viergever (1998) proposed a classification of image registration methods according to: image modality (Section 3.1), image dimensionality (Section 3.2), nature of the registration basis (Section 3.3), geometric transformation (Section 3.4), user interaction (Section 3.5), optimization procedure (Section 3.6), subject ( Section 3.7), and object of registration (Section 3.8). Publications, dealing with 3D/2D CT or MR to X-ray image registrations, are next reviewed in view of this

Evaluation of 3D/2D registration methods

As in any other discipline of medical image processing, evaluation has become an integral part of peer-reviewed publications on 3D/2D registration. Evaluation is paramount as it allows to determine the performance and limitations of a proposed method. Furthermore, evaluation also clarifies the potential clinical applications and added value of a method (Jannin et al., 2002). A prerequisite for evaluation of (3D/2D) image registration is standardization of evaluation methodology which includes:

Conclusion

An overview of 3D/2D data registration methods that utilize 3D pre-interventional CT or MR images and 2D intra-interventional X-ray projection images has been presented. The publications were surveyed according to the classification proposed by Maintz and Viergever (1998). Using such a classification several aspects of 3D/2D registration were examined.

As could be expected, the review of relevant literature did not put forth any group of methods as clearly superior to others. Rather, the choice

Acknowledgment

The authors are grateful to J. Spoerk and W. Birkfellner, Center for Biomedical Engineering and Physics, Medical University Vienna, for providing the DRR images. This work was supported by the Ministry of Higher Education, Science and Technology, Republic of Slovenia under the Grants P2-0232, L2-7381, L2-9758, Z2-9366, J2-0716, L2-2023 and J7-2246.

References (232)

  • A.C.F. Colchester et al.

    Development and preliminary evaluation of VISLAN, a surgical planning and guidance system using intra-operative video imaging

    Med. Image Anal.

    (1996)
  • P.W. de Bruin et al.

    Image-based RSA: Roentgen stereophotogrammetric analysis based on 2D–3D image registration

    J. Biomech.

    (2008)
  • D.A. Dennis et al.

    In vivo determination of normal and anterior cruciate ligament-deficient knee kinematics

    J. Biomech.

    (2005)
  • S. DiMaio et al.

    Challenges in image-guided therapy system design

    Neuroimage

    (2007)
  • J. Feldmar et al.

    3D–2D projective registration of free-form curves and surfaces

    Comput. Vis. Image Und.

    (1997)
  • K.P. Gall et al.

    Experience using radiopaque fiducial points for patient alignment during radiotherapy

    Int. J. Radiat. Oncol. Biol. Phys.

    (1993)
  • K.G.A. Gilhuijs et al.

    Interactive three dimensional inspection of patient setup in radiation therapy using digital portal images and computed tomography data

    Int. J. Radiat. Oncol. Biol. Phys.

    (1996)
  • M. Goitein et al.

    Multidimensional treatment planning. 2. Beam eye-view, back projection, and projection through CT sections

    Int. J. Radiat. Oncol. Biol. Phys.

    (1983)
  • M. Groher et al.

    Planning and intraloperative visualization of liver catheterizations: new CTA protocol and 2D–3D registration method

    Acad. Radiol.

    (2007)
  • A. Gueziec et al.

    Providing visual information to validate 2-D to 3-D registration

    Med. Image Anal.

    (2000)
  • W.A. Hoff et al.

    Three-dimensional determination of femoral-tibial contact positions under in vivo conditions using fluoroscopy

    Clin. Biomech.

    (1998)
  • M. Imura et al.

    Insertion and fixation of fiducial markers for setup and tracking of lung tumors in radiotherapy

    Int. J. Radiat. Oncol. Biol. Phys.

    (2005)
  • D.A. Jaffray et al.

    Flat-panel cone-beam computed tomography for image-guided radiation therapy

    Int. J. Radiat. Oncol. Biol. Phys.

    (2002)
  • J.R. Adler et al.

    Image-guided robotic radiosurgery

    Neurosurgery

    (1999)
  • N. Agazaryan et al.

    Image-guided radiosurgery for spinal tumors: methods, accuracy and patient intrafraction motion

    Phys. Med. Biol.

    (2008)
  • E.G.A. Aird et al.

    CT simulation for radiotherapy treatment planning

    Brit. J. Radiol.

    (2002)
  • N. Alperin et al.

    Retrospective registration of X-ray angiograms with MR images by using vessels as intrinsic landmarks

    J. Magn. Reson. Imag.

    (1994)
  • K.S. Arun et al.

    Least-squares fitting of 2 3-D point sets

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1987)
  • M. Avanzo et al.

    Spinal radiosurgery: technology and clinical outcomes

    Neurosurg. Rev.

    (2009)
  • R.J. Bale et al.

    Computer-assisted neurosurgery by using a noninvasive vacuum-affixed dental cast that acts as a reference base: another step toward a unified approach in the treatment of brain tumors

    J. Neurosurg.

    (2000)
  • S.A. Banks et al.

    Accurate measurement of three-dimensional knee replacement kinematics using single-plane fluoroscopy

    IEEE Trans. Biomed. Eng.

    (1996)
  • R. Bansal et al.

    A minimax entropy registration framework for patient setup verification in radiotherapy

    Comput. Aided Surg.

    (1999)
  • R. Bansal et al.

    Entropy-based dual-portal-to-3-DCT registration incorporating pixel correlation

    IEEE Trans. Med. Imag.

    (2003)
  • S. Benameur et al.

    Three-dimensional biplanar reconstruction of scoliotic rib cage using the estimation of a mixture of probabilistic prior models

    IEEE Trans. Biomed. Eng.

    (2005)
  • S. Benameur et al.

    A hierarchical statistical modeling approach for the unsupervised 3-D biplanar reconstruction of the scoliotic spine

    IEEE Trans. Biomed. Eng.

    (2005)
  • P.J. Besl et al.

    A method for registration of 3-D shapes

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1992)
  • J. Bijhold

    Three-dimensional verification of patient placement during radiotherapy using portal images

    Med. Phys.

    (1993)
  • W. Birkfellner et al.

    A faster method for 3D/2D medical image registration – a simulation study

    Phys. Med. Biol.

    (2003)
  • W. Birkfellner et al.

    Fast DRR generation for 2D/3D registration

  • W. Birkfellner et al.

    Wobbled splatting – a fast perspective volume rendering method for simulation of X-ray images from CT

    Phys. Med. Biol.

    (2005)
  • W. Birkfellner et al.

    Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images

    Med. Phys.

    (2007)
  • W. Birkfellner et al.

    Stochastic rank correlation: a robust merit function for 2D/3D registration of image data obtained at different energies

    Med. Phys.

    (2009)
  • D. Brandenberger et al.

    Positioning accuracy in a registration-free CT-based navigation system

    Phys. Med. Biol.

    (2007)
  • I. Bricault et al.

    Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy

    IEEE Trans. Med. Imag.

    (1998)
  • J.V. Byrne et al.

    Assessment of a technique for 2D–3D registration of cerebral intra-arterial angiography

    Brit. J. Radiol.

    (2004)
  • B. Cabral et al.

    Accelerated volume rendering and tomographic reconstruction using texture mapping hardware

  • Chan, H.M., Chung, A.C.S., Yu, S., Wells, W.M.I.I.I., 2004. 2D–3D vascular registration between digital subtraction...
  • S.D. Chang et al.

    An analysis of the accuracy of the cyberknife: a robotic nameless stereotactic radiosurgical system

    Neurosurgery

    (2003)
  • Chen, X., Varley, M.R., Shark, L.K., Shentall, G.S., Kirby, M.C., 2006. An extension of iterative closest point...
  • X. Chen et al.

    Automatic 3D-to-2D registration for CT and dual-energy digital radiography for calcification detection

    Med. Phys.

    (2007)
  • Cited by (0)

    View full text