A review of 3D/2D registration methods for image-guided interventions
Introduction
Image registration is one of the enabling technologies for image-guided radiation therapy (IGRT) (Jaffray et al., 2007, Jaffray et al., 2008), image-guided radiosurgery (IGRS) (Chang et al., 2003, Dieterich et al., 2008, Romanelli et al., 2006a) and image-guided minimally invasive therapy (IGMIT) which includes a wide variety of therapies in surgery (Peters, 2006, Germano, 2000, Peters and Cleary, 2008), endoscopy (Mayberg et al., 2005) and interventional radiology (Mauro et al., 2008). Registration is concerned with bringing the pre-intervention data (patient’s images or models of anatomical structures obtained from these images and treatment plan) and intra-intervention data (patient’s images, positions of tools, radiation fields, etc.) into the same coordinate frame (Peters, 2006, Sauer, 2005, Galloway, 2001, DiMaio et al., 2007, Yaniv and Cleary, 2006, Romanelli et al., 2006a, Romanelli et al., 2006b, Avanzo and Romanelli, 2009). Currently, the pre-interventional data are three-dimensional (3D) computed tomography (CT) and magnetic resonance (MR) images, while the intra-intervention data are either two-dimensional (2D) ultrasound (US), projective X-ray (fluoroscopy), CT-fluoroscopy, and optical images, or 3D images like cone-beam CT (CBCT) and US, or 3D digitized points or surfaces. MR images are still seldom used intra-interventionaly. With respect to intra-interventional data dimensionality, registration is thus either 3D/2D or 3D/3D.
All the above mentioned medical specialties benefit from image registration through easier and better guidance of an intervention leading to reduced invasiveness and/or increased accuracy. In image-guided minimally invasive surgery, the registration of pre- and intra-interventional data and instrument tracking provide surgeon with information about the current position of his instruments relative to the planned trajectory, nearby vulnerable structures, and the ultimate target. In image-guided endoscopy, 3D virtual images of the anatomy and pathology are generated from pre-interventional images and registered to real-time live endoscopic images to provide augmented reality which enables display of anatomical structures that are hidden from the direct view by currently exposed tissues. In interventional radiology, registration of the pre-interventional image to the X-ray fluoroscopic or US image allows visualization of tools, like catheters and needles, in 3D which can greatly improve guidance. In external beam radiotherapy, registration of planning CT images and daily pre-treatment images allow precise patient positioning, which is of utmost importance for exact dose delivery to the target and for avoiding irradiation of healthy critical tissue. Throughout this survey the term image-guided interventions (IGI) is used to describe IGRT, IGRS, and IGMIT because, as already Peters and Cleary (2008) noted, IGI covers the widest range of surgical and therapeutic procedures. In the literature the terms image-guided therapy (IGT), image-guided surgery (IGS) and image-guided procedures (IGP) are often used instead of IGI.
The aim of this paper is to survey those 3D/2D data registration methods which use a 3D CT or MR pre-interventional image and one or more intra-interventional 2D X-ray projection images as sources of data to be registered. The 3D/3D registration methods where the intra-interventional image is a CBCT, CT, MR, or US image (Jaffray et al., 2002, Pouliot et al., 2005, Jolesz, 2005, Penney et al., 2006) and 3D/2D volume-to-slice (Comeau et al., 2000, Wein et al., 2008, Micu et al., 2006, Birkfellner et al., 2007, Frühwald et al., 2009, Hummel et al., 2008, Fei et al., 2003) and (endoscopic) volume-to-video (Mori et al., 2002, Mori et al., 2005, Bricault et al., 1998, Deligianni et al., 2004, Deligianni et al., 2006, Burschka et al., 2005) registrations are beyond the scope of this review. Besides, registration methods that are based on intra-interventional data extracted from the patient’s skin surface or surfaces of exposed anatomical structures are also not reviewed. Nevertheless, other means of establishing 3D/2D registration that do not rely solely on X-ray images are mentioned where suitable. Although the term 2D/3D registration is more frequently used to describe registration in scope of this paper, we consistently use the term 3D/2D registration instead, since the 3D image is transformed to achieve the best possible correspondence with the 2D image(s), as shown in the following section.
Section snippets
Alignment of pre- and intra-interventional patient data
In IGI, registration is used to align the pre- and intra-interventional data just before and often also during an intervention in such a way that corresponding anatomical structures in the two data sets are aligned. The data sets to be registered are defined in distinct spaces or coordinate systems. The 3D pre-interventional data is defined in the data (image) coordinate system Spre. The intra-interventional 3D data is defined either in some world (patient, treatment room) coordinate system Sw
Survey of 3D/2D registration methods
Maintz and Viergever (1998) proposed a classification of image registration methods according to: image modality (Section 3.1), image dimensionality (Section 3.2), nature of the registration basis (Section 3.3), geometric transformation (Section 3.4), user interaction (Section 3.5), optimization procedure (Section 3.6), subject ( Section 3.7), and object of registration (Section 3.8). Publications, dealing with 3D/2D CT or MR to X-ray image registrations, are next reviewed in view of this
Evaluation of 3D/2D registration methods
As in any other discipline of medical image processing, evaluation has become an integral part of peer-reviewed publications on 3D/2D registration. Evaluation is paramount as it allows to determine the performance and limitations of a proposed method. Furthermore, evaluation also clarifies the potential clinical applications and added value of a method (Jannin et al., 2002). A prerequisite for evaluation of (3D/2D) image registration is standardization of evaluation methodology which includes:
Conclusion
An overview of 3D/2D data registration methods that utilize 3D pre-interventional CT or MR images and 2D intra-interventional X-ray projection images has been presented. The publications were surveyed according to the classification proposed by Maintz and Viergever (1998). Using such a classification several aspects of 3D/2D registration were examined.
As could be expected, the review of relevant literature did not put forth any group of methods as clearly superior to others. Rather, the choice
Acknowledgment
The authors are grateful to J. Spoerk and W. Birkfellner, Center for Biomedical Engineering and Physics, Medical University Vienna, for providing the DRR images. This work was supported by the Ministry of Higher Education, Science and Technology, Republic of Slovenia under the Grants P2-0232, L2-7381, L2-9758, Z2-9366, J2-0716, L2-2023 and J7-2246.
References (232)
- et al.
Accurate and precise 2D–3D registration based on X-ray intensity
Comput. Vis. Image Und.
(2008) - et al.
Measurements of intrafraction motion and interfraction and intrafraction rotation of prostate by three-dimensional analysis of daily portal imaging with radiopaque markers
Int. J. Radiat. Oncol. Biol. Phys.
(2004) - et al.
An integrated range-sensing, segmentation and registration framework for the characterization of intra-surgical brain deformations in image-guided surgery
Comput. Vis. Image Und.
(2003) - et al.
3D/2D registration and segmentation of scoliotic vertebrae using statistical models
Comput. Med. Imag. Graph.
(2003) - et al.
Pre- and intra-irradiation multimodal image registration: principles and first experiments
Radiother. Oncol.
(1993) - et al.
Registration of 3D cerebral vessels with 2D digital angiograms: clinical evaluation
Acad. Radiol.
(1999) - et al.
Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery
Med. Image Anal.
(2005) - et al.
A gradient feature weighted minimax algorithm for registration of multiple portal images to 3DCT volumes in prostate radiotherapy
Int. J. Radiat. Oncol. Biol. Phys.
(2006) - et al.
Magnetic resonance-based treatment planning for prostate intensity-modulated radiotherapy: creation of digitally reconstructed radiographs
Int. J. Radiat. Oncol. Biol. Phys.
(2007) - et al.
Patient setup error measurement using 3D intensity-based image registration techniques
Int. J. Radiat. Oncol. Biol. Phys.
(2003)