Iconic feature based nonrigid registration: the PASHA algorithm
Introduction
Nonrigid image registration is an important task of image processing. In medical image analysis, it is a fundamental step as soon as we want to quantify the evolution of a patient in a follow-up study, or when comparing two different patients. Consequently, it is a very creative field of research; techniques are numerous and inspired from a wide range of theories or techniques: statistics and information theory, theory of continuum mechanics or viscoelastic fluids, theory of thermodynamics, optical flow, splines, wavelets, block matching, and so on.
To get a better understanding of the different technical choices one faces when designing a registration algorithm, several classifications have been proposed [8], [36], [38], [61]. One major axis shared by all these classifications is the image feature axis, i.e., the kind of information that drives the registration process.
Most, if not all, classifications split this axis into two parts: on the one hand, geometric algorithms, which use a geometric distance between segmented features in the images; on the other hand, intensity based algorithms, which use a similarity measure between the image intensities.
However, we found that the group of intensity based algorithms includes two different registration approaches that behave very differently, independently of the similarity measure or the deformation model. We formalize their difference in Section 2 by introducing the notion of iconic (i.e., image intensity related) feature based registration and show how it changes the standard classification of registration algorithms. In Section 3, we propose a new registration energy for iconic feature based registration. We show that this energy generalizes the “demons” algorithm, as well as Feldmar’s “generalized ICP,” and enables a better insight of the behavior of these algorithms. Based on this energy, we develop the PASHA2 algorithm. This energy is general and may use different similarity or regularization energies: in this respect, we present Gaussian-weighted local similarity measures in Section 4, which are efficiently computed using an original convolution based technique, as well as an original mixed elastic/fluid regularization in Section 5. We compare PASHA to close algorithms in Section 6. Finally, we present in Section 7 a clinical application of PASHA for brain motion recovery during deep brain stimulation of Parkinsonian patients, and show how our algorithm recovers and propagates deformation in the brain with smooth, realistic displacement fields.
Section snippets
A new classification
Despite the large number of techniques used in registration, the main classifications found in the literature all use at least the following two major axes:
- •
The deformation model, used to regularize the registration problem. It expresses the prior knowledge we have on the shape of the transformation.
- •
The image features used by the algorithm to guide the deformation model towards (hopefully) the desired transformation.
We first briefly detail in Section 2.1 the different kinds of deformation
An energy for iconic feature based registration
Looking at the existing IFB algorithms, one remarks that they generally do not minimize a global energy. IFB algorithms like the “demons” or block matching, proceed in alternating two steps. In a first step, they search for a set of correspondences C, using an intensity similarity measure. In a second step, they search for a transformation T that approximates this set of correspondences, using one of the regularization techniques presented in Section 2.1. However, even if each of these steps
Local statistic based similarity measures
In the previous section, we introduced a registration energy (2) without specifying any similarity or regularization energy. Indeed, registration algorithms seldom have a unique set of tools: most are flexible and propose a panel of energies to better suit any registration problem [49]. This is also the case with PASHA.
In this section, we present similarity measures based on local statistics, as they are implemented in PASHA. We believe that our technique could be interesting for a wide range
Regularization
A registration algorithm can also propose different kind of regularization energies—although this is far less common than for similarity measures. A study of the different regularization energies is far beyond our scope here; one can take a look at [11] for a panel of possibilities. The important feature of PASHA is that when R is quadratic and uniform, the regularization is done using convolutions. This enables particularly simple and fast regularization techniques; we illustrate it here by
Comparison with the “demons” algorithm and its extensions
The “demons” algorithm [60] and its extensions [6], [14], [30] are a limit case of PASHA when the parameter σ of Eq. (2) tends to zero (S being then the SSD, and R the energy yielding Gaussian filtering). Indeed, in that case, the closeness constraint between T and C disappears during the first step, and we end up minimizing the SSD alone: this is exactly how these algorithms work. Following Section 3.2, this means that these algorithms assume that the images are noiseless, or more generally,
A clinical application to neurosurgery
Previous sections presented theoretical aspects of PASHA, as well as some comparisons with other approaches. Despite the excellent results obtained previously, it is important to show that the algorithm works for real studies as well. Actually, our algorithm has been used already in several studies, mainly for brain tracking in ultrasound images [47], and for multipatient MRI registration [12]—not to mention all the studies based on the “demons” algorithm (e.g., [23], [30]), of which PASHA is a
Conclusion
In this paper, we have first highlighted a fundamental difference that exists between intensity based registration algorithms. On the one hand, standard intensity based (SIB) algorithms use an intensity similarity measure to quantify the quality of the registration. On the other hand, iconic feature based (IFB) algorithms use a geometric distance between homologous geometric features, whose pairing is based on intensities. This last category includes the “demons” algorithm, the “generalized
References (68)
- et al.
Extension of the ICP algorithm to nonrigid intensitybased registration of 3D volumes
Comp. Vision Image Understanding
(1997) On matching brain volumes
Pattern Recog.
(1999)- et al.
A survey of hierarchical non-linear medical image registration
Pattern Recog.
(1999) - et al.
A survey of medical image registration
Med. Image Anal.
(1998) - et al.
Reconstructing a 3D structure from serial histological sections
Image Vision Comp.
(2001) - et al.
Rigid point feature registration using mutual information
Med. Image Anal.
(1999) - et al.
Estimating tissue deformation between functional images induced by intracranial electrode implantation using anatomical MRI
NeuroImage
(2001) - et al.
A general scheme for automatically building 3D morphometric anatomical atlases: application to a skull atlas
Med. Image Anal.
(1998) Image matching as a diffusion process: an analogy with Maxwell’s demons
Med. Image Anal.
(1998)- et al.
The role of image registration in brain mapping
Image Vision Comput.
(2001)
Multi-modal volume registration by maximization of mutual information
Med. Image Anal.
A rapid and automatic image registration algorithm with subpixel accuracy
IEEE Trans. Med. Imag.
Surface-bounded growth modeling applied to human mandibles
IEEE Trans. Med. Imag.
Image registration using a symmetric prior—in three dimensions
Human Brain Mapp.
Nonlinear spatial normalization using basis functions
Human Brain Mapp.
Principal warps: thin-plate splines and the decomposition of deformations
IEEE Trans. Pattern Anal. Mach. Intell.
Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy
IEEE Trans. Med. Imag.
Fast fluid registration of medical images
A survey of image registration techniques
ACM Comput. Surveys
Multisubject non-rigid registration of brain MRI using intensity and geometric features
3D non-rigid registration by gradient descent on a Gaussian-windowed similarity measure using convolutions
Symmetrization of the non-rigid registration problem using inversion-invariant energies: application to multiple sclerosis
Geodesic interpolating splines
Volumetric transformation of brain anatomy
IEEE Trans. Med. Imag.
Deformable templates using large deformation kinematics
IEEE Trans. Image Process.
Auxiliary variables and two-step iterative algorithms in computer vision problems
J. Math. Imag. Vision
Automated multi-modality image registration based on information theory
ANIMAL validation and applications of nonlinear registration based segmentation
Int. J. Pattern Recog. Artif. Intell.
Non-linear cerebral registration with sulcal constraints
Brain atlas deformation in the presence of large space-occupying tumors
Cited by (209)
Enhancing MR image segmentation with realistic adversarial data augmentation
2022, Medical Image AnalysisBrain-mimicking phantom for biomechanical validation of motion sensitive MR imaging techniques
2021, Journal of the Mechanical Behavior of Biomedical MaterialsTissue-specific deformable image registration using a spatial-contextual filter
2021, Computerized Medical Imaging and GraphicsDevelopment, calibration, and testing of 3D amplified MRI (aMRI) for the quantification of intrinsic brain motion
2021, Brain MultiphysicsCitation Excerpt :Trilinear interpolation is then used to interpolate between the sampling points which can result in the measurement of the sub-voxel displacement. Overall, the original Demons algorithm could be considered as an optimization procedure over the whole displacement field space [47,48]. The main idea of 3D Slicer implementation of this algorithm is to adapt this optimization procedure to a diffeomorphic transformation space, to achieve transformations smoother and closer to reality.
Thoracic hybrid PET/CT registration using improved hybrid feature intensity multimodal demon
2020, Radiation Physics and Chemistry
- 1
Now with the International University in Germany.