Abstract
Registration is a fundamental task in image processing used to match two or more pictures taken, for example, at different times, from different sensors, or from different viewpoints. Virtually all large systems which evaluate images require the registration of images, or a closely related operation, as an intermediate step. Specific examples of systems where image registration is a significant component include matching a target with a real-time image of a scene for target recognition, monitoring global land usage using satellite images, matching stereo images to recover shape for autonomous navigation, and aligning images from different medical modalities for diagnosis.
Over the years, a broad range of techniques has been developed for various types of data and problems. These techniques have been independently studied for several different applications, resulting in a large body of research. This paper organizes this material by establishing the relationship between the variations in the images and the type of registration techniques which can most appropriately be applied. Three major types of variations are distinguished. The first type are the variations due to the differences in acquisition which cause the images to be misaligned. To register images, a spatial transformation is found which will remove these variations. The class of transformations which must be searched to find the optimal transformation is determined by knowledge about the variations of this type. The transformation class in turn influences the general technique that should be taken. The second type of variations are those which are also due to differences in acquisition, but cannot be modeled easily such as lighting and atmospheric conditions. This type usually effects intensity values, but they may also be spatial, such as perspective distortions. The third type of variations are differences in the images that are of interest such as object movements, growths, or other scene changes. Variations of the second and third type are not directly removed by registration, but they make registration more difficult since an exact match is no longer possible. In particular, it is critical that variations of the third type are not removed. Knowledge about the characteristics of each type of variation effect the choice of feature space, similarity measure, search space, and search strategy which will make up the final technique. All registration techniques can be viewed as different combinations of these choices. This framework is useful for understanding the merits and relationships between the wide variety of existing techniques and for assisting in the selection of the most suitable technique for a specific problem.
- ALLINEY, S., AND MORANDI, C. 1986. Digital image registration using projections. IEEE Trans. Pattern Anal. Machtne In{ell. PAMI-8, 2 (Mar.), 222-233.]] Google ScholarDigital Library
- BA.JSCY. R., AND BROIT, C. 1982 Matching of deformed images. In The 6th International Conference on Pattern Recognition. pp. 351-353.]]Google Scholar
- B^.Jsc~, R., AND KOVACIC, S. 1989. Multiresolution elastic matching. Comput. Vtston Graph. Image Process. 46, 1-21.]] Google ScholarDigital Library
- BAIRn, H. S. 1984. Model-based image matching using location. In An ACM Distinguished Dissertation. MIT Press, Cambridge, Mass.]] Google ScholarDigital Library
- BALLARD, D. H. 1981. Generalizing the Hough Transform to detect arbitrary shapes. Part. Recog. 13, 2, 111-122.]]Google ScholarCross Ref
- BARNARD, S. T., AND FISCHLER, M. A. 1982. Computational stereo. ACM Comput. Surv. 14, 4 (Dec.), 553 572.]] Google ScholarDigital Library
- BARNEA, D. I., AND SmVERMAN, H. F. 1972. A class of algorithms for fast digital registration. IEEE Trans. Comput. C-21, 179-186.]]Google ScholarDigital Library
- BARROW, H. G., TENENBAUM, J. M., BOLLES, R. C., AND WOLF, a. C. 1977. Parametric correspondence and chamfer matching: Two new techtuques for image matching. In Proceedings' of the International Joint Conference ill Artt/)'clal Intelhgence. pp. 659-663.]]Google Scholar
- BERGSTt~0M, M.,'ETHIUS, B. J., ERIKSSON, L., GREITZ, T., RIBBE, T., AND WI~EN, L. 1981. Head fixation device for reproducible position alignment in transmission CT and positron emission tomography. J. Comput. Assisted Tomogr. 5, (Feb.), 136-141.]]Google ScholarCross Ref
- BERNSTEIN, R. 1976. Digital image processing of Earth observation sensor data. IBM J. Res. Devel. 20, (Jan.), 40-67.]]Google ScholarDigital Library
- BERNSTEIN, R., AND SILVERMAN. H. 1971. Digital techniques for Earth resource image data processing. In Proceedings of the American Institute of Aeronauttcs and Astronauttcs 8th Annual Meeting, vol. 21. AIAA.]]Google ScholarCross Ref
- BtESZK, J. A., AND FRAM, I. 1987. Automatic elastic image registration. In Proceedings of Computers in Cardiology (Leuven, Belgium, Sept.). pp. 3-5.]]Google Scholar
- BOHM, C., AND GREITZ, T. 1989. The construction of a functional brain atlas--Elimination of bias from anatomical varmtions at PET by reforming 3-D data into a standardized anatomy. In Visualization of Brain Functtons, D. Ottoson and W l~o~t~r~6~ Eet~. Wonlaer C~ren In{~ernational Symposium Series, vol. 53. pp. 137-140.]]Google Scholar
- BOHM, C., GREITZ, T., KINGSLEY, D., BERGGREN, B. M., AND OLSSON, L. 1983. Adjustable computerized stereotaxic brain atlas for transmission and emission tomography. Amer. J. Neuroradtol. 4, (Mar.), 731 733.]]Google Scholar
- BRESLER, Y., AND MERttAV, S. J. 1987. Recursive image registratiofi with application to motion estimation. IEEE 7kans. Acoust. Speech Stgnal Proc. ASSP-35, I (Jan.), 70 85.]]Google Scholar
- BRan', C. 1981. Optimal registrations of deformed images. Ph.D. Dissertation, Univ. of Pennsylvania.]]Google Scholar
- BUNKE, H., AND SANFELIU, A., EDS. 1990. Syntacttc and Structural Pattern Recognition, Theory and Apphcations. World Ckientific, Teaneck, N.J.]]Google Scholar
- BURR, D. J. 1981. A dynamic model for image registration. Comput. Graphics Image Proc. 15, 102 112.]]Google ScholarCross Ref
- CLOUGH, R. W., AND TOCHER, J. L. 1965. Finite element stiffness matrices for analysis of plates in bending. In Proceeclings of the Conference on Matrix Methods tn Structural Mechanics (Wright-Patterson A.F.B., Ohio). pp. 515-545.]]Google Scholar
- DANN, R., HOFORD, J., KOVACIC, S., REIVICH, M., AND BAJCSY, R. 1989. Evaluation of elastic matching system for anatomic (CT, MR) and functional (PET) cerebral images, J. Comput. Assi,sted Tomogr. 13, (July/Aug.), 603-611.]]Google Scholar
- DAVIS, L. S. 1982. Hierarchical Generalized Hough Transform and line segment based Generalized Hough Transforms. F'att. Recog. 15, 277-285.]]Google ScholarCross Ref
- DE CASTRO, E., AND MORANDI, C. 1987. Registration of translated and rotated images using finite Fourier Transfi)rms. IEEE Trans. Patt. Anal. Machine Intell. PAMI-9 , 5 (Sept.), 700-703.]] Google ScholarDigital Library
- DEGUCHI, K., 1986. R%qstration techniques for partially covered image sequence. In Proceedlngs of the 8th Inter, e, tional Conference on Pattern Recognition (Paris, Oct.). IEEE, New York, pp. 1186 1189.]]Google Scholar
- DENGLER, J. 1986. Local motion estimation with the dynamic pyramid. In The 8th Internattonal Conference on Pattern Recognition (Paris). pp. 1289-1292.]]Google ScholarCross Ref
- DHOND, U. R., AND AGGAP~WAL, J. K. 1989. Structure from stereo A rewew. IEEE Trans. Syst. Man Cybernetics 19, 6 (Nov./Dec.), 1489 1510.]]Google ScholarCross Ref
- DUDA, R. O., AND HART, P. E. 1973. Pattern Classtfication and Scene Analysis. John Wiley & Sons, New York.]]Google Scholar
- EvANs, A. C., BELL, C., MARRETT, S., THOMPSON, C. J., AND HAKIM, A. 1988. Anatomical-functional correlation using an adjustable MRI- based region of interest atlas with positron emission tomography. J. Cerebral Blood Flow Metabol. 8, 513 530.]]Google ScholarCross Ref
- FAUGERAS, O., AND PRICE, K. 1981. Semantic deaeriptlon of aa~ia} images usln~ s~ochas~le labeling. IEEE Trans. Patt. Anal. Machine Intell. PAMI-3, (Nov.), 638-642.]]Google Scholar
- FLUSSER, J. 1992. An adaptive method for image registration. Patt. Reeog. 25, 1, 45-54.]] Google ScholarDigital Library
- FORSYTHE, G. E., MALCOLM, M. A., AND MOLER, C. B. 1977. Computer Methods for Mathematical Computattons. Prentice-Hall, Engiewood Cliffs, N.J.]] Google ScholarDigital Library
- Fox, P. T., PERLMU'rrER, J. S., AND RAICHLE, M. E. 1985. Stereotactic method of anatomicallocalization for positron emission tomography. J. Comput. Asststed Tomogr. 9, 141 153.]]Google ScholarCross Ref
- Fr~NKE, R. 1979. A critical comparison of some methods for interpolation of scattered data. Tech. Rep. NPS-53-79-003, Naval Postgraduate School.]]Google Scholar
- Fu, K. S., ANn ICKIKAWA, T. 1982. Special Cornputer Architectures for Pattern Processing. CRC Press, Boca Raton, Fla.]] Google ScholarDigital Library
- GERLOT, P., AND BIZAIS, Y. 1987. Image registration: A review and a strategy for medical applications. In Proceedings of the lOth Internattonal Conference on Informatton Processing in Medical Imaging (Utretcht, Netherlands). pp. 81-89.]]Google Scholar
- GMUR, E., AND BUNKU, H. 1990. 3-D object recognition based on subgraph matching in polynomial time. In Structural Pattern Analysts. World Scientific, Teaneck, N.J.]]Google Scholar
- GONZALEZ, R. C., AND WfNTZ, P. 1977. Digital Image Processing. Addison-Wesley, Reading, Mass.]] Google ScholarDigital Library
- GOSHTASBY, h. 1988. Image registration by local approximation. Image Vision Comput. 6, 4 (Nov.), 255 261.]] Google ScholarDigital Library
- GOSHTASBY, A. 1987. Piecewise cubic mapping functions for image registration. Part. Recog. 20, 5, 525-533.]] Google ScholarDigital Library
- GOSHTASBY, A. 1986. Piecewise linear mapping functions for image registration. Patt. Recog. 19, 6, 459 466]] Google ScholarDigital Library
- GOSHTASBY, A. 1985. Template matching in rotated images. IEEE Trans. Part. Anal. Machine Intell. 7, 3 (May), 338-344.]]Google Scholar
- GOSHTASBY, A., AND STOCKMAN, G. C. 1985. Point pattern matching using convex hull edges. IEEE Trans. Syst. Man Cybernetics SMC-15, 5 (Sept./Oct.t, 631 637.]]Google ScholarCross Ref
- GOSHTASBY, A.. STOCKMAN. G. C., AND PAGE, C. V. 1986. A region-based approach to digital image registration with subpixel accuracy. IEEE Trans. Geoscz. Remote Sensing 24, 3, 390-399]]Google ScholarCross Ref
- C~UILLOLIX. Y I.E 1.Q86 A matching algorithm for horizontal motion, application to tracking, in Proceedings of the 8th International IEEE Conference on Pattern Recognttion (Paris, Oct.). IEEE, New York, pp. 1190-1192.]]Google Scholar
- HALL, E. L. 1979 Computer Image Processtng and Recognitlo,. Academic Press, New York.]]Google Scholar
- HARALICK, R. M. 1979. Automatic remote sensor image registration. In Toptcs zn Applied Phystcs. Vot. 11, Digital Picture Analysts, A. Rosenfeld, Ed. Springer-Verlag, New York, pp. 5-63.]]Google Scholar
- HERBIN, M., VENOT, A., DEVAUX, J. Y., WALTER, E., LEBRUCHEC, J. F., DUBERTRET, L., AND ROUCAY- ROL, J. C. 1989. Automated registration of dissimilar images: Application to medical imagery. Comput. Vision Graph Image Process. 47, 77 88.]] Google ScholarDigital Library
- HORN, B. K. P. 1989.Robot V~sion. MIT Press, Cambridge, Mass.]]Google Scholar
- HUMMEL, R., AND ZUCKER, S 1983. On the foundations of relaxation labeling processes. IEEE Trans. Patt. Anal. Machtne Intell. 5, (May), 267 287.]]Google ScholarDigital Library
- JENSEN, J. R. 1986. Introductory Dzgltal Image Processing, A Remote Sensing Perspecttve. Prentice-Hall, Englewood Cliffs, N.J]] Google ScholarDigital Library
- KANAL, L. N., LAMBIRD, B. A., LAVINE, D., AND STOCKMAN, G. C. 1981. Digital registration of images from similar and dissimilar sensors. In Proceedtngs of the International Conference on Cybernetzcs and Society. pp. 347-351.]]Google Scholar
- KATURI, R., AND JAIN, R. C 1991. Computer V~ston: Principles. IEEE Computer Society Press, Los Alamitos, Calif.]]Google Scholar
- KIREMIDJIAN, G. 1987. Issues in image registration. In IEEE Procee&ngs of SPIE: Image Understandtng and the Man-Machine Interface, vol. 758. IEEE, New York, pp 80-87]]Google Scholar
- KUGHN, C. D., AND HINES, D. C. 1975. The phase correlation image alignment method. In Proceedings of the IEEE 1975 International Conference on Cybernettcs and Society (Sept.). IEEE, New York, pp. 163-165.]]Google Scholar
- KUHL, F. P., AND GIARDINA, C. R. 1982. Elliptic Fourier features of a closed contour. Comput. Graph. Image Process. 18,236 258.]]Google ScholarCross Ref
- LEE, D. J., KRILE, T. F., AND MITRA, S. 1987. Digital registration techniques for sequential Fundus images. In IEEE Procee&ngs of SPIE: Appltcattons of Digital Image Processing X, vo}. 829. IEEE, New York, pp. 293 300.]]Google Scholar
- MAGHSOODI, R., AND REZAIE, B. 1987. Image registration using a fast adaptive algorithm. In IEEE Proceedmgs of SPIE: Methods of Handling and Processing Imagery, vol. 757. IEEE, New York, pp. 58-63.]]Google ScholarCross Ref
- MAGUIRE, G. Q., JR., Noz, M. E., LEE, E. M., AND SHIMPF, J. H. 1985. Correlation methods for tomographic images using two and three dimensional techniques. In Proceedzngs of the 9th Conference of Inl%rraat~on Processing ~n Medical Imaging (Washington, D.C., June 10-14). pp 266-279.]]Google Scholar
- MAGUIRE, G. Q., JR., NOZ, M. E., AND RUSINEK, H. 1990. Software tools to standardize and automate the correlation of images with and between diagnostic modalities. IEEE Comput. Graph. Appl.]]Google Scholar
- MAITRE, H., AND Wu, Y. 1987. hnproving dynamic programming to solve image registration. Part. Recog. 20, 4, 443-462.]] Google ScholarDigital Library
- MEDIONI, G., AND NEVATIA, R. 1984. Matching images using linear features. IEEE Trans. Patt. Anal. Machine Intell. PAMI-6, 675-685.]]Google ScholarDigital Library
- MERICKEL, M. 1988. 3D reconstruction: The registration problem. Comput. Vision Graph. Image Process. 42, 2, 206-219.]] Google ScholarDigital Library
- MILIOS, E. E. 1989. Shape matching using curvature processes. Comput. Vision Graph. Image Process. 47, 203-226.]] Google ScholarDigital Library
- MITICHE, A., AND AGGARWAL, J. K. 1983. Contour registration by shape-specific points fbr shape matching. Comput. Vision Graph. Image Process. 22,296-408.]]Google ScholarCross Ref
- MOHR, R., PAVLIDIS, T., AND SANFELm, A. 1990. Structural Pattern Analysis. World Scientific, Teaneck, N.J.]]Google Scholar
- MOr~VEC, H. 1981. Rover visual obstacle avoidance. In Proceedings of the 7th International Conference on Arttficial Intelligence (Vancouver, B.C., Canada, Aug.). pp. 785-790.]]Google Scholar
- MORT, M. S., AND SRINATH, M. n. 1988. Maximum likelihood image registration with subpixel accuracy. In IEEE Proceedings of SPIE: Applications of Digital Image Processing, vol. 974. IEEE, New York, pp. 38-43.]]Google ScholarCross Ref
- NACK, M. L. 1977. Rectification and registration of digital images and the effect of cloud detection. In Proceedings of Machine Processing' of Remotely Sensed Data. pp. 12-23.]]Google Scholar
- NAHMIAS, C., AND GARNETT, E. S. 1986. Correlation between CT, NMR and PT findings in the brain. NATO ASI SerGes. Vol. F19. Pictorial Informatmn Systems m Me&cme, K. H. Hohne, Ed. Springer-Verlag, Berlin, pp. 507 514.]] Google ScholarDigital Library
- Noz, M. E., AND MAC;UmE, G. Q., JR. 1988. QSH: A minimal but highly portable image display and processing Toolkit. Comput. Methods Program. Biomed. 27, 11 (Nov. 1988), 229 240.]]Google Scholar
- OHTA, Y., TAKANO, K., AND IKEDA, K. 1987. A highspeed stereo matching system based on dynamic programming. In Proceedings of the International Conference in Computer Vision (London, England). pp. 335-342.]]Google Scholar
- PAAR, G., AND KROPATSCH, W. G. 1990. Hierarchical cooperation between numerical and symbolic image representations. In Structural Pattern Analysts. World Scientific, Teaneck, N.J.]]Google Scholar
- PARIZEAU, M., AND PLAMONDON, R. 1990. A comparative analysis of regional correlation, dynamie time warping, and skeletal tree matching for signature verification. IEEE Trans. Patt. Anal. Machine Intell. 12, 7 (July), 710 717.]] Google ScholarDigital Library
- PAVLiDIS, T. 1978. Survey: A review of algorithms for shape analysis. Comput. Graph. Image Process. 7, 243 258.]]Google ScholarCross Ref
- PRATT, W. K. 1978. Digital Image Processing. John Wiley & Sons, New York.]] Google ScholarDigital Library
- PEL~, E. ET AL. 1987. Feature-based registration of retinal images. IEEE Trans. Med. Imaging MI-6, 3 (Sept.), 272-278.]]Google ScholarCross Ref
- PELIZARRI, C. A., CHEN, G. T. Y., SPELBRING, D. R., WEICHSELBAUM, R. R., AND CHEN. C. T. 1989. Accurate three-dimensional registration of CT, PET and/or MR images of the brain. J. Cornput. Assisted Tomogr. 13, (Jan./Feb.), 20 26.]]Google ScholarCross Ref
- PRmE, K. E. 1985. Relaxation matching techniques A comparison. IEEE Trans. Patt. Anal. Machine Intell. 7, 5 (Sept.), 617 623.]]Google Scholar
- RANADE, S., AND ROSENFELD, A. 1980. Point pattern matching by relaxation. Patt. Recog. 12, 269-275.]]Google ScholarCross Ref
- RATIB, O., BIDAUT, L., SCtJELBERT, H. R., AND PHELPS, M. E. 1988. A new technique for elastic registration of tomographic images. In IEEE Proceedings of SPIE: Medical Imaging H, vol. 914. IEEE, New York, pp. 452-455.]]Google ScholarCross Ref
- i~OSENFELD, A., AND KAK, A. C. 1982. Digital P~cture Processing. Vol. I and II. Academic Press, Orlando, Fla.]] Google ScholarDigital Library
- SANFELm, A. 1990. Matching complex structures: The cyclic-tree representation scheme. In Structural Pattern Analysis. World Scientific, Teaneck, N.J.]]Google Scholar
- SHAPIRO, L. G., AND HARArICK, R. M. 1990. Matching relational structures using discrete relaxation. In Syntactic and Structural Pattern Recognition, Theory and Apphcattons. World cientific, Teaneck, N.J.]]Google Scholar
- SINGH,-M., FREI, W., SHIBATA, T., HUTH, G. C., AND TELFER, N. E. 1979. A digital technique for accurate change detection in nuclear medical images With application to myocardial perfusion studies using Thallium-201. IEEE Trans. Nuclear Sci. NS-26, 1 (Feb.).]]Google Scholar
- SLAME, C. C., ED. 1980. Manual of Photogrammetry, 4th ed. American Society of PhotogTammetry, Falls Church, Va.]]Google Scholar
- SOLINA, F., AND BAJCS~, R. 1990. Recovery of parametric models from range images: The case for superquadrics with global deformations. IEEE Trans. Patt. Anal. Machine Intell. 12.2 (Feb.), 131-147.]] Google ScholarDigital Library
- STOCKMAN, G. C., KOPSTEIN, S., AND BENETT, S. 1982. Matching images to models for registration and object detection via clustering. IEEE Trans. Patt. Anal. Machine Intell. 4, 229-241.]]Google ScholarDigital Library
- STYTZ, M. R., FRIEDER, G, AND FRIEDER, O. 1991. Three-dimensional medical imaging: Algorithms and computer systems. ACM Comput. Surv. 23, 4 (Dec.), 42} 424.]] Google ScholarDigital Library
- SUETENS, P., FUA, P., HANSON, A. J. 1992. Computational strategies for object recognition. ACM Comput. Surv. 24, 1 (Mar.), 5-62.]] Google ScholarDigital Library
- SVEDLOW, M., McGILLEM, C. D., AND ANUTA, P. E. 1976. Experimental examination of mmilarity measures and preprocessing methods used for image registration. In The Symposium on Machine Processing of Remotely Sensed Data (Westville, Ind., June). pp. 4A-9.]]Google Scholar
- TERZOPOULOS, D., WITKIN, A., AND KASS, M. 1987. Energy constraints on deformable models: Recovering shape and non-rigid motion. In Proceedtngs AAAI 87 (July), voI. 2. AAAI, Menlo Park, Calif., pp. 755 760.]]Google Scholar
- THOMAS, I. L., BENNING, V. M., AND CHING, N. P. 1986. Classzficatzon of Remotely Sensed Images. Adam Hilger, Bristol, England.]]Google Scholar
- TON, J., AND JAIN, A. K. 1989. Registering Landsat images by point matching. IEEE Trans. Geosc~. Remote Sensing 27, (Sept.), 642 651.]]Google ScholarCross Ref
- VAN DEN ELSEN, P A., POL, E.-I. D., AND VIERGEVER, M. A. 1992. Medical image matching' A review with classification. IEEE Eng. Med. Brol , in press.]]Google Scholar
- VAN WIE, P, AND STEIN, M. 1977 A LANDSAT digStal image rectification system. IEEE Trans. Geoscr. Electr. GE-15, 3 (July), 130 137.]]Google ScholarCross Ref
- VENOT, A., LEBRUCHEC, J. F., AND ROUCAYROL, J. C. 1984. A new class of similarity measures for robust image registration. Comput. Vision Graph. Image Process. 28, 176-184.]] Google ScholarDigital Library
- W~DROW, B. 1973. The rubber mask technique, Part I and II. Putt Recog. 5, 3, 175-211.]]Google Scholar
- WOLBERG, G. 1990. Digital Image Warpzng IEEE Computer Society Press, Los Alamitos, Calif.]] Google ScholarDigital Library
- WOLBERG, G., AND BOULT, T. 1989. Separable image warping with spatial lookup tables. In ACM SIGGRAPH '89 (Boston, July/Aug.). ACM, New York, pp. 369 377.]] Google ScholarDigital Library
- WONG, A. K. C., AND YOU, M. 1985. Entropy and distance of random graphs with application to structural pattern recogmtion. IEEE Trans'. Port Anal. Machine Intell. PAMI-7, 5 (Sept.), 599 609.]]Google ScholarDigital Library
- WONG, R. Y. 1977. Sensor transformations. IEEE Trans. Syst. Man CyberTzetzes SMC-7, 12 (Dec.), 836-840.]]Google ScholarCross Ref
Index Terms
- A survey of image registration techniques
Recommendations
Image registration by local histogram matching
We previously presented an image registration method, referred to hierarchical attribute matching mechanism for elastic registration (HAMMER), which demonstrated relatively high accuracy in inter-subject registration of MR brain images. However, the ...
Image Registration by Template Matching Using Normalized Cross-Correlation
ACT '09: Proceedings of the 2009 International Conference on Advances in Computing, Control, and Telecommunication TechnologiesTemplate matching is used for many applications in image processing. Cross Correlation is the basic statistical approach to image registration. It is used for template matching or pattern recognition. Template can be considered a sub-image from the ...
The role of intensity standardization in medical image registration
Acquisition-to-acquisition signal intensity variations (non-standardness) are inherent in MR images. Standardization is a post processing method for correcting inter-subject intensity variations through transforming all images from the given image gray ...
Comments