James Hays
Alexei A. Efros
Abstract
What can you do with a million images? In this paper, we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless, but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks, we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data driven, requiring no annotations or labeling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of image completions and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches.
- Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. Interactive digital photomontage. ACM Trans. Graph., 23(3):294--302, 2004. Google Scholar
Digital Library
- Agrawal, A., Raskar, R., and Chellappa, R. What is the range of surface reconstructions from a gradient field? European Conference on Computer Vision (ECCV), May 2006. Google ScholarDigital Library
- Boykov, Y., Veksler, O., and Zabih, R. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 23(11):1222--1239, 2001. Google ScholarDigital Library
- Criminisi, A., Perez, P., and Toyama, K. Object removal by exemplar-based inpainting. CVPR, 02:721, 2003.Google Scholar
- Diakopoulos, N., Essa, I., and Jain, R. Content based image synthesis. Conference on Image and Video Retrieval (CIVR), 2004.Google ScholarCross Ref
- Drori, I., Cohen-Or, D., and Yeshurun, H. Fragment-based image completion. ACM Trans. Graph., 22(3):303--312, 2003. Google ScholarDigital Library
- Efros, A. A. and Freeman, W. T. Image quilting for texture synthesis and transfer. Proceedings of SIGGRAPH 2001, pages 341--346, August 2001. Google ScholarDigital Library
- Efros, A. A. and Leung, T. K. Texture synthesis by non-parametric sampling. ICCV, pages 1033--1038, Corfu, Greece, September 1999. Google ScholarDigital Library
- Hays, J. and Efros, A. A. Scene completion using millions of photographs. ACM Trans. Graph. (SIGGRAPH 2007), 26(3): 4, 2007. Google ScholarDigital Library
- Hays, J. and Efros, A. A. im2gps: estimating geographic information from a single image. CVPR, 2008.Google ScholarCross Ref
- Jia, J., Sun, J., Tang, C.-K., and Shum H.-Y Drag-and-drop pasting. ACM Trans. Graph., 25(3): 631--637, 2006. Google ScholarDigital Library
- Johnson, M., Brostow, G. J., Shotton, J.: Arandjelovik, O., Kwatra, V., and Cipolla, R. Semantic photo synthesis. Comput. Graph. Forum (Proa Eurographics), 25(3):407--413, 2006.Google Scholar
- King, D. The Commissar Vanishes. Henry Holt and Company, London, 1997.Google Scholar
- Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. Texture optimization for example-based synthesis. ACM Trans. Graph., 24: 795--802, 2005. Google ScholarDigital Library
- Kwatra, V., Schodl, A., Essa, I., Turk, G., and Bobick, A. Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graph., 22(3):277--286,2003. Google ScholarDigital Library
- Oliva, A. and Torralba, A. Building the gist of a scene: The role of global image features in recognition. Visual perception. Progr. Brain Res., 155:23--36, 2006.Google ScholarCross Ref
- Perez, P., Gangnet, M., and Blake, A. Poisson image editing. ACM Trans. Graph., 22(3):313--318, 2003. Google ScholarDigital Library
- Russell, B. C., Torralba, A., Murphy K. P., and Freeman, W. T. LabelMe: A database and web-based tool for image annotation. Technical Report, MIT, 2005, 2005.Google Scholar
- Snavely, N., Seitz, S. M., and Szeliskr R. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph., 25(3):835--846, 2006. Google ScholarDigital Library
- Sun, J., Yuan, L., Jia, J., and Shum H.-Y. Image completion with structure propagation. ACM Trans. Graph., 24(3):861--868, 2005. Google ScholarDigital Library
- Torralba, A., Fergus, R., and Freeman, W. T. Tiny images. Technical Report MIT-CSAIL-TR-2007-024, 2007Google Scholar
- Torralba, A., Murphy, K. P., Freeman, W. T., and Rubin, M. A. Context-based vision system for place and object recognition. ICCV, 2003. Google ScholarDigital Library
- Wertheimer, M. Laws of organization in perceptual forms (partial translation). In: W. Ellis, editor, A Sourcebook of Gestatt Psychology, pages 71--88. Harcourt Brace and Company, London, 1938.Google Scholar
- Wexler, Y., Shechtman, E., and Irani, M. Space-time video completion. CVPR. 01:120--127, 2004.Google Scholar
- Wilczkowiak, M., Brostow, G. J., Tordoff, B., and Cipolla, R. Hole filling through photomontage. BMVC, 492--501, July 2005.Google Scholar
Index Terms
- Scene completion using millions of photographs
Recommendations
Scene completion using millions of photographs
What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database ...
Scene Completion Using Millions of Photographs
Seminal Graphics Papers: Pushing the Boundaries, Volume 2What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database ...
Scene completion using millions of photographs
SIGGRAPH '07: ACM SIGGRAPH 2007 papersWhat can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database ...
Comments