Alexander Hornung
Abstract
This article presents a new method to animate photos of 2D characters using 3D motion capture data. Given a single image of a person or essentially human-like subject, our method transfers the motion of a 3D skeleton onto the subject's 2D shape in image space, generating the impression of a realistic movement. We present robust solutions to reconstruct a projective camera model and a 3D model pose which matches best to the given 2D image. Depending on the reconstructed view, a 2D shape template is selected which enables the proper handling of occlusions. After fitting the template to the character in the input image, it is deformed as-rigid-as-possible by taking the projected 3D motion data into account. Unlike previous work, our method thereby correctly handles projective shape distortion. It works for images from arbitrary views and requires only a small amount of user interaction. We present animations of a diverse set of human (and nonhuman) characters with different types of motions, such as walking, jumping, or dancing.
Supplemental Material
- Agarwal, A. and Triggs, B. 2006. Recovering 3d human pose from monocular images. IEEE Trans. Pattern Anal. Mach. Intel. 28, 1, 44--58.]] Google Scholar
Digital Library
- Alexa, M., Cohen-Or, D., and Levin, D. 2000. As-Rigid-As-Possible shape interpolation. In Proceedings of the SIGGRAPH Conference. 157--164.]] Google ScholarDigital Library
- Barrett, W. A. and Cheney, A. S. 2002. Object-Based image editing. In Proceedings of the SIGGRAPH Conference. 777--784.]] Google ScholarDigital Library
- Bregler, C., Loeb, L., Chuang, E., and Deshpande, H. 2002. Turning to the masters: Motion capturing cartoons. In Proceedings of the SIGGRAPH Conference. 399--407.]] Google ScholarDigital Library
- Chuang, Y.-Y., Goldman, D. B., Zheng, K. C., Curless, B., Salesin, D., and Szeliski, R. 2005. Animating pictures with stochastic motion textures. ACM Trans. Graph. 24, 3, 853--860.]] Google ScholarDigital Library
- CMU Graphics Lab Motion Capture Database. 2007. http://mocap.cs.cmu.edu/.]]Google Scholar
- Corrêa, W. T., Jensen, R. J., Thayer, C. E., and Finkelstein, A. 1998. Texture mapping for cel animation. In Proceedings of the SIGGRAPH Conference. 435--446.]] Google ScholarDigital Library
- Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-Based image completion. ACM Trans. Graph. 22, 3, 303--312.]] Google ScholarDigital Library
- Efros, A. A. and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In Proceedings of the 7th International Conference on Computer Vision. 1033--1038.]] Google ScholarDigital Library
- Gavrila, D. M. 1999. The visual analysis of human movement: A survey. Comput. Vision Image Understanding. 73, 1, 82--98.]] Google ScholarDigital Library
- Hartley, R. and Zisserman, A. 2003. Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press, New York.]] Google ScholarDigital Library
- Horn, B. 1987. Closed form solutions of absolute orientation using unit quaternions. J. Optical Soci. America 4, 4 (Apr.), 629--642.]]Google ScholarCross Ref
- Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-Rigid-As-Possible shape manipulation. ACM Trans. Graph. 24, 3, 1134--1141.]] Google ScholarDigital Library
- Jojic, N. and Frey, B. J. 2001. Learning flexible sprites in video layers. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 199--206.]]Google Scholar
- Kass, M., Witkin, A., and Terzopoulos, D. 1987. Snakes: Active contour models. Int. J. Comput. Vis. 321--331.]]Google Scholar
- Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. In Proceedings of the SIGGRAPH Conference. 491--500.]] Google ScholarDigital Library
- Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the SIGGRAPH Conference. 165--172.]] Google ScholarDigital Library
- Liu, C., Torralba, A. B., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph. 24, 3, 519--526.]] Google ScholarDigital Library
- Moeslund, T. B. and Granum, E. 2001. A survey of computer vision-based human motion capture. Comput. Vision Image Understanding. 81, 3, 231--268.]] Google ScholarDigital Library
- Mori, G., Berg, A., Efros, A., Eden, A., and Malik, J. 2004. Video based motion synthesis by splicing and morphing. Tech. Rep. UCB//CSD-04-1337, University of California, Berkeley. June.]]Google Scholar
- Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-Based modeling and photo editing. In Proceedings of the SIGGRAPH Conference. 433--442.]] Google ScholarDigital Library
- Ono, Y., Chen, B.-Y., and Nishita, T. 2004. 3D character model creation from cel animation. In IEEE Cyberworlds. 210--215.]] Google ScholarDigital Library
- Parameswaran, V. and Chellappa, R. 2004. View independent human body pose estimation from a single perspective image. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 16--22.]]Google Scholar
- Pavić, D., Schönefeld, V., and Kobbelt, L. 2006. Interactive image completion with perspective correction. Vis. Comput. 22, 9, 671--681.]] Google ScholarDigital Library
- Rademacher, P. 1999. View-Dependent geometry. In Proceedings of the SIGGRAPH Conference. 439--446.]] Google ScholarDigital Library
- Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Proceedings of the Symposium on Geometry Processing. 179--188.]] Google ScholarDigital Library
- Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Trans. Graph. 23, 3, 315--321.]] Google ScholarDigital Library
- Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Trans. Graph. 24, 3, 861--868.]] Google ScholarDigital Library
- Taylor, C. J. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1677--1684.]]Google ScholarCross Ref
- Thorne, M., Burke, D., and van de Panne, M. 2004. Motion doodles: An interface for sketching character motion. ACM Trans. Graph. 23, 3, 424--431.]] Google ScholarDigital Library
- Triggs, B., McLauchlan, P., Hartley, R., and Fitzgibbon, A. 2000. Bundle adjustment---A modern synthesis. In Vision Algorithms: Theory and Practice, B. Triggs et al., Eds. Lecture Notes in Computer Science. Springer Verlag, 298--375.]] Google ScholarDigital Library
Index Terms
- Character animation from 2D pictures and 3D motion data
Recommendations
Secondary Motion for Performed 2D Animation
UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and TechnologyWhen bringing animated characters to life, artists often augment the primary motion of a figure by adding secondary animation -- subtle movement of parts like hair, foliage or cloth that complements and emphasizes the primary motion. Traditionally, ...
Shape-aware skeletal deformation for 2D characters
This paper presents a skeleton-based method for deforming 2D characters. While previous skeleton-based methods drive the shape deformation by binding the skeleton to the shape, our method does so by propagating the skeleton transformations over the ...
Sketch-based skeleton-driven 2D animation and motion capture
Transactions on edutainment VIWe present a novel sketch-based 2D animation technique, which allows the user to produce 2D character animations efficiently. It consists of two parts, sketch-based skeleton-driven 2D animation production and 2D motion capture. The user inputs one image ...
Comments