Abstract
We present a highly scalable distributed media transcoding system that reduces the time required for batch transcoding of multimedia files into several output formats. To implement this system we propose a fully distributed architecture that leverages proven technologies to create a highly scalable and fault-tolerant platform. Also a new task-oriented parallel processing framework that improves on MapReduce is developed in order to express transcoding tasks as distributed processes and execute them on top of the distributed platform. Preliminary results show a significant reduction in time resources required to transcode large batches of media files with little effects on the quality of the output transcoded files.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Pantos, A.I.R., May, E.W.: HTTP Live Streaming - draft (September 2011) (expires: April 2, 2012)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51, 107–113 (2008)
Isard, M., Budiu, M., Yu, Y., Andrew, B., Fetterly, D.: Dryad: Distributed data-parallel programs from sequential building blocks. In: European Conference on Computer Systems, EuroSys (March 2007)
Sambe, Y., Watanabe, S., Yu, D., Nakamura, T., Wakamiya, N.: High-speed distributed video transcoding for multiple rates and formats. IEICE Transactions on Information and Systems 88(8), 1923–1931 (2005)
Dongmahn, S., Jongwoo, K., Inbum, J.: Load distribution algorithm based on transcoding time estimation for distributed transcoding servers. In: International Conference on Inforamtion Science and Applications (ICISA), pp. 1–8 (April 2010)
Yang, C., Chen, Y., Shen, Y.: The research on a p2p transcoding system based on distributed farming computing architecture. Knowledge Engineering and Software Engineering (KESE), 55–58 (December 2009)
Ravindra, G., Kumar, S., Chintada, S.: Distributed media transcoding using a p2p network of set top boxes. In: Consumer Communcications and Networking Conference (CCNC), pp. 1–2 (January 2009)
Deneke, T.: Scalable Distributed Video Transcoding Architecture, Master’s thesis. Åbo Akademi University (2011)
Redis key-value database, http://redis.io
Gluster file system architecture, tech. rep., http://download.gluster.com/pub/gluster/documentation/Gluster_Architecture.pdf
Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP 2009, pp. 261–276. ACM, New York (2009)
Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., Stoica, I.: Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In: Proceedings of the 5th European Conference on Computer Systems, EuroSys 2010, pp. 265–278. ACM, New York (2010)
Beaumont, O., Carter, L., Ferrante, J., Legrand, A., Marchal, L., Robert, Y.: Centralized versus distributed schedulers for bag-of-tasks applications. IEEE Transactions on Parallel and Distributed Systems (May 2008)
Ghatpande, A., Nakazato, H., Beaumont, O.: Scheduling of divisible loads on heterogeneous distributed systems. In: Ros, A. (ed.) Parallel and Distributed Computing, pp. 179–202. In-Tech (2010)
Raman, R., Livny, M., Solomon, M.: Matchmaking: Distributed resource management for high throughput computing. In: Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing, pp. 28–31 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sanson, H., Loyola, L., Pereira, D. (2012). Scalable Distributed Architecture for Media Transcoding. In: Xiang, Y., Stojmenovic, I., Apduhan, B.O., Wang, G., Nakano, K., Zomaya, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2012. Lecture Notes in Computer Science, vol 7439. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33078-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-33078-0_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33077-3
Online ISBN: 978-3-642-33078-0
eBook Packages: Computer ScienceComputer Science (R0)