Skip to main content
SpringerLink
Log in

Development and validation of the MOOC success scale (MOOC-SS)

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

The objective of this study is to develop and validate the MOOC Success Scale (MOOC-SS) in the Malaysian context. Further, this study ascertains the influence of MOOC success factors on learner satisfaction. Based on an elaborate literature study, six factors related to MOOC success were derived: (1) system quality, (2) information quality, (3) service quality, (4) attitude, (5) course quality, and (6) satisfaction. The data were collected from 1000 undergraduate students from 5 public universities in Malaysia with a return of 622 responses. The instrument was tested in several ways: (a) review the literature, (b) formulating hypotheses (c) items generation, (d) expert opinion, and (e) pilot test. Thereafter, the scale’s reliability and validity were calculated, which appeared to be excellent. The results of the principal component analysis (PCA) empirically confirmed that the MOOC success scale with six factors and 33 items is strong enough to recommend its use in MOOC settings. Further, the finding of the predictive validity indicated that system quality, attitude, and course quality appeared to predict satisfaction toward MOOC. The instrument can be used by academicians, practitioners, and policymakers to monitor the MOOC success factors. Further research would be needed to generalize this research to other populations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Ajoye, M. B., & Nwagwu, W. E. (2014). Information systems user satisfaction: A survey of the postgraduate school portal, University of Ibadan, Nigeria. Library Philosophy and Practice, paper 1192.

  • Albelbisi, N. A. (2019). The role of quality factors in supporting self-regulated learning (SRL) skills in MOOC environment. Education and Information Technologies, 1-18.

  • Albelbisi, N. A., & Yusop, F. D. (2019). Factors influencing learners’ self–regulated learning skills in a massive open online course (MOOC) environment. Turkish Online Journal of Distance Education, 20(3), 1–16.

    Article  Google Scholar 

  • Albelbisi, N., Yusop, F. D., & Salleh, U. K. M. (2018). Mapping the factors influencing success of massive open online courses (MOOC) in higher education. EURASIA Journal of Mathematics, Science and Technology Education, 14(7), 2995–3012.

    Article  Google Scholar 

  • Alraimi, K. M., Zo, H., & Ciganek, A. P. (2015). Understanding the MOOCs continuance: The role of openness and reputation. Computers & Education, 80, 28–38. https://doi.org/10.1016/j.compedu.2014.08.006.

    Article  Google Scholar 

  • Alsabawy, A. Y., Cater-Steel, A., & Soar, J. (2012). A model to measure e-learning systems success. In Measuring organizational information systems success: New technologies and practices (pp. 293-317). IGI global.

  • Aparicio, M., & Bacao, F. (2013, July). E-learning concept trends. In Proceedings of the 2013 International Conference on Information Systems and Design of Communication (pp. 81-86). ACM.

  • Azevedo, J., & Marques, M. M. (2017). MOOC success factors: Proposal of an analysis framework. Journal of Information Technology Education: Innovations in Practice, 16(233), 251.

    Google Scholar 

  • Bagozzi, R. P., & Yi, Y. (2012). Specification, evaluation, and interpretation of structural equation models. Journal of the Academy of Marketing Science, 40(1), 8–34.

    Article  Google Scholar 

  • Bayne, S., & Ross, J. (2014). The pedagogy of the Massive Open Online Course: the UK view (pp. 1–76). York: The Higher Education Academy.

  • Chang, R. I., Hung, Y. H., & Lin, C. F. (2015). Survey of learning experiences and influence of learning style preferences on user intentions regarding MOOCs. British Journal of Educational Technology, 46(3), 528–541.

    Article  Google Scholar 

  • Creelman, A., Ehlers, U.-D., & Ossiannilsson, E. (2014). Perspectives on MOOC quality – An account of the EFQUEL MOOC quality project. International Journal for Innovation and Quality and in Learning (INNOQUAL), September (3), 78-87

  • DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30.

    Article  Google Scholar 

  • Dong, T.-P., Cheng, N.-C., & Wu, Y.-C. J. (2014). A study of the social networking website service in digital content industries: The Facebook case in Taiwan. Computers in Human Behavior, 30, 708–714.

    Article  Google Scholar 

  • Downes, S. (2016). The quality of massive open online courses [personal page post]. Retrieved from http://www.downes.ca/post/66145

  • Drake, J. R., O'Hara, M., & Seeman, E. (2015). Five principles for MOOC design: With a case study. Journal of Information Technology Education: Innovations in Practice, 14, 125–143 Retrieved from https://www.informingscience.org/Publications/2250.

    Google Scholar 

  • Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.

    Article  Google Scholar 

  • Gamage, D., Fernando, S., & Perera, I. (2015). Quality of MOOCs: A review of literature on effectiveness and quality aspects. In Ubi-Media Computing (UMEDIA), 2015 8th International Conference on (pp. 224-229). IEEE. https://doi.org/10.1109/UMEDIA.2015.7297459.

  • Gameel, B. G. (2017). Learner satisfaction with massive open online courses. American Journal of Distance Education, 31(2), 98–111.

    Article  MathSciNet  Google Scholar 

  • George, D., & Mallery, P. (2012). IBM SPSS statistics 19 step by step: A simple guide and reference (12th ed). Boston: Pearson.

    Google Scholar 

  • Gutiérrez-Santiuste, E., Gámiz-Sánchez, V. M., & Gutiérrez-Pérez, J. (2015). MOOC & B-learning: Students' barriers and satisfaction in formal and non-formal learning environments. Journal of Interactive Online Learning, 13(3).

  • Hair, J. F., Black, W. C., Babin, B., Anderson, R. E., & Ronald, L. T. (2006a). Multivariate data analysis (5th ed.). Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Hair, J., Black, W., Babin, B., Anderson, R., & Tatham, R. (2006b). Multivariate Data Analysis (6th ed.). Upper Saddle River: Pearson Prentice Hall.

    Google Scholar 

  • Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modeling (PLS-SEM). Thousand Oaks: SAGE.

    MATH  Google Scholar 

  • Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45–58. https://doi.org/10.1016/j.edurev.2014.05.001.

    Article  Google Scholar 

  • Jansen, D., Rosewell, J., & Kear, K. (2017). Quality frameworks for MOOCs. In Open Education: from OERs to MOOCs (pp. 261–281). Springer, Berlin.

  • Kevan, J. M., Menchaca, M. P., & Hoffman, E. S. (2016). Designing MOOCs for success: A student motivation-oriented framework. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 274-278). ACM.

  • King, N. (2012). Doing template analysis. Qualitative organizational research: Core methods and current challenges, 426. https://doi.org/10.4135/9781526435620.n24.

  • Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., & Hatala, M. (2015). What public media reveals about MOOCs: A systematic analysis of news reports. British Journal of Educational Technology, 46(3), 510–527. https://doi.org/10.1111/bjet.12277.

    Article  Google Scholar 

  • Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575.

    Article  Google Scholar 

  • Liaw, S. S., & Huang, H. M. (2013). Perceived satisfaction, perceived usefulness and interactive learning environments as predictors to self-regulation in e-learning environments. Computers & Education, 60(1), 14–24.

  • Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning, 14(3), 202–227. https://doi.org/10.19173/irrodl.v14i3.1455.

    Article  Google Scholar 

  • Lwoga, E. T. (2014). Critical success factors for the adoption of web-based learning management systems in Tanzania. International Journal of Education and Development using Information and Communication Technology, 10(1), 4–21.

    Google Scholar 

  • Manalo, J. M. A. (2014). An evaluation of participants’ levels of satisfaction and perceived learning regarding the MOOC in@ RAL platform. Malaysian Journal of Distance Education, 16(1), 101–121.

    Google Scholar 

  • Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77–83.

    Article  Google Scholar 

  • Mohammadi, H. (2015). Investigating users’ perspectives on e-learning: An integration of TAM and IS success model. Computers in Human Behavior, 45, 359–374. https://doi.org/10.1016/j.chb.2014.07.044.

    Article  Google Scholar 

  • Nagashima, T. (2014). What makes open education thrive? Examination of factors contributing the success of open education initiatives. International Journal for Innovation and Quality and in Learning (INNOQUAL), September (3), 10-21.

  • Newman, I., Newman, D., & Newman, C. (2011). Writing research articles using mixed methods: methodological considerations to help you get published. In The handbook of scholarly writing and publishing (pp. 191–208). San Francisco: Jossey-Bass.

  • Nunnally, J., & Bernstein, I. H. (1991). Psychometric theory. New York: McGraw-Hill.

    Google Scholar 

  • Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management systems: Adoption of hexagonal e-learning assessment model in higher education. Transforming Government: People, Process and Policy, 3(2), 111–130.

    Article  Google Scholar 

  • Parr, C. (2013). MOOCs completion rates “below 7%”, Times Higher Education. Retrieved from http://www.timeshighereducation.co.uk/news/moocs-completion-ratesbelow-7/2003710.

  • Peugh, J. L., & Enders, C. K. (2005). Using the SPSS mixed procedure to fit cross-sectional and longitudinal multilevel models. Educational and Psychological Measurement, 65, 714–741.

    Article  MathSciNet  Google Scholar 

  • Poy, R., & Gonzales-Aguilar, A. (2014). MOOC success factors: Some critical considerations. RISTI-Revista Iberica de Sistemas e Tecnologias de Informacao, 1, 105–118. https://doi.org/10.4304/risti.e1.105-118.

    Article  Google Scholar 

  • Rai, L., & Chunrao, D. (2016). Influencing factors of success and failure in MOOC and general analysis of learner behavior. International Journal of Information and Education Technology, 6(4), 262–268.

    Article  Google Scholar 

  • Rhema, A., & Miliszewska, I. (2014). Analysis of student attitudes towards e-learning: The case of engineering students in Libya. Issues in informing science and information Technology, 11(1), 169–190.

    Article  Google Scholar 

  • Rivard, R. (2013). No-bid MOOCs. Inside Higher Ed. Retrieved from http://www.insidehighered.com/news/2013/07/17/MOOCs-spread-quickly-aided-no-bid-dealspublic-universities

  • Rosewell, J., & Jansen, D. (2014). The OpenupEd quality label: Benchmarks for MOOCs. INNOQUAL: The International Journal for Innovation and Quality in Learning, 2(3), 88–100.

    Google Scholar 

  • Saadatdoost, R., Sim, A. T. H., Jafarkarimi, H., & Mei Hee, J. (2015). Exploring MOOC from education and information systems perspectives: A short literature review. Educational Review, 67(4), 505–518.

    Article  Google Scholar 

  • Samarasinghe, S. M. (2012). E-learning systems success in an organisational context: A thesis presented in partial fulfilment of the requirements for the degree of doctor of philosophy in management information Systems at Massey University, Palmerston North, New Zealand (Doctoral dissertation, Massey University). Retrieved from http://hdl.handle.net/10179/4726

  • Sang, S., Lee, J. D., & Lee, J. (2010). E-government adoption in Cambodia: A partial least squares approach. Transforming Government: People, Process and Policy, 4(2), 138–157.

    Article  Google Scholar 

  • Sarstedt, M., & Mooi, E. A. (2014). A concise guide to market research. The process, data, and methods using IBM SPSS statistics. Berlin: Springer.

    Google Scholar 

  • Soffer, T., & Cohen, A. (2015). Implementation of Tel Aviv University MOOCs in academic curriculum: A pilot study. The International Review of Research in Open and Distributed Learning, 16(1).

  • Sun, P., Tasi, R. J., Finger, G., & Chen, Y. (2008). What drives a successful e- learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183–1202.

    Article  Google Scholar 

  • Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics., 6th edition. Boston: Pearson Education.

    Google Scholar 

  • Walker, L., & Loch, B. (2014). Academics’ perceptions on the quality of MOOCs: An empirical study. INNOQUAL-International Journal for Innovation and Quality in Learning, 2(3), 53–63.

    Google Scholar 

  • Yakubu, M. N., & Dasuki, S. (2018). Assessing eLearning systems success in Nigeria: An application of the DeLone and McLean information systems success model. Journal of Information Technology Education: Research, 17, 183–203.

    Article  Google Scholar 

  • Yang, H. H., & Su, C. H. (2017). Learner behaviour in a MOOC practice-oriented course: In empirical study integrating TAM and TPB. The International Review of Research in Open and Distributed Learning, 18(5).

  • Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development, 65(5), 1195–1214.

    Article  Google Scholar 

  • Yepes-Baldó, M., Romeo, M., Martín, C., García, M. Á., Monzó, G., & Besolí, A. (2016). Quality indicators: Developing “MOOCs” in the european higher education area. Educational Media International, 53(3), 184–197.

    Article  Google Scholar 

  • Yousef, A. M. F., Chatti, M. A., Schroeder, U., & Wosnitza, M. (2014). What drives a successful MOOC? An empirical examination of criteria to assure design quality of MOOCs. In Advanced Learning Technologies (ICALT), 2014 IEEE 14th International Conference on (pp. 44–48). IEEE. https://doi.org/10.1109/ICALT.2014.23.

  • Zhao, H. (2016). Factors influencing self-regulation in E-learning 2.0: confirmatory factor model. Canadian Journal of Learning and Technology, 42(2). https://doi.org/10.21432/T2C33K.

Download references

Availability of data and material

Please contact the author for the datasets.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for profit sectors.

Author information

Authors and Affiliations

  1. Department of Curriculum and Instructional Technology, Faculty of Education, University of Malaya, 50603, Kuala Lumpur, Malaysia

    Nour Awni Albelbisi

Authors
  1. Nour Awni Albelbisi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nour Awni Albelbisi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Albelbisi, N.A. Development and validation of the MOOC success scale (MOOC-SS). Educ Inf Technol 25, 4535–4555 (2020). https://doi.org/10.1007/s10639-020-10186-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-020-10186-4

Keywords

Search

Navigation