Assessing the Relative Importance of an E-learning system’s Usability Design Characteristics Based on Students' Preferences
This study considers the interaction between an e-learning system, the Blackboard system, and the students who use it in Saudi Arabia. While previous .
- Pub. date: July 15, 2019
- Pages: 839-855
- 1190 Downloads
- 878 Views
- 25 Citations
This study considers the interaction between an e-learning system, the Blackboard system, and the students who use it in Saudi Arabia. While previous work exists, there is limited consideration of the assessment of the preferences of e-learning system usability variables based on students’ perspectives, especially in developing countries such as Saudi Arabia. This paper attempts to fill the gap by investigating the relative importance of the design criteria developed for e-learning system usability evaluation from the students’ perspective in Saudi tertiary education. Based on reviewed literature, a set of usability principles was developed that have had an influence in the students’ learning process and use of the e-learning system. The list includes system navigation, system learnability, visual design, information quality, instructional assessment and system interactivity. An exploratory study was carried out to identify the most important usability design characteristics from a student’s perspective and then evaluate the overall usability of the current e-learning system, based on this subset. A quantitative approach was adopted to weigh usability design characteristics, based on 181 learners’ perceptions. The sample consists of undergraduates who are users of a web-based e-learning system in a university in Saudi Arabia. The research instrument was tested for construct validity and reliability. The analysed results have shown that information quality is the most important dimension followed by the navigation of the e-learning system. The study has also revealed that the system learnability and visual design came third and fourth in order of importance of e-learning system usability assessment. Finally, the least important design categories that influenced the e-learning system usability assessment were instructional assessment and system interactivity. The empirical results of this study may help to provide insights for designers and evaluators leading to a more effective approach to improve the usability and uptake of the e-learning system.
Keywords: E-learning system, higher education, usability evaluation, usability principles preference, Saudi Arabia.
References
Agarwal, R., & Venkatesh, V. (2002). Assessing a firm’s web presence: A heuristic evaluation procedure for the measurement of usability. Information Systems Research, 13(2), 168–186.
Aldiab, A., Chowdhury, H., Kootsookos, A., Alam, F., & Allhibi, H. (2019). Utilization of Learning Management Systems (LMSs) in higher education system: A case review for Saudi Arabia. Energy Procedia, 160, 731–737. http://doi.org/https://doi.org/10.1016/j.egypro.2019.02.186
Alenezi, A. R., Abdul Karim, A. M., & Veloo, A. (2011). Institutional support and e-learning acceptance : An extension of the technology acceptance model. International Journal of Instructional Technology and Distance Learning, 8(2), 3-16.
Asarbakhsh, M., & Sandars, J. (2013). E-learning: the essential usability perspective. Clinical Teacher, 10(1), 47–50. http://doi.org/10.1111/j.1743-498X.2012.00627.x
Bouznif, M. M. (2018). Business students’ continuance intention toward Blackboard usage: An empirical investigation of UTAUT model. International Journal of Business and Management, 13(1), 120. http://doi.org/10.5539/ijbm.v13n1p120
Bringula, R. P., & Basa, R. S. (2011). Factors affecting faculty web portal usability. Educational Technology and Society, 14(4), 253–265.
Bryman, A., & Bell, E. (2015). Business Research Methods. Oxford, UK: Oxford University Press.
Calisir, F., Elvan Bayraktaroğlu, A., Altin Gumussoy, C., İlker Topcu, Y., & Mutlu, T. (2010). The relative importance of usability and functionality factors for online auction and shopping web sites. Online Information Review, 34(3), 420–439. http://doi.org/10.1108/14684521011037025
Cebi, S. (2013). Determining importance degrees of website design parameters based on interactions and types of websites. Decision Support Systems, 54(2), 1030–1043. http://doi.org/10.1016/j.dss.2012.10.036
Cho, V., Cheng, T. C. E., & Lai, W. M. J. (2009). The role of perceived user-interface design in continued usage intention of self-paced e-learning tools. Computers and Education, 53(2), 216–227. http://doi.org/10.1016/j.compedu.2009.01.014
Dix, A., Finlay, J. E., Abowd, G. D., & Beale, R. (2003). Human-Computer Interaction (3rd Edition). Upper Saddle River, NJ: Prentice-Hall, Inc.
Dringus, L. P., & Cohen, M. S. (2005). An adaptable usability heuristic checklist for online courses. Proceedings Frontiers in Education 35th Annual Conference, T2H–6–T2H–11. http://doi.org/10.1109/FIE.2005.1611918
Ellahi, A., & Bokhari, R. H. (2013). Key quality factors affecting users’ perception of social networking websites. Journal of Retailing & Consumer Services, 20(1), 120–129. Retrieved from http://10.0.3.248/j.jretconser.2012.10.013%5Cnhttp://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=83869898&lang=it&site=ehost-live
Fang, X., & Holsapple, C. W. (2007). An empirical study of web site navigation structures’ impacts on web site usability. Decision Support Systems, 43(2), 476–491. http://doi.org/https://doi.org/10.1016/j.dss.2006.11.004
Gable, G., Sedera, D., & Taizan, C. (2008). Re-conceptualizing information system success: The IS-impact measurement model. Journal of the Association for Information Systems, 9(7), 1–32.
Grossman, T., Fitzmaurice, G., & Attar, R. (2009). A survey of software learnability: metrics, methodologies and guidelines. Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09, 649–658. http://doi.org/10.1145/1518701.1518803
Guo, Y., Wang, J., Moore, J., Liu, M., & Chen, H. L. (2009). A case study of usability testing on an asynchronous e-Learning platform. In 2009 Joint Conferences on Pervasive Computing, JCPC 2009 (pp. 693–697). Red Hook, NY: IEEE eXpress Conference Publishing http://doi.org/10.1109/JCPC.2009.5420093
Hair, Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate Data Analysis. Faculty Publications. New Jersey: Prentice-Hall. http://doi.org/10.1016/j.ijpharm.2011.02.019
Hasan, L. (2014). Evaluating the usability of educational websites based on students’ preferences of design characteristics. International Arab Journal of E-Technology, 3(3), 179–193.
Holden, H., & Rada, R. (2011). Understanding the influence of perceived usability and technology self-efficacy on teachers’ technology acceptance. Journal of Research on Technology in Education, 43(4), 343–367. http://doi.org/10.1080/15391523.2011.10782576
Horton, W. K. (2000). Designing Web-Based Training : How to Teach Anyone Anything Anywhere Anytime. New York, NY: Wiley.
Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51–55. http://doi.org/citeulike-article-id:8060645
Huang, X., & Hsiao, E. L. (2012). Synchronous and asynchronous communication in an online environment: Faculty experiences and perceptions. Quarterly Review of Distance Education, 13(1), 15–30.
Ivanovic, M., Putnik, Z., Komlenov, Z., Welzer, T., Holbl, M., & Schweighofer, T. (2013). Usability and privacy aspects of moodle: Students’ and teachers’ perspective. Informatica (Slovenia), 37(3), 221–230.
Kiget, N. K., Wanyembi, P. G., & Peters, A. I. (2014). Evaluating usability of e-Learning systems in universities. International Journal of Advanced Computer Science and Applications, 5(8), 97–102.
Kirsh, D. (2014). Metacognition, distributed cognition, and visual design. Cognition, Education, and Communication Technology, 147–180. http://doi.org/10.4324/9781410612892
Kline, R. B. (2016). Principles and practice of structural equation modeling, 4th ed. Principles and practice of structural equation modeling (4th ed). New York, NY: Guilford Press.
Koohang, A., & Du Plessis, J. (2004). Architecting usability properties in the e-learning instructional design process. International Journal on ELearning, 3(3), 38–44.
Lanzilotti, R., Ardito, C., Costabile, M. F., Informatica, D., Bari, U., & Angeli, A. De. (2006). eLSE methodology: a systematic approach to the e-learning systems evaluation. Educational Technology & Society, 9(4), 42–53.
Lin, H., Choong, Y. Y., & Salvendy, G. (1997). A proposed index of usability: A method for comparing the relative usability of different software systems. Behaviour and Information Technology, 16(4–5), 267–277. http://doi.org/10.1080/014492997119833
Lindgaard, G. (1994). Usability testing and system evaluation: A guide for designing useful computer systems. London, UK: Chapman & Hall.
Martin, F. (2013). Blackboard as the learning management system of a computer literacy course. Journal of Online Learning and Teaching, 4(2), 138–145. http://doi.org/10.1080/03043790903038841
Mehlenbacher, B., Bennett, L., Bird, T., Ivey, M., Lucas, J., Morton, J., & Whitman, L. (2005). Usable e-learning: A conceptual model for evaluation and design. In Proceedings of HCI International 2005: 11th International Conference on Human-Computer Interaction- Volume 4, (pp.1–10), Las Vegas, NV: Mira Digital.
Ministry of Education Saudi Arabia. (2019). Higher Education Statistics. Retrieved from https://www.moe.gov.sa/en/Pages/default.aspx
Moreno, V., Cavazotte, F., & Alves, I. (2017). Explaining university students’ effective use of e-learning platforms. British Journal of Educational Technology, 48(4), 995–1009. http://doi.org/10.1111/bjet.12469
Mustafa, S. H., & Al-Zoua’bi, L. F. (2009). Usability of the academic websites of Jordan’s universities an evaluation study. In the Proceedings of the 9th Arab Conference on Information Technology - ACIT 2008 (pp. 1-9). Zarqa, Jordan: ACIT International.
Nielsen, J. (1993). Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers Inc.
Noorulhasan, Q., Muhammad, A., Sanober, S., Rafik, M., & Shah, A. (2017). A mixed method study for investigating critical success factors (CSFs) of e-learning in Saudi Arabian universities. International Journal of Advanced Computer Science and Applications, 8(5). http://doi.org/10.14569/IJACSA.2017.080522
Orehovački, T., Granic, A., & Kermek, D. (2013). Evaluating the perceived and estimated quality in use of Web 2.0 applications. Journal of Systems and Software, 86(12), 3039–3059. http://doi.org/https://doi.org/10.1016/j.jss.2013.05.071
Oztekin, A., Kong, Z. J., & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. International Journal of Industrial Ergonomics, 40(4), 455–469. http://doi.org/10.1016/j.ergon.2010.04.001
Oztekin, A., Nikov, A., & Zaim, S. (2009). UWIS: An assessment methodology for usability of web-based information systems. Journal of Systems and Software, 82(12), 2038–2050. http://doi.org/https://doi.org/10.1016/j.jss.2009.06.047
Pallant, J. (2016). SPSS survival manual : a step by step guide to data analysis using IBM SPSS (6th editio). Berkshire, UK : McGraw-Hill Education.
Pearson, J. M., Pearson, A., & Green, D. (2007). Determining the importance of key criteria in web usability. Management Research News, 30(11), 816–828. http://doi.org/10.1108/01409170710832250
Pituch, K. A., & Lee, Y. kuei. (2006). The influence of system characteristics on e-learning use. Computers and Education, 47(2), 222–244. http://doi.org/10.1016/j.compedu.2004.10.007
Preece, J., Rogers, Y., & Sharp, H. (2015). Interaction Design: Beyond Human–Computer Interaction. Interaction Design: Beyond Human–Computer Interaction (4th ed). Chichester, UK: John Wiley & Sons.
Reeves, T. C., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., … Loh, C. S. (2002). Usability and instructional design heuristics for e-learning evaluation. In P. Barker & S. Rebelsky (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 1615–1621). Denver, Colorado, USA: Association for the Advancement of Computing in Education (AACE). Retrieved from http://eric.ed.gov/?id=ED477084
Rubin, B., Fernandes, R., Avgerinou, M. D., & Moore, J. (2010). The effect of learning management systems on student and faculty outcomes. The Internet and Higher Education, 13(1–2), 82–83. http://doi.org/10.1016/J.IHEDUC.2009.10.008
Salloum, S. A., & Shaalan, K. (2019). Factors affecting students’ acceptance of e-learning system in higher education using UTAUT and structural equation modeling approaches. In A. E. Hassnien, K. Shaalan & A.T. Azar (Eds), Proceedings of the International Conference on Advanced Intelligent Systems and Informatics (pp. 469–480). Cham, Switzerland: Springer. http://doi.org/10.1007/978-3-319-99010-1_43
Scholtz, B., Mahmud, I., Mahmud, I., & Ramayah, T. (2016). Does usability matter? An analysis of the impact of usability on technology acceptance in ERP settings. Interdisciplinary Journal of Information, Knowledge, and Management, 11, 309–330.
Selim, H. M. (2007). Critical success factors for e-learning acceptance: Confirmatory factor models. Computers and Education, 49(2), 396–413. http://doi.org/10.1016/j.compedu.2005.09.004
Shee, D. Y., & Wang, Y.-S. (2008). Multi-criteria evaluation of the web-based e-learning system: A methodology based on learner satisfaction and its applications. Computers & Education, 50(3), 894–905. http://doi.org/10.1016/j.compedu.2006.09.005
Shneiderman, B. (1997). Designing the user interface : strategies for effective human-computer-interaction (3rd ed.). Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc.
Smith, L., & Abouammoh, A. (2013). Higher education in Saudi Arabia: Achievements, challenges and opportunities. Springer, 40, 198. http://doi.org/10.1007/978-94-007-6321-0
Squires, D., & Preece, J. (1996). Usability and learning: Evaluating the potential of educational software. Computers and Education, 27(1), 15–22. http://doi.org/10.1016/0360-1315(96)00010-3
Ssemugabi, S., & De Villiers, R. (2007). Usability and learning: A framework for evaluation of web-based e-learning applications. World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2007, 906–913.
Storey, M. A., Phillips, B., Maczewski, M., & Wang, M. (2002). Evaluating the usability of Web-based learning tools. Educational Technology & Society, 5(3), 91–100.
Sun, P., Tsai, R., Finger, G., Chen, Y. Y., & Yeh, D. (2008). What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers and Education, 50(4), 1183–1202. http://doi.org/10.1016/j.compedu.2006.11.007
Tarafdar, M., & Zhang, J. (2005). Analyzing the influence of website design parameters on website usability. Information Resources Management Journal, 18(4), 62–80.
Tee, S. S., Wook, T. S. M. T., & Zainudin, S. (2013). User testing for moodle application. International Journal of Software Engineering and Its Applications, 7(5), 243–252. http://doi.org/10.14257/ijseia.2013.7.5.22
Thowfeek, M. H., & Salam, M. N. A. (2014). Students’ assessment on the usability of E-leaming websites. In Procedia - Social and Behavioral Sciences, 141, 916–922. http://doi.org/10.1016/j.sbspro.2014.05.160
Thuseethan, S., Achchuthan, S., & Kuhanesan, S. (2014). Usability evaluation of learning management systems in Sri Lankan universities. Global Journal of Computer Science and Technology, 15(1), 15-24. Retrieved from http://arxiv.org/abs/1412.0197
Triacca, L., Bolchini, D., Botturi, L., & Inversini, A. (2004). Mile: Systematic usability evaluation for e-Learning web applications. In L. Cantoni & C. McLoughlin (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2004 (pp. 4398–4405). Lugano, Switzerland: Association for the Advancement of Computing in Education (AACE). Retrieved from https://www.learntechlib.org/p/11709
Usability.gov. (2013). Visual Design Basics. Retrieved from http://www.usability.gov/what-and-why/visual-design.html
Van Nuland, S. E., Eagleson, R., & Rogers, K. A. (2017). Educational software usability: Artifact or Design? Anatomical Sciences Education, 10(2), 190–199. http://doi.org/10.1002/ase.1636
Wu, Y.-L., Tao, Y.-H., & Yang, P.-C. (2009). The discussion on influence of website usability towards user acceptability. In Proceedings - International Conference on Management and Service Science (MASS 2009) (pp. 1–4). Red Hook, NY: IEEE eXpress Conference Publishing. http://doi.org/10.1109/ICMSS.2009.5304443
Zaharias, P. (2009). Usability in the context of e-learning: A framework augmenting “traditional” usability constructs with instructional design and motivation to learn. International Journal of Technology and Human Interaction, 5(4), 37–59. http://doi.org/10.4018/jthi.2009062503
Zaharias, P., & Koutsabasis, P. (2011). Heuristic evaluation of e‐learning courses: a comparative analysis of two e‐learning heuristic sets. Campus-Wide Information Systems, 29(1), 45–60. http://doi.org/10.1108/10650741211192046
Zaharias, P., & Poylymenakou, A. (2009). Developing a usability evaluation method for e-learning applications: Beyond functional usability. International Journal of Human-Computer Interaction, 25(1), 75–98. http://doi.org/10.1080/10447310802546716
Zhang, P., von Dran, G. M., Blake, P., & Pipithsuksunt, V. (2001). Important design features in different web site domains: An empirical study of user perceptions. E-Service Journal, 1(1), 77–91. http://doi.org/10.2979/ESJ.2001.1.1.77