logo logo European Journal of Educational Research

EU-JER is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
7321 Parkway Drive South, Hanover, MD 21076, USA
Eurasian Society of Educational Research
Headquarters
7321 Parkway Drive South, Hanover, MD 21076, USA
computer based test meta analysis motivation r studio

Effect of Computer Based Test on Motivation: A Meta-Analysis

Achmad Rante Suparman , Eli Rohaeti , Sri Wening

This meta-analysis study investigates the effect of computer-based tests on motivation. The random effect model is the basis for calculating the summa.

T

This meta-analysis study investigates the effect of computer-based tests on motivation. The random effect model is the basis for calculating the summary effect, and it is found that the 30 articles obtained through the PRISMA method, which consist of four parts, namely identification, screening, eligibility, and included, can be generalized. Data analysis was performed using R studio and OpenMEE. Based on the research results, the p-value <.05 was obtained, so it was concluded that using a computer-based test significantly affected motivation. In the moderator analysis, it was found that computer-based tests were most effective for intrinsic motivation for the motivation variable, university students for the participants variable, and the country of Iran for the region variable. Evaluation of publication bias using the funnel plot and regression method shows no publication bias, proving that the 30 articles analyzed represent all existing studies on computer-based tests.

Keywords: Computer-based test, meta-analysis, motivation, R studio.

cloud_download PDF
Cite
Article Metrics
Views
390
Download
421
Citations
Crossref
0

Scopus
0

References

Agustini, K., & Wahyuni, D. S. (2013). Pengaruh penggunaan simulasi binary tree berbasis CAI terhadap motivasi dan hasil belajar matematika diskrit mahasiswa jurusan PTI undiksha [Effect of using CAI-based binary tree simulation on motivation and learning outcomes of discrete mathematics students majoring in PTI undiksha]. Jurnal Pendidikan Indonesia, 2(1), 162–172. https://doi.org/10.23887/jpi-undiksha.v2i1.6369

Ahmad, N. J., Yakob, N., Bunyamin, M. A. H., Winarno, N., & Akmal, W. H. (2021). The effect of interactive computer animation and simulation on students’ achievement and motivation in learning electrochemistry. Jurnal Pendidikan IPA Indonesia, 10(3), 311–324. https://doi.org/10.15294/JPII.V10I3.26013

Akhtar, H., Silfiasari, Vekety, B., & Kovacs, K. (2022). The effect of computerized adaptive testing on motivation and anxiety: A systematic review and meta-analysis. Assessment, 30(5), 1379-1390. https://doi.org/10.1177/10731911221100995

Aremu, B. V. (2021). The use of mobile learning to improve students’ motivation and the achievement of learning outcomes. KIU Journal of Humanities, 6(2), 175–183. http://bit.ly/3QUG2FO

Bailey, D., Almusharraf, N., & Hatcher, R. (2021). Finding satisfaction: Intrinsic motivation for synchronous and asynchronous communication in the online language learning context. Education and Information Technologies, 26, 2563–2583. https://doi.org/10.1007/s10639-020-10369-z 

Baragilly, M., & Willis, B. H. (2022). On estimating a constrained bivariate random effects model for meta-analysis of test accuracy studies. Statistical Methods in Medical Research, 31(2), 287–299. https://doi.org/10.1177/09622802211065157

Beisemann, M., Doebler, P., & Holling, H. (2020). Comparison of random-effects meta-analysis models for the relative risk in the case of rare events: A simulation study. Biometrical Journal, 62(7), 1597–1630. https://doi.org/10.1002/bimj.201900379

Boevé, A. J., Meijer, R. R., Albers, C. J., Beetsma, Y., & Bosker, R. J. (2015). Introducing computer-based testing in high-stakes exams in higher education: Results of a field experiment. PLoS ONE, 10(12), Article e0143616. https://doi.org/10.1371/journal.pone.0143616

Borenstein, M., & Higgins, J. P. T. (2013). Meta-analysis and subgroups. Prevention Science, 14, 134–143. https://doi.org/10.1007/s11121-013-0377-7

Bridgeman, B., Lennon, M. L., & Jackenthal, A. (2003). Effects of screen size, screen resolution, and display rate on computer-based test performance. Applied Measurement in Education, 16(3), 191–205. https://doi.org/10.1207/S15324818AME1603_2

Butcher, J. N., Perry, J. N., & Atlis, M. M. (2000). Validity and utility of computer-based test interpretation. Psychological Assessment, 12(1), 6–18. https://doi.org/10.1037/1040-3590.12.1.6

Çakir, R. (2019). Effect of web-based intelligence tutoring system on students’ achievement and motivation. Malaysian Online Journal of Educational Technology, 7(4), 45–59. https://bit.ly/44pNVbW

Card, N. A. (2012). Applied meta-analysis for social science research. The Guilford Press. https://psycnet.apa.org/record/2011-26930-000

Carpenter, R., & Alloway, T. (2019). Computer versus paper-based testing: Are they equivalent when it comes to working memory? Journal of Psychoeducational Assessment, 37(3), 382–394. https://doi.org/10.1177/0734282918761496

Çelen, B. (2019). Motivations for teaching english: A review of journal articles on L2 teacher motivation. In G. Y. Ekşi, L. Guerra, D. Werbińska, & Y. Bayyurt (Eds.), Research trends in english language teacher education and english language teaching (pp. 241–258). University of Évora.

Cheng, L., Ritzhaupt, A. D., & Antonenko, P. (2019). Effects of the flipped classroom instructional strategy on students’ learning outcomes: A meta-analysis. Educational Technology Research and Development, 67(4), 793-824. https://doi.org/10.1007/s11423-018-9633-7

Chua, Y. P. (2012). Effects of computer-based testing on test performance and testing motivation. Computers in Human Behavior, 28(5), 1580–1586. https://doi.org/10.1016/j.chb.2012.03.020

Chua, Y. P., & Don, Z. M. (2013). Effects of computer-based educational achievement test on test performance and test takers’ motivation. Computers in Human Behavior, 29(5), 1889–1895. https://doi.org/10.1016/j.chb.2013.03.008

Davies, R. S., & West, R. E. (2014). Technology integration in schools. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 841–853). Springer. https://doi.org/10.1007/978-1-4614-3185-5_68

Dettori, J. R., Norvell, D. C., & Chapman, J. R. (2022). Fixed-effect vs random-effects models for meta-analysis: 3 points to consider. Global Spine Journal, 12(7), 1624–1626. https://doi.org/10.1177/21925682221110527

Dewi, A. R. (2020). Students’ perception towards computer-based testing-edubox-and its impact to their English learning motivation at SMP negeri 45 bandung [Doctoral dissertation, Universitas Terbuka]. Universitas Terbuka Digital Archive. http://repository.ut.ac.id/9702/1/cover%20dafis.pdf

Dewi, A. R., Saehu, A., & Budiman, R. (2021). Students’ perception of computer-based testing-edubox–and its impact on their english learning motivation. Indonesian Journal of Learning and Instruction, 4(1), 19–28. https://bit.ly/3HhTSis

Dolan, R., & Burling, K. (2012). Computer-based testing in higher education. In C. Secolsky & D. B. Denison (Eds.), Handbook on measurement, assessment, and evaluation in higher education (pp. 321–335). Routledge.

Drasgow, F., & Mattern, K. (2006). New tests and new items: Opportunities and issues. In D. Bartram & R. K. Hambleton (Eds.), Computer-based testing and the internet (pp. 59–76). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470712993.ch3

Dresel, M., & Haugwitz, M. (2008). A computer-based approach to fostering motivation and self-regulated learning. The Journal of Experimental Education, 77(1), 3–20. https://doi.org/10.3200/JEXE.77.1.3-20

Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(9), 629–634. https://doi.org/10.1136/bmj.315.7109.629

Gholami, M., Changaee, F., Karami, K., Shahsavaripour, Z., Veiskaramian, A., & Birjandi, M. (2021). Effects of multiepisode case-based learning (CBL) on problem-solving ability and learning motivation of nursing students in an emergency care course. Journal of Professional Nursing, 37(3), 612–619. https://doi.org/10.1016/j.profnurs.2021.02.010

Good, V., Hughes, D. E., Kirca, A. H., & McGrath, S. (2022). A self-determination theory-based meta-analysis on the differential effects of intrinsic and extrinsic motivation on salesperson performance. Journal of the Academy of Marketing Science, 50, 586–614. https://doi.org/10.1007/s11747-021-00827-6

Granito, M., & Chernobilsky, E. (2012, October 19). The effect of technology on a student’s motivation and knowledge retention [Paper presentation]. Northeastern Educational Research Association (NERA) Annual Conference, University of Connecticut, United States.

Grund, A., Galla, B. M., & Fries, S. (2022). Achievement motivation in students’ everyday lives: Its relationship to momentary positive and negative activation and the moderating role of mindfulness. Learning and Individual Differences, 97, Article 102176. https://doi.org/10.1016/j.lindif.2022.102176

Hakim, B. M. (2018). Comparative study on validity of paper-based test and computer-based test in the context of educational and psychological assessment among arab students. International Journal of English Linguistics, 8(2), 85–91. https://doi.org/10.5539/ijel.v8n2p85

Hamman, E. A., Pappalardo, P., Bence, J. R., Peacor, S. D., & Osenberg, C. W. (2018). Bias in meta-analyses using Hedges’ d. Ecosphere, 9(9), Article e02419. https://doi.org/10.1002/ecs2.2419

Hariri-Akbari, M., Shokrvash, B., Mahmoodi, F., Jahanjoo-Aminabad, F., Yousefi, B., & Azabdaftari, F. (2018). Conversion of extrinsic into intrinsic motivation and computer based testing (CBT). BMC Medical Education, 18, Article 143. https://doi.org/10.1186/s12909-018-1249-4

Harrer, M., Cuijpers, P., Furukawa, T., & Ebert, D. (2021). Doing meta-analysis with R: A hands-on guide. Chapmann & Hall Press. https://doi.org/10.1201/9781003107347

Haryanto. (2012). Pemrograman visual C++ konsep dasar, algoritma dan aplikasi [Visual C++ programming basic concepts, algorithms and applications]. UNY Press.

Hidayah, H. (2016). Pengaruh penggunaan instrumen tes berbasis wondershare quiz creator dengan paper and pencil test terhadap motivasi belajar siswa pada mata pelajaran PPKn [The effect of using a wondershare quiz creator-based test instrument with a paper and pencil test on student learning motivation in Civics subjects] [Unpublished master's thesis]. Universitas Sebelas Maret.

Hoerunnisa, A., Suryani, N., & Efendi, A. (2019). The effectiveness of the use of e-learning in multimedia classes to improve vocational students’ learning achievement and motivation. Kwangsan: Jurnal Teknologi Pendidikan, 7(2), 123–137. https://doi.org/10.31800/jtp.kw.v7n2.p123--137

Ibrahim, D. S., & Suardiman, S. P. (2014). Pengaruh penggunaan e-learning terhadap motivasi dan prestasi belajar matematika siswa sd negeri tahunan yogyakarta [The effect of the use of e-learning on the motivation and achievement of learning mathematics in elementary school tahunan Yogyakarta]. Jurnal Prima Edukasia, 2(1), 66–79. https://doi.org/10.21831/jpe.v2i1.2645

Istiyono, E., Dwandaru, W. S. B., Erfianti, L., & Astuti, W. (2020). Applying CBT in physics learning to measure students’ higher order thinking skills. Journal of Physics: Conference Series, 1440, Article 012061 https://doi.org/10.1088/1742-6596/1440/1/012061

Jackson, D., & Turner, R. (2017). Power analysis for random-effects meta-analysis. Research Synthesis Methods, 8(3), 290–302. https://doi.org/10.1002/jrsm.1240

Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers and Education, 55(2), 427–443. https://doi.org/10.1016/j.compedu.2010.02.007

Keter, J. K. (2018). Effects of computer based cooperative mastery learning on secondary school students’ skills acquisition, motivation and achievement in chemistry practicals in bomet county, Kenya [Doctoral dissertation, Egerton University]. Egerton University Institutional Repositary. https://bit.ly/44lGvGS

Khoshsima, H., Hosseini, M., & Toroujeni, S. M. H. (2017). Cross-mode comparability of computer-based testing (CBT) versus paper-pencil based testing (PPT): An investigation of testing administration mode among iranian intermediate EFL learners. English Language Teaching, 10(2), 23-32. https://doi.org/10.5539/elt.v10n2p23

Kotera, Y., Taylor, E., Fido, D., Williams, D., & Tsuda-McCaie, F. (2021). Motivation of UK graduate students in education: Self-compassion moderates pathway from extrinsic motivation to intrinsic motivation. Current Psychology. 42, 10163–10176. Advance online publication. https://doi.org/10.1007/s12144-021-02301-6

Lestari, I. (2022). Pengaruh ujian online terhadap kecemasan, motivasi dan prestasi siswa [The effect of online exams on students' anxiety, motivation and achievement][Master's thesis, Universitas Muhammadiyah Malang]. Universitas Muhammadiyah Malang Digital Archive. https://eprints.umm.ac.id/84571/1/Indah%20Lestari.pdf

Lowyck, J. (2014). Bridging learning theories and technology-enhanced environments: A critical appraisal of its history. In J. M. Spector, M. D. Merrill, J. Elen, & M.J.Bishop (Eds.), Handbook of research on educational communications and technology (pp. 3–20). Springer. https://doi.org/10.1007/978-1-4614-3185-5_1

Luecht, R. M. (2006). Operational issues in computer-based testing. In D. Bartram & R. K. Hambleton (Eds.), Computer based testing and the internet (pp. 91–114). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470712993.ch5

Maqableh, M., Masa’deh, R. M. T., & Mohammed, A. B. (2015). The acceptance and use of computer based assessment in higher education. Journal of Software Engineering and Applications, 8(10), 557–574. https://doi.org/10.4236/jsea.2015.810053

Mehrabi, T., Behzadi, S., Sabouri, F., & Alavi, M. (2016). Assessment the effect of the CBT on motivation of the nursing students. Iranian Journal of Nursing and Midwifery Research, 21(2), 118–123. https://bit.ly/3XD0Rcn

Morris, L. S., Grehl, M. M., Rutter, S. B., Mehta, M., & Westwater, M. L. (2022). On what motivates us: A detailed review of intrinsic v. extrinsic motivation. Psychological Medicine, 52(10), 1801-1816. https://doi.org/10.1017/S0033291722001611

Nasri, M., Shafiee, S., & Sepehri, M. (2021). An investigation of Iranian intermediate EFL learners’ L2 motivation and attitude in a computer-assisted language learning environment. Issues in Language Teaching, 10(1), 355–389. https://doi.org/10.22054/ilt.2021.62359.614

Nikou, S. A., & Economides, A. A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55(B), 1241–1248. https://doi.org/10.1016/j.chb.2015.09.025

Ockey, G. J. (2009). Developments and challenges in the use of computer-based testing for assessing second language ability. Modern Language Journal, 93(1), 836–847. https://doi.org/10.1111/j.1540-4781.2009.00976.x

Partovi, T., & Razavi, M. R. (2019). The effect of game-based learning on academic achievement motivation of elementary school students. Learning and Motivation, 68, Article 101592. https://doi.org/10.1016/j.lmot.2019.101592

Piaw, C. Y. (2011, January 15). Comparisons between computer-based testing and paper-pencil testing: Testing effect, test scores, testing time and testing motivation [Paper presentation]. The Informatics Conference, University of Malaya, Malaysia.

Piaw, C. Y. (2012). Replacing paper-based testing with computer-based testing in assessment: Are we doing wrong? Procedia - Social and Behavioral Sciences, 64, 655–664. https://doi.org/10.1016/j.sbspro.2012.11.077

Proske, A., Roscoe, R. D., & McNamara, D. S. (2014). Game-based practice versus traditional practice in computer-based writing strategy training: Effects on motivation and achievement. Educational Technology Research and Development, 62, 481–505. https://doi.org/10.1007/s11423-014-9349-2

Ramdani, D., Susilo, H., Suhadi, & Sueb. (2022). The effectiveness of collaborative learning on critical thinking, creative thinking, and metacognitive skill ability: Meta-analysis on biological learning. European Journal of Educational Research, 11(3), 1607–1628. https://doi.org/10.12973/eu-jer.11.3.1607

Romero, C., Ventura, S., & de Bra, P. (2009). Using mobile and web-based computerized tests to evaluate university students. Computer Applications in Engineering Education, 17(4), 437–477. https://doi.org/10.1002/cae.20242

Ronoh, P. K., Wachanga, S. W., & Keraro, F. N. (2014). Effects of computer based mastery learning approach on students’ motivation to learn biology. Journal of Education and Practice, 5(32), 117–126. http://bit.ly/3wlTkC6

Saepuzaman, D., Retnawati, H., Istiyono, E., & Haryanto. (2021). Can innovative learning affect students’ HOTS achievements?: A meta-analysis study. Pegem Egitim ve Ogretim Dergisi, 11(4), 290–305. https://doi.org/10.47750/pegegog.11.04.28

Safaruddin, Taufan, J., Mahdi, A., Intania, & Silvia, R. (2021). Developing a computer based test application to measure braille literation ability for special teacher. In Ifdil, H. Ardi, Z. Ardi, P. Susanto, & Nofrion (Eds.), Proceedings of the 2nd Progress in Social Science, Humanities and Education Research Symposium (PSSHERS 2020) (pp. 36-38). Atlantis Press. https://doi.org/10.2991/assehr.k.210618.007

Schunk, D. H., & DiBenedetto, M. K. (2020). Motivation and social cognitive theory. Contemporary Educational Psychology, 60, Article 101832. https://doi.org/10.1016/j.cedpsych.2019.101832

Sins, P. H. M., van Joolingen, W. R., Savelsbergh, E. R., & van Hout-Wolters, B. (2008). Motivation and performance within a collaborative computer-based modeling task: Relations between students’ achievement goal orientation, self-efficacy, cognitive processing, and achievement. Contemporary Educational Psychology, 33(1), 58–77. https://doi.org/10.1016/j.cedpsych.2006.12.004

Soimah, I. (2018). Pengaruh media pembelajaran berbasis komputer terhadap hasil belajar IPA ditinjau dari motivasi belajar siswa [The influence of computer-based learning media on science learning outcomes in terms of student learning motivation]. Natural: Jurnal Ilmiah Pendidikan IPA, 5(1), 38–44. https://doi.org/10.30738/natural.v5i1.2559

Suparman, A. R., Rohaeti, E., & Wening, S. (2022). Development of attitude assessment instruments towards socio-scientific issues in chemistry learning. European Journal of Educational Research, 11(4), 1947-1958. https://doi.org/10.12973/eu-jer.11.4.1947

Taranilla, R. V., Tirado-Olivares, S., Cózar-Gutiérrez, R., & González-Calero, J. A. (2022). Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review, 35, Article 100434. https://doi.org/10.1016/j.edurev.2022.100434

Tüzün, H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y., & Kızılkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education, 52(1), 68–77. https://doi.org/10.1016/j.compedu.2008.06.008

van der Linden, W. J. (2006). Model based innovations in computer based testing. In D. Bartram & R. K. Hambleton (Eds.), Computer based testing and the internet issues and advances (pp. 39–58). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470712993.ch2

Wallace, B. C., Lajeunesse, M. J., Dietz, G., Dahabreh, I. J., Trikalinos, T. A., Schmid, C. H., & Gurevitch, J. (2017). OpenMEE: Intuitive, open-source software for meta-analysis in ecology and evolutionary biology. Methods in Ecology and Evolution, 8(8), 941–947. https://doi.org/10.1111/2041-210X.12708

Wu, Q., Wang, J., Lin, X., Han, D., Hu, H., & Gao, H. (2023). Determining the efficacy and safety of acupuncture for the preventive treatment of menstrual migraine: A protocol for a PRISMA-compliant systematic review and meta-analysis. Journal of Pain Research, 16, 101-109. https://doi.org/10.2147/JPR.S394446

Zarei, A. A., & Hashemipour, M. (2015). The effect of computer-assisted language instruction on improving EFL learners’ autonomy and motivation written corrective feedback view project the effect of computer-assisted language instruction on improving EFL learners’ autonomy and motivation. Journal of Applied Linguistics, 1(1), 40–58. https://bit.ly/3wjrjej

Zenisky, A. L., & Luecht, R. M. (2016). The future of computer‑based testing: Some new paradigms. In C. S. Wells & M. Faulkner-Bond (Eds.), Educational measurement from foundations to future (pp. 221–238). The Guilford Press.

...