The Development of Computerized Economics Item Banking for Classroom and School-Based Assessment
The advancement of information and technology resulted in the change in conventional test methods. The weaknesses of the paper-based test can be minim.
- Pub. date: January 15, 2020
- Pages: 293-303
- 521 Downloads
- 560 Views
- 2 Citations
The advancement of information and technology resulted in the change in conventional test methods. The weaknesses of the paper-based test can be minimized using the computer-based test (CBT). The development of a CBT desperately needs a computerized item bank. This study aimed to develop a computerized item bank for classroom and school-based assessments. A research and development method is used in this study, which consisted of four phases, i.e., planning, item development, system development, and field testing. Data is collected through documentation, expert judgment, and field testing. The data were analyzed using descriptive statistics and item response theory. The sample of this study was teachers and high school students in West Sumatera province selected using purposive random sampling techniques. The results of the study are as follows. 1) The computerized item bank has excellent quality based on expert validation. 2) There are 120 items inputted into the item bank system that has a moderate difficulty and good discriminant index based on item response theory. 3) The field testing indicated the computerized economics item banking has high effectiveness of usability, usefulness for the teachers, and feasible for classroom and school-based assessment.
Keywords: Item banking, CBT, assessment, economics.
References
Abubakar, A. S., & Adebayo, F. O. (2014). Using computer based test method for the conduct of examination in Nigeria: Prospects, challenges and strategies. Mediterranean Journal of Social Sciences, 5(2), 47-55. https://doi.org/ 10.5901/mjss.2014.v5n2p47
Al-Amri, S. (2007). Computer-based vs. Paper-based testing: Does the test administration mode matter? In Proceedings of the British Association of Applied Linguistics Conference (pp. 101-110). Edinburgh, Scotland.
Archer, E., & Brown, G. T. (2013). Beyond rhetoric: Leveraging learning from New Zealand's assessment tools for teaching and learning for South Africa. Education as Change, 17(1), 131-147. https://doi.org/ 10.1080/16823206.2013.773932
Ata, N. A., & Ani, W. (2012). Analisis butir soal tes kendali mutu kelas XII SMA mata pelajaran ekonomi akuntansi di kota yogyakarta tahun 2012 [Item Analysis of Quality Control Tests for Class XII High School on Economics in Yogyakarta in 2012]. Indonesian Accounting Education Journal/Jurnal Pendidikan Akuntansi Indonesia, X(1), 1-26.
Baker, F. B., & Kim, S.-H. (2017). The basics of item response theory using R. New York, NY: Springer.
Bodmann, S. M., & Robinson, D. H. (2004). Speed and performance differences among computer-based and paper-pencil tests. Journal of Educational Computing Research, 31(1), 51-60.
Brennan, R. (2006). Educational measurement. Westport, CT: Greenwood Publishing Group.
Brown, G. T. (2014). AsTTle–A national testing system for formative assessment: How the national testing policy ended up helping schools and teachers. In M. K. Lai & S. Kushner (Eds.), A developmental and negotiated approach to school self-evaluation (pp. 39-56). London, England: Emerald Group Publishing.
Carlson, J. E., & von Davier, M. (2017). Item response theory. In R. E. Bennett & M. v. Davier (Eds.), Advancing Human Assessment: The Methodological, Psychological and Policy Contributions of ETS 2017 (pp. 1-69). Princeton, NJ: Springer International Publishing.
Carnegie-Harding, S. (2016). E-asTTle as a catalyst for change (Unpublished masters’s thesis). Massey University, Palmerston North, New Zealand.
Chen, L.-J., Ho, R.-G., & Yen, Y.-C. (2010). Marking strategies in metacognition-evaluated computer-based testing. Journal of Educational Technology & Society, 13(1), 246-259.
Chien, S.-P., Wu, H.-K., & Hsu, Y.-S. (2014). An investigation of teachers’ beliefs and their use of technology-based assessments. Computers in Human Behavior, 31(2), 198-210.
Cizek, G. J. (2001). An overview of issues concerning cheating on large-scale tests. In Proceedings of Annual Meeting of the National Conference on Measurement in Education (pp. 4-30). Seattle, Washington.
Friyatmi, F. (2011). Faktor-faktor penentu perilaku mencontek di kalangan mahasiswa fakultas ekonomi UNP [Determinant factors of cheating behavior among UNP Students in economic faculty]. TINGKAP, 7(2), 173-188.
Hambleton, R. K., & Swaminathan, H. (2013). Item response theory: Principles and applications. Boston, MA: Springer Science & Business Media.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publication.
Hattie, J. A., & Brown, G. T. (2007). Technology for school-based assessment and assessment for learning: Development principles from New Zealand. Journal of Educational Technology Systems, 36(2), 189-201. https://doi.org/ 10.2190%2FET.36.2.g
Hattie, J. A., Brown, G. T. L., Ward, L., Irving, S. E., & Keegan, P. J. (2006). Formative evaluation of an educational assessment technology innovation: Developers’ insights into Assessment Tools for Teaching and Learning (asTTle). Journal of Multi-Disciplinary Evaluation, 5(3), 1-54.
Hayati, N., & Mardapi, D. (2014). Pengembangan butir soal matematika SD di Kabupaten Lombok Timur sebagai upaya dalam pengadaan bank soal[Development of elementary school mathematics items in East Lombok Regency as an effort in the procurement of item banks]. Journal of Education/Jurnal Kependidikan: Penelitian Inovasi Pembelajaran, 44(1), 26-38.
He, Q., & Tymms, P. (2005). A computer‐assisted test design and diagnosis system for use by classroom teachers. Journal of Computer Assisted Learning, 21(6), 419-429. https://doi.org/ 10.1111/j.1365-2729.2005.00148.x
Hwang, G.-J., & Chang, H.-F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education, 56(4), 1023-1031. https://doi.org/ 10.1016/j.compedu.2010.12.002
Kemdikbud. (2019). Ujian nasional siap digelar: 8,3 juta peserta dengan 91 persen berbasis komputer [National examinations ready to be held: 8.3 million participants with 91 percent computer-based]. Jakarta: Kementerian Pendidikan dan Kebudayaan.
Mardapi, D. (2017). Pengukuran, penilaian dan evaluasi pendidikan [Measurement, assessment and evaluation of education]. Yogyakarta, Indonesia: Parama Publishing.
Millman, J., & Arter, J. A. (1984). Issues in item banking. Journal of Educational Measurement, 21(4), 315-330. https://doi.org/10.1111/j.1745-3984.1984.tb01037.x
Morales-Martinez, G. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Trevino, M. G., & Gonzales-Trujillo, C. J. (2017). Cognitive analysis of meaning and acquired mental representations as an alternative measurement method technique to innovate e-assessment. European Journal of Educational Research, 6(4), 455-464.
Nicol, D. (2007). E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922
Novitasari, I. (2012). Sindikasi Jual-beli kunci jawaban ujian nasional 2010 (studi kasus jaringan jual-beli kunci jawaban ujian nasional 2010 di kota X) [ Syndication of the answer keys trading for national examination in the 2010 (Case Study of the sindicate in city X)]. Indonesian Criminology Journal/Jurnal Kriminologi Indonesia, 7(2), 267-286.
Ockey, G. J. (2009). Developments and challenges in the use of computer‐based testing for assessing second language ability. The Modern Language Journal, 93, 836-847. https://doi.org/10.1111/j.1540-4781.2009.00976.x
Puspendik. (2013). Ujian Nasional [National examination]. Jakarta, Indonesia: Puspendik Balitbang Kemdikbud.
Puspendik. (2014). Ujian Nasional [National examination]. Jakarta, Indonesia: Puspendik Balitbang Kemdikbud.
Puspendik. (2015). Ujian Nasional [National examination]. Jakarta, Indonesia: Puspendik Balitbang Kemdikbud.
Quellmalz, E. S., & Pellegrino, J. W. (2009). Technology and testing. Science, 323(5910), 75-79. https://doi.org/ 10.1126/science.1168046
Retnawati, H., & Hadi, S. (2014). Sistem bank soal daerah terkalibrasi untuk menyongsong era desentralisasi [Calibrated regional item bank system to welcome the era of decentralization]. Educational Science Journal/Jurnal Ilmu Pendidikan, 20(2), 183-193.
Rice, G. (2003). Designing on line tests: The role of assessment in web-based learning. Journal of Adventist Education, (April/May), 31-35.
Rubie-Davies, C. M., & Rosenthal, R. (2016). Intervening in teachers' expectations: A random effects meta-analytic approach to examining the effectiveness of an intervention. Learning and Individual differences, 50(6), 83-92.
Rudner, L. (1998). Item banking. Practical Assessment, Research & Evaluation, 6(4), 1-3.
Sahin, A., & Anil, D. (2017). The effects of test length and sample size on item parameters in item response theory. Educational Sciences: Theory and Practice, 17(1n), 321-335. https://doi.org/10.12738/estp.2017.1.0270
Scibinetti, P., Tocci, N., & Pesce, C. (2011). Motor creativity and creative thinking in children: The diverging role of inhibition. Creativity Research Journal, 23(3), 262-272. https://doi.org/10.1080/10400419.2011.595993
Setiawan, S. D. (2017). UNBK dinilai kurangi praktik kecurangan saat ujian nasional [UNBK was assessed reducing the practice of cheating during the national examination, Republika. Retrieved from https://republika.co.id/berita/pendidikan/eduaction/17/12/24/p1g4wn438-unbk-dinilai-kurangi-praktik-kecurangan-saat-ujian-nasional
Suyata, P., Mardapi, D., Kartowagiran, B., & Retnawati, H. (2011). Model pengembangan bank soal berbasis guru dan mutu pendidikan [Teacher-based item bank development model and education quality]. Journal of Education/Jurnal Kependidikan: Penelitian Inovasi Pembelajaran, 41(2), 120-128.
Umar, J. (1999). Item Banking. In G. N. Master & J. P. Keeves (Eds.), Advanced in Measurement in Educational Research and Assessment (pp. 207-218). Amsterdam, Netherlands: Pergamon.
Utantoro, A. (2019). UN Berbasis Komputer Tekan Kecurangan [Computer-based national exam press cheating]. Media Indonesia. Retrieved from https://mediaindonesia.com/read/detail/225397-un-berbasis-komputer-tekan-kecurangan
Valenti, S., Cucchiarelli, A., & Panti, M. (2002). Computer based assessment systems evaluation via the ISO9126 quality model. Journal of Information Technology Education: Research, 1(1), 157-175.
Wang, H., & Shin, C. D. (2009). Computer-based & paper-pencil test comparability studies. Test, Measurement & Research Services Bulletin, 9, 1-6.
Wang, T.-H., Wang, K.-H., Wang, W.-L., Huang, S.-C., & Chen, S. Y. (2004). Web‐based assessment and test analyses (WATA) system: development and evaluation. Journal of Computer Assisted Learning, 20(1), 59-71. https://doi.org/doi: 10.1111/j.1365-2729.2004.00066.x
Weiss, D. J. (2013). Item banking, test development, and test delivery. In K. F. Geisinger (Ed.), The APA Handbook on Testing and Assessment. Washington, WA: American Psychological Association.
Winkley, J. (2010). E-assessment and innovation emerging technologies. Coventry, England: Becta.
Wood, R., & Skurnik, L. S. (1969). Item banking. London, England: King, Thorne & Stace Ltd.
Xing, D., & Hambleton, R. K. (2004). Impact of test design, item quality, and item bank size on the psychometric properties of computer-based credentialing examinations. Educational and Psychological Measurement, 64(1), 5-21. https://doi.org/10.1177%2F0013164403258393
Zvarych, I., Kalaur, S. M., Prymachenko, N. M., RomashchenkO, I. V., & Romanyshyna, O. I. (2019). Gamification as a tool for stimulating the educational activity of students of higher educational institutions of ukraine and the united states. European Journal of Educational Research, 8(3), 875-891.