Design and Validation of Mathematical Literacy Instruments for Assessment for Learning in Indonesia
Suciati
,
Sudji Munadi
,
Sugiman
,
Wiwin Dwi Ratna Febriyanti
This study aims to design mathematical literacy instruments that have evidence of content and construct validity and are reliable for use as an Assess.
- Pub. date: April 15, 2020
- Pages: 865-875
- 1044 Downloads
- 1014 Views
- 9 Citations
This study aims to design mathematical literacy instruments that have evidence of content and construct validity and are reliable for use as an Assessment for Learning. The research involved eight experts as instrument validators and 273 eighth-grade students of junior high school in Yogyakarta Province. The results showed that the ten mathematical literacy items developed had the V Aiken coefficient index calculated from 0.781 to 0.906 (> 0.75). The results of adequacy testing of samples with KMO and Bartlett show Chi-Square in the Bartlett test of 608,608, the p-value <0.05 and KMO value of 0.781 (> 0.5). The results of testing of the measurement model with Confirmatory Factor Analysis (CFA) produce a Root Mean Square Error of Approach (RMSEA) value of 0.049 (≤ 0.08), chi-s Square of 33.92 (<2df), the p-value of 0.05004 (≥ 0.05). Nine out of the ten items developed had t-value> 1.96, Standardized Loading Factor (SLF) was greater than the critical limit (> 0.3), and Construct Reliability (CR) of 0.78 (> 0.7). It can be concluded that the developed mathematical literacy instrument can measure what must be measured and nine items significantly reflect the construct or latent variable, as well as the level of consistency of a good score.
Keywords: Instruments, mathematics literacy, content validity, construct validity, construct reliability.
References
American Educational Research Association (2014). Standards for educational and psychological testing. AERA.
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and psychological measurement, 45(1), 131-142.
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Brooks/Cole Publishing Company.
Anderson, L. W. (2003). Classroom assessment: Enhancing the quality of teacher decision making. Lawrence Erlbaum Associates.
Azwar, S. (1996). Tes prestasi: Fungsi dan pengembangan pengukuran prestasi belajar [Achievement tests: Function and development of learning achievement measurements]. Pustaka Pelajar.
Azwar, S. (2007). Validitas dan reliabilitas [Validity and reliability]. Pustaka Pelajar.
Bennett, R. E., & Gitomer, D. H. (2009). Transforming K–12 assessment: Integrating accountability testing, formative assessment, and professional support. In C. Wyatt-Smith & J. J. Cumming (Eds.), Educational assessment in the 21st century (pp. 43-61). Springer.
Bishop, A. J., Clements, M. K., Clements, K., Keitel, C., Kilpatrick, J., & Laborde, C. (1996). International handbook of mathematics education (Vol. 4). Kluwer Academic Publishers.
Black, P., Harrison, C., & Lee, C. (2003). Assessment for learning: Putting it into practice. McGraw-Hill Education (UK).
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1), 5. https://doi.org/10.1007/s11092-008-9068-5
Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Publications.
Burton, M., Silver, E. A., Mills, V. L., Audrict, W., Strutchens, M. E., & Petit, M. (2018). Formative assessment and mathematics teaching: Leveraging powerful linkages in the US context. In D. Thompson, M. Burton, A. Cusiie & D. Wright (Eds.), Classroom assessment in Mathematics: Perspectives from around the globe (pp. 193-205). Springer International Publishing.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt Rinehard and Winston Inc.
del Prado Hill, P., Friedland, E. S., & McMillen, S. (2016). Mathematics-literacy checklists: A pedagogical innovation to support teachers as they implement the common core. Journal of Inquiry and Action in Education, 8(1), 23-38. https://doi.org/10.1598/JAAL.55.1.6
Friyatmi, Mardapi, D., Haryanto, & Rahmi, E. (2020). The development of computerized economics item banking for classroom and school-based assessment. European Journal of Educational Research, 9(1), 293-303. https://doi.org/10.12973/eu-jer.9.1.293.
Ghozali, I., & Fuad (2015). Structural equation modeling: Teori, konsep, dan aplikasi dengan Program Lisrel 9.10 [Structural equation modeling: Theories, concepts, and applications with the Lisrel 9.10 Program]. Diponegoro University Publisher Agency.
Gorsuch, R. L. (1983). Factor analysis. Psychology press.
Hidayati, K., Budiyono, & Sugiman (2018). Development and validation of student’s responsibility scale on mathematics learning using subject scaling model. International Journal of Instruction, 11(4), 499-512. https://doi.org/10.12973/iji.2018.11431a
Hsu, C.-C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12(10), 1-8.
Ichsan, I. Z., Sigit, D. V., Miarsyah, M., Ali, A., Arif, W. P., & Prayitno, T. A. (2019). HOTS-AEP: Higher order thinking skills from elementary to master students in environmental learning. European Journal of Educational Research, 8(4), 935-942. https://doi.org/10.12973/eu-jer.8.4.935
Istiyono, E. (2018). Pengembangan instrumen penilaian dan analisis hasil belajar fisika dengan teori tes klasik dan modern [Development of assessment instruments and analysis of physics learning outcomes with classical and modern test theory]. UNY Press.
Kemendikbud (2017). Panduan penilaian oleh pendidik dan satuan pendidikan sekolah menengah pertama [Guidance assessment by educators and middle school education units]. Ministry of Education and Culture.
Kline, P. (2014). An easy guide to factor analysis. Routledge.
Kuger, S., Klieme, E., Jude, N., & Kaplan, D. (2016). Assessing contexts of learning: An international perspective. Springer.
Kula Unver, S., Hidiroglu, C. N., Tekin Dede, A., & Bukova Guzel, E. (2018). Factors revealed while posing mathematical modelling problems by Mathematics student teachers. European Journal of Educational Research, 7(4), 941-952. https://doi.org/10.12973/eu-jer.7.4.941
Mardhapi, D. (2017). Pengukuran, penilaian, dan evaluasi pendidikan [Measurement, assessment and evaluation of education]. Parama Publishing.
Maskur, R., Sumarno, S., Rahmawati, Y., Pradana, K., Syazali, M., Septian, A., & Kinarya Palupi, E. (2020). The effectiveness of problem based learning and aptitude treatment interaction in improving mathematical creative thinking skills on Curriculum 2013. European Journal of Educational Research, 9(1), 375-383. https://doi.org/10.12973/eu-jer.9.1.375
Matters, G. (2009). A problematic leap in the use of test data: From performance to inference. In C. Wyatt-Smith & J. J. Cumming (Eds.), Educational assessment in the 21st century (pp. 209-225). Springer. https://doi.org/10.1007/978-1-4020-9964-9.
Naga, D. S. (2012). Teori sekor pada pengukuran mental [Theory of scores on mental measurement]. Nagarani Citrayasa.
Nitko, A. J., & Brookhart, S. M. (2007). Educational assessment of students. Pearson Education, Inc.
Nunnally, J. C., & Bernstein, I. (1978). Psychometric testing (3rd ed.). McGraw-Hill.
Ojose, B. (2011). Mathematics literacy: Are we able to put the mathematics we learn into everyday use. Journal of Mathematics Education, 4(1), 89-100.
Organisation for Economic Co-operation and Development (2013). PISA 2015 Draft Mathematics Framework. OECD.
Organisation for Economic Co-operation and Development (2016). Program for international students assessment (PISA) results from PISA 2015. OECD.
Owens, J. E. (1988, August). Curriculum and evaluation standards for school mathematics: Report of the national council of teachers of mathematics' commission on standards for school mathematics. Paper presented at the Summer Workshop of The Association of Teacher Educators. The University of Alabama.
Permendikbud (2018). Peraturan menteri pendidikan dan kebudayaan nomor 37 tahun 2018 tentang perubahan atas peraturan menteri pendidikan dan kebudayaan nomor 24 tahun 2016 tentang kompetensi inti dan kompetensi dasar pelajaran pada kurikulum 2013 pada pendidikan dasar dan menengah [Minister of education and culture regulation number 37 of 2018 concerning amendment to the minister of education and culture regulation number 24 of 2016 concerning core competencies and basic competencies of learning in the 2013 curriculum in primary and secondary education]. Ministry of Education and Culture.
Retnawati, H. (2015). Validitas reliabilitas dan karakteristik butir [Validity, reliability and item characteristics]. Parama Publishing.
Stacey, K., & Turner, R. (2015). The evolution and key concepts of the PISA mathematics frameworks. In K. Stacey & R. Turner (Eds.), Assessing mathematical literacy (pp. 5-33). Springer international Publishing.
Stanley, J. C. (1971). Reliability. In R. L. Thorndike (Ed.), Educational measurement (2nd ed.). American Council and Education.
Steen, L. A., Turner, R., & Burkhardt, H. (2007). Developing mathematical literacy. In B. W., G. P.L., H. HW. & N. M (Eds.), Modelling and applications in mathematics education. New ICMI Study Series, vol 10 (pp. 285-294). Springer.