logo logo European Journal of Educational Research

EU-JER is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
7321 Parkway Drive South, Hanover, MD 21076, USA
Eurasian Society of Educational Research
Headquarters
7321 Parkway Drive South, Hanover, MD 21076, USA
critical thinking learning management approaches meta analysis propensity score matching research synthesis

Developing Students' Critical Thinking: Examining the Influence of Learning Management Approaches Through Meta-Analysis and Propensity Score Matching

Supansa Surin , Suntonrapot Damrongpanit

Critical thinking is a skill that enables individuals to keep pace with changes and enhances crucial competencies for contemporary competitiveness. Ma.

C

Critical thinking is a skill that enables individuals to keep pace with changes and enhances crucial competencies for contemporary competitiveness. Many researchers have studied learning management approaches to develop students' critical thinking, resulting in a substantial body of knowledge but lacking clear systematic summaries. The researchers aimed to (a) examine the effect sizes and research characteristics influencing students' critical thinking, and (b) compare the effect sizes of learning management approaches after adjusting with propensity score matching from 108 graduate research published between 2002 and 2021. Data were collected using research characteristics recording forms and research quality assessment questionnaires. Effect sizes were calculated using Glass's method and analyzed through random effect, fixed effect, and regression meta-analysis. Findings revealed that (a) research on developing learning management approaches influences students' critical thinking at a high level (d ̅ = 1.669), with nine research characteristics, including the field of publication, courses, total duration, teacher learning process, learning media, measurement and evaluation, research design, research statistics, and research quality, statistically significantly influencing students' critical thinking, and (b) after adjustment, inquiry-based learning significantly influences students' critical thinking. Recommendations for developing students' critical thinking include learning activities that encourage problem exploration, expanding thinking through collaborative analysis, and applying diverse media and activity sheets tailored to context suitability.
 

Keywords: Critical thinking, learning management approaches, meta-analysis, propensity score matching, research synthesis.

cloud_download PDF
Cite
Article Metrics
Views
60
Download
144
Citations
Crossref
0

Scopus
0

References

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(2), 275-314. https://doi.org/10.3102/0034654314551063

Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102-1134. https://doi.org/10.3102/0034654308326084

Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the use of tests: A meta-analysis of practice testing. Review of Educational Research, 87(3), 659-701. https://doi.org/10.3102/0034654316689306

Akinbadewa, B. O., & Sofowora, O. A. (2020). The effectiveness of multimedia instructional learning packages in enhancing secondary school students’ attitudes toward Biology. International Journal on Studies in Education, 2(2), 119-133. https://doi.org/10.46328/ijonse.19

Allen, M. (2020). Understanding the practice, application, and limitations of meta-analysis. American Behavioral Scientist, 64(1), 74-96. https://doi.org/10.1177/0002764219859619

Alsaleh, N. J. (2020). Teaching critical thinking skills: Literature review. Turkish Online Journal of Educational Technology, 19(1), 21-39. https://bit.ly/49QkEcI

Amineh, R. J., & Asl, H. D. (2015). Review of constructivism and social constructivism. Journal of Social Sciences, Literature and Languages, 1(1), 9-16. http://bit.ly/3v23J8v

Austin, P. C. (2009). The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Medical Decision Making, 29(6), 661-677. https://doi.org/10.1177/0272989X09341755

Badhiwala, J. H., Karmur, B. S., & Wilson, J. R. (2021). Propensity score matching: A powerful tool for analyzing observational nonrandomized data. Clinical Spine Surgery, 34(1), 22-24. https://doi.org/10.1097/BSD.0000000000001055

Bai, H. (2011). A comparison of propensity score matching methods for reducing selection bias. International Journal of Research and Method in Education, 34(1), 81-107. https://doi.org/10.1080/1743727X.2011.552338

Balta, N., & Sarac, H. (2016). The effect of 7e learning cycle on learning in science teaching: a meta-analysis study. European Journal of Educational Research, 5(2), 61-72. https://doi.org/10.12973/eu-jer.5.2.61

Benedetto, U., Head, S. J., Angelini, G. D., & Blackstone, E. H. (2018). Statistical primer: Propensity score matching and its alternatives. European Journal of Cardio-Thoracic Surgery, 53(6), 1112-1117. https://doi.org/10.1093/ejcts/ezy167

Berkhout, S. W., Haaf, J. M., Gronau, Q. F., Heck, D. W., & Wagenmakers, E.-J. (2024). A tutorial on Bayesian model-averaged meta-analysis in JASP. Behavior Research Methods, 56, 1260-1282. https://doi.org/10.3758/s13428-023-02093-6

Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (Eds.). (2021). Introduction to meta-analysis. John Wiley and Sons. https://bit.ly/3SGZV4M

Borenstein, M., & Higgins, J. P. T. (2013). Meta-analysis and subgroups. Prevention Science, 14, 134-143. https://doi.org/10.1007/s11121-013-0377-7

Card, N. A. (2012). Applied meta-analysis for social science research. The Guilford Press. https://bit.ly/3HWcMuP

Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., & Lee, C. W. Y. (2017). 21st century skills development through inquiry-based learning: From theory to practice. Springer. https://doi.org/10.1007/978-981-10-2481-8

Cooper, H., Hedges, L. V., & Valentine, J. C. (2019). The handbook of research synthesis and meta-analysis (3rd ed.). Russell Sage Foundation. https://doi.org/10.7758/9781610448864

Czodrowski, P. (2014). Count on kappa. Journal of Computer-Aided Molecular Design, 28, 1049-1055. https://doi.org/10.1007/s10822-014-9759-6

Donoghue, G. M., & Hattie, J. A. C. (2021). A meta-analysis of ten learning techniques. Frontiers in Education, 6, Article 581216. https://doi.org/10.3389/feduc.2021.581216

Dowdy, A., Tincani, M., & Schneider, W. J. (2020). Evaluation of publication bias in response interruption and redirection: A meta‐analysis. Journal of Applied Behavior Analysis, 53(4), 2151-2171. https://doi.org/10.1002/jaba.724

Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking skills and Creativity, 12, 43-52. https://doi.org/10.1016/j.tsc.2013.12.004

Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315, 629-634. https://doi.org/10.1136/bmj.315.7109.629

Eisenkraft, A. (2003). Expanding the 5e model. The Science Teacher, 70(6), 56-59. https://bit.ly/3GLas9A

Elder, L., & Paul, R. (2021). Critical thinking: Tools for taking charge of your learning and your life (4th ed.). Rowman & Littlefield.

Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron & R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). W H Freeman/Times Books/ Henry Holt & Co.

Ennis, R. H. (1993). Critical thinking assessment. Theory Into Practice, 32(3), 179-186. https://doi.org/10.1080/00405849309543594

Ennis, R. H. (2015). Critical thinking: A streamlined conception. In M. Davies, & R. Barnett, (Eds.), The Palgrave handbook of critical thinking in higher education (pp. 31-47). Palgrave Macmillan. https://doi.org/10.1057/9781137378057_2

Ennis, R. H. (2018). Critical thinking across the curriculum: A vision. Topoi, 37, 165-184. https://doi.org/10.1007/s11245-016-9401-4

Esterhuizen, T. M., & Thabane, L. (2016). Con: Meta-analysis: some key limitations and potential solutions. Nephrology Dialysis Transplantation, 31(6), 882-885. https://doi.org/10.1093/ndt/gfw092

Facione, P. A. (2023). Critical thinking: What it is and why it counts. Insight Assessment. https://bit.ly/44g6a4L

Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17(1), 120-128. https://doi.org/10.1037/a0024445

Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3-8. https://doi.org/10.3102/0013189X005010003

Green, P. (2015). Teaching critical thinking for lifelong learning. In M. Davies, & R. Barnett (Eds.), The Palgrave handbook of critical thinking in higher education (pp. 107-121). Palgrave Macmillan. https://doi.org/10.1057/9781137378057_7

Harris, H., & Horst, S. J. (2016). A brief guide to decisions at each step of the propensity score matching process, Practical Assessment, Research, and Evaluation, 21(4), 1-11. https://doi.org/10.7275/yq7r-4820

Harrison, F. (2011). Getting started with meta‐analysis. Methods in Ecology and Evolution, 2(1), 1-10. https://doi.org/10.1111/j.2041-210X.2010.00056.x

Haukoos, J. S., & Lewis, R. J. (2015). The propensity score. JAMA, 314(15), 1637-1638. https://doi.org/10.1001/jama.2015.13480

Heinrich, C., Maffioli, A., & Vazquez, G. (2010). A primer for applying propensity-score matching. Inter-American Development Bank. https://bit.ly/3HXEuHw

Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539-1558. https://doi.org/10.1002/sim.1186

Hitchcock, D. (2017). Critical thinking as an educational ideal. In On reasoning and argument. (Vol. 30, pp. 477-497). Springer. https://doi.org/10.1007/978-3-319-53562-3_30

Indah, R. N., & Kusuma, A. W. (2016). Factors affecting the development of critical thinking of Indonesian learners of English language. Journal of Humanities and Social Science, 21(6), 86-94. https://bit.ly/4aoSkQg

Itsarangkul Na Ayutthaya, T., & Damrongpanit, S. (2022a). A meta-analysis of instructional management models affecting creative thinking development. European Journal of Educational Research, 11(4), 2069-2085. https://doi.org/10.12973/eu-jer.11.4.2069

Itsarangkul Na Ayutthaya, T., & Damrongpanit, S. (2022b). A meta-analysis of instructional management model for students' creative thinking development: An application of propensity score matching. European Journal of Educational Research, 11(4), 2429-2444. https://doi.org/10.12973/eu-jer.11.4.2429

Jacobs, G. M., & Renandya, W. A. (2019). Student centered cooperative learning: Linking concepts in education to promote student learning. Springer. https://doi.org/10.1007/978-981-13-7213-1

Kane, L. T., Fang, T., Galetta, M. S., Goyal, D. K. C., Nicholson, K. J., Kepler, C. K., Vaccaro, A. R., & Schroeder, G. D. (2020). Propensity score matching: a statistical method. Clinical Spine Surgery, 33(3), 120-122.  https://doi.org/10.1097/BSD.0000000000000932

Karampelas, K. (2023). Critical thinking in national primary science curricula. Eurasian Journal of Science and Environmental Education, 3(2), 51-60. https://doi.org/10.30935/ejsee/13271

Kennedy, T. J., & Sundberg, C. W. (2020). 21st century skills. In B. Akpan, & T. J. Kennedy (Eds.), Science education in theory and practice (pp. 479-496). Springer. https://doi.org/10.1007/978-3-030-43620-9_32

Kocak, O., Coban, M., Aydin, A., & Cakmak, N. (2021). The mediating role of critical thinking and cooperativity in the 21st century skills of higher education students. Thinking Skills and Creativity, 42, Article 100967. https://doi.org/10.1016/j.tsc.2021.100967

Lai, E. R. (2011). Critical thinking: A literature review. Research Reports. https://bit.ly/42zDaUG

Lambert, C. T., & Guillette, L. M. (2021). The impact of environmental and social factors on learning abilities: a meta‐analysis. Biological Reviews, 96(6), 2871-2889. https://doi.org/10.1111/brv.12783

Lampert, M., & Graziani, F. (2009). Instructional activities as a tool for teachers' and teacher educators' learning. The Elementary School Journal, 109(5), 491-509. https://doi.org/10.1086/596998

Lee, W.-S. (2013). Propensity score matching and variations on the balancing test. Empirical Economics, 44, 47-80. https://doi.org/10.1007/s00181-011-0481-0

Lee, Y. H. (2019). Strengths and limitations of meta-analysis. The Korean Journal of Medicine, 94(5), 391-395. https://doi.org/10.3904/kjm.2019.94.5.391

Mahapoonyanont, N. (2010). Factors related to critical thinking abilities; a meta-analysis. Procedia Social and Behavioral Sciences, 9, 986-990. https://doi.org/10.1016/j.sbspro.2010.12.272

McPeck, J. E. (Ed.). (2016). Critical thinking and education. Routledge. https://doi.org/10.4324/9781315463698

Moallem, M., Hung, W., & Dabbagh, N. (Eds.). (2019). The Wiley handbook of problem-based learning. John Wiley & Sons Inc. https://doi.org/10.1002/9781119173243

Morgan, C. J. (2018). Reducing bias using propensity score matching. Journal of Nuclear Cardiology, 25(2), 404-406. https://doi.org/10.1007/s12350-017-1012-y

Moust, J., Bouhuijs, P., & Schmidt, H. (2021). Introduction to problem-based learning (4th ed.). Routledge. https://doi.org/10.4324/9781003194187

Muthma’Innah, M., Dahlan, J. A., & Suhendra, S. (2019). Ability of mathematical critical thinking–what about learning cycle 7e model? Journal of Physics: Conference Series, 1157(3), Article 032129. https://doi.org/10.1088/1742-6596/1157/3/032129

Nicol, C., Yakuba, E., & Habinshuti, G. (2020). An overview of learning cycles in science inquiry-based instruction. African Journal of Educational Studies in Mathematics and Sciences, 16(2), 67-81. https://doi.org/10.4314/ajesms.v16i2.5

Niu, L., Behar-Horenstein, L. S., & Garvan, C. W. (2013). Do instructional interventions influence college students’ critical thinking skills? A meta-analysis. Educational Research Review, 9, 114-128. https://doi.org/10.1016/j.edurev.2012.12.002

Oliveira, L. B., Díaz, L. J., Carbogim, F. C., Rodrigues, A. R. B., & Püschel, V. A. A. (2016). Efetividade das estratégias de ensino no desenvolvimento do pensamento crítico de graduandos de Enfermagem: uma metanálise [Effectiveness of teaching strategies on the development of critical thinking in undergraduate nursing students: a meta-analysis]. Journal of School of Nursing – University of São Paulo/Revista da Escola de Enfermagem da USP, 50(2), 355-364. https://doi.org/10.1590/S0080-623420160000200023

O'Reilly, C., Devitt, A., & Hayes, N. (2022). Critical thinking in the preschool classroom-a systematic literature review.  Thinking Skills and Creativity, 46, Article 101110. https://doi.org/10.1016/j.tsc.2022.101110

Paul, R., & Elder, L. (2005). A guide for educators to critical thinking competency standards. Foundation for Critical Thinking. https://bit.ly/48gtYFY

Paul, R., & Elder, L. (2020). The miniature guide to critical thinking concepts and tools. Rowman and Littlefield. https://bit.ly/3OHHBHr

Rannikmäe, M., Holbrook, J., & Soobard, R. (2020). Social constructivism—Jerome Bruner. In B. Akpan, & T. J. Kennedy (Eds.), Science education in theory and practice (pp. 259-275). Springer. https://doi.org/10.1007/978-3-030-43620-9_18

Reiffel, J. A. (2020). Propensity score matching: the ‘devil is in the details’ where more may be hidden than you know. American Journal of Medicine, 133(2), 178-181. https://doi.org/10.1016/j.amjmed.2019.08.055

Ripley, D. (2015). An examination of flipped instructional method on sixth graders’ mathematics learning: Utilizing propensity score matching. University of Nevada. https://bit.ly/49cwezm

Rosenbaum, P. R. (2020). Modern algorithms for matching in observational studies. Annual Review of Statistics and Its Application, 7, 143-176. https://doi.org/10.1146/annurev-statistics-031219-041058

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55. https://doi.org/10.1093/biomet/70.1.41

Rothstein, H. R. (2008). Publication bias as a threat to the validity of meta-analytic results. Journal of Experimental Criminology, 4, 61-81. https://doi.org/10.1007/s11292-007-9046-9

Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127(8-2), 757-763. https://doi.org/29d

Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008). Undue reliance on /2 in assessing heterogeneity may mislead. BMC Medical Research Methodology, 8, Article 79. https://doi.org/10.1186/1471-2288-8-79

Sainani, K. L. (2012). Propensity scores: uses and limitations. PM&R, 4(9), 693-697. https://doi.org/10.1016/j.pmrj.2012.07.002

Saiz, C., & Rivas, S. F. (2016). New teaching techniques to improve critical thinking. The Diaprove methodology. Educational Research Quarterly, 40(1), 3-36.

Saylor, J. G., Alexander, W. M., & Lewis, A. J. (1981). Curriculum planning for better teaching and learning (4th ed.). Holt, Rinehart and Winston.

Sedgwick, P., & Marston, L. (2015). How to read a funnel plot in a meta-analysis. BMJ, 351, Article h4718. https://doi.org/10.1136/bmj.h4718

Slavin, R. E. (2014). Aprendizaje cooperativo y rendimiento académico: ¿por qué funciona el trabajo en grupo? [Cooperative learning and academic achievement: Why does groupwork work?]. Annals of Psychology/Anales De Psicología, 30(3), 785-791. https://doi.org/10.6018/analesps.30.3.201201

Staffa, S. J., & Zurakowski, D. (2018). Five steps to successfully implement and evaluate propensity score matching in clinical research studies. Anesthesia and Analgesia, 127(4), 1066-1073. https://doi.org/10.1213/ANE.0000000000002787

StataCorp. (2023). Stata 18 meta-analysis reference manual: Release 18. Stata Press. https://www.stata.com/manuals/meta.pdf

Sternberg, R. J. (1986). Critical thinking: Its nature, measurement, and improvement. National Inst. of Education.

Stone, D. L., & Rosopa, P. J. (2017). The advantages and limitations of using meta-analysis in human resource management research. Human Resource Management Review, 27(1), 1-7. https://doi.org/10.1016/j.hrmr.2016.09.001

Suardana, I. N., Redhana, I. W., Sudiatmika, A. A. I. A. R., & Selamat, I. N. (2018). Students’ critical thinking skills in chemistry learning using local culture-based 7e learning cycle model. International Journal of Instruction, 11(2), 399-412. https://doi.org/10.12973/iji.2018.11227a

Tang, J.-L., & Liu, J. L. (2000). Misleading funnel plot for detection of bias in meta-analysis. Journal of Clinical Epidemiology, 53(5), 477-484. https://doi.org/10.1016/S0895-4356(99)00204-8

Thongnuypram, C., & Sopheerak, S. (2013). Factor influencing the critical thinking of teacher students studying at the faculty of education in Suratthani Rajabhat University. Procedia-Social and Behavioral Sciences, 103, 386-391. https://doi.org/10.1016/j.sbspro.2013.10.349

Thornton, A., & Lee, P. (2000). Publication bias in meta-analysis: its causes and consequences. Journal of Clinical Epidemiology, 53(2), 207-216. https://doi.org/10.1016/S0895-4356(99)00161-4

Van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced methods in meta‐analysis: multivariate approach and meta‐regression. Statistics in Medicine, 21(4), 589-624. https://doi.org/10.1002/sim.1040

Varoglu, L., Yilmaz, A., & Sen, S. (2023). Effect of 5E learning cycle assisted with concept maps on conceptual understanding. Pedagogical Research, 8(3), Article em0161. https://doi.org/10.29333/pr/13167

Vevea, J. L., Coburn, K. M., & Sutton, A. J. (2019). Publication bias. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (3rd ed., pp. 383-429). Russell Sage Foundation, https://doi.org/10.7758/9781610448864.21

Xu, E., Wang, W., & Wang, Q. (2023). The effectiveness of collaborative problem solving in promoting students’ critical thinking: A meta-analysis based on empirical literature. Humanities and Social Sciences Communications, 10, Article 16. https://doi.org/10.1057/s41599-023-01508-1

Zhao, C., Pandian, A., & Singh, M. K. M. (2016). Instructional strategies for developing critical thinking in EFL classrooms. English Language Teaching, 9(10), 14-21. https://doi.org/10.5539/elt.v9n10p14

...