CSEdPad

Investigating and Scaffolding Students' Mental Models during Computer Programming Tasks to Improve Learning, Engagement, and Retention

In this project, we explore several ideas including:

Student self-explanations when working with solved programming exercises by gauging their similarity with expected expert self-explanations [1].

About the project

This project is a joint effort of research teams at the University of Memphis and the University of Pittsburgh to explore the process of code comprehension by students and build a AI-based tools to support practice and assessment of code comprehension. The project was funded by the National Science Foundation by Award 1822752 between 2018 and 2023. In the course of the project, we developed and evaluated open-learned modeling approaches, code-tracing tools to practice code comprehension, and an intelligent tutor for assessing code comprehension through explanations.

Motivation

Computing skills, such as computer programming, are an integral part of many disciplines, including the fields of science, technology, engineering, and math (STEM). Although such skills are in high demand, and the number of aspiring Computer Science (CS) students is encouraging, a large gap between the supply of CS graduates and the demand persists because, for instance, college CS programs suffer from high attrition rates in introductory CS courses. One reason for the high attrition rates in introductory CS courses is the inherent complexity of CS concepts and tasks. To help students better cope with the high level of complexity, this project investigates a novel education technology, called CSEdPad (CS Education Pad), meant to ease students' introduction to programming during their early encounters with CS concepts and tasks. Moreover, the project forges new frontiers in CS education through a research program that advances our understanding of students' source code comprehension, learning, and motivational processes. The CSEdPad project has the potential to transform how students perceive computer science, increase their programming skills and self-efficacy, and lead to increased retention rates. The result will be a win-win-win situation for aspiring students, CS programs and their organizations, and the overall economy.

The CSEdPad system design brings to bear proven educational technologies and techniques to improve students' mental model construction, learning, engagement, and retention in CS education. In particular, the system targets source code comprehension, a critical skill for both learners and professionals. It monitors and scaffolds source code comprehension processes while students engage in a variety of code comprehension tasks. Key approaches being explored include Animated Pedagogical Agents, self-explanation, and the Open Social Learner Model. Outcome variables include comprehension measures, learning gains, engagement level, retention, and self-efficacy. Due to its interdisciplinary nature, the project will impact several fields including Computer Science education, cognitive psychology, intelligent tutoring systems, and artificial intelligence. Students participating in the experiments will be selected from a diverse student body with respect to gender, ethnicity, and socioeconomic status. An increase in recruitment and retention of students from these populations will have far-reaching implications.

Teams

University of Pittsburgh Team

• PIs: Vasile Rus, Peter Brusilovsky

• Graduate researchers: Zak Risha, Kamil Akhuseyinoglu, Arun LR

University of Memphis Team

• PI: Vasile Rus

• Graduate researchers: Lasang Tamang, Priti Oli, Jeevan Chapagian, Rabin Banjade

Publications

• Tamang, L. J., Alshaikh, Z., Khayi, N. A., and Rus, V. (2020) The Effects of Open Self-Explanation Prompting During Source Code Comprehension. In: Proceedings of The Thirty-Third International Florida Artificial Intelligence Research Society Conference (FLAIRS-32), Miami, FL, Association for the Advancement of Artificial Intelligence, pp. 451-456.
• Alshaikh, Z., Tamang, L. J., and Rus, V. (2020) Experiments with a Socratic Intelligent Tutoring System for Source Code Understanding. In: Proceedings of The Thirty-Third International Florida Artificial Intelligence Research Society Conference (FLAIRS-32), Miami, FL, Association for the Advancement of Artificial Intelligence, pp. 457-460.
• Ait Khayi, N. and Rus, V. (2020) Attention Based Transformer for Student Answers Assessment. In: Proceedings of The Thirty-Third International Florida Artificial Intelligence Research Society Conference (FLAIRS-32), Miami, FL, Association for the Advancement of Artificial Intelligence, pp. 3-8.
• Risha, Z., Barria-Pineda, J., Akhuseyinoglu, K., and Brusilovsky, P. (2021) Stepwise Help and Scaffolding for Java Code Tracing Problems With an Interactive Trace Table. In: Proceedings of 21st Koli Calling International Conference on Computing Education Research, ACM, pp. 1-10.
• Rus, V., Akhuseyinoglu, K., Chapagain, J., Tamang, L., and Brusilovsky, P. (2021) Prompting for Free Self-Explanations Promotes Better Code Comprehension. . In: Proceedings of 5th Educational Data Mining in Computer Science Education (CSEDM) Workshop at EDM2021, Paris, France, Julne 29, 2021, CEUR.
• Chapagain, J., Tamang, L., Banjade, R., Oli, P., and Rus, V. (2022) Automated Assessment of Student Self-explanation During Source Code Comprehension. In: Proceedings of The Thirty-Fifth International Florida Artificial Intelligence Research Society Conference (FLAIRS-35), Jensen Beach, FL, May 15-18, 2022, Association for the Advancement of Artificial Intelligence.
• Rus, V., Brusilovsky, P., Tamang, L. J., Akhuseyinoglu, K., and Fleming, S. (2022) DeepCode: An Annotated Set of Instructional Code Examples to Foster Deep Code Comprehension and Learning. In: S. Crossley and E. Popescu (eds.) Proceedings of 18th International Conference on Intelligent Tutoring Systems, ITS 2022, Bucharest, Romania, June 29 - July 1, 2022, Springer International Publishing, pp. 36--50.
• Banjade, R., Oli, P., Tamang, L. J., and Rus, V. (2022) Preliminary Experiments with Transformer based Approaches To Automatically Inferring Domain Models from Textbooks. In: A. Mitrovic and N. Bosch (eds.) Proceedings of the 15th International Conference on Educational Data Mining (EDM 2022), Durham, UK, July 24-27, 2022, pp. 667--672.
• Tamang, L. J., Banjade, R., Chapagain, J., and Rus, V. (2022) Automatic Question Generation for Scaffolding Self-explanations for Code Comprehension. In: M. M. Rodrigo, N. Matsuda, A. I. Cristea and V. Dimitrova (eds.) Proceedings of 23rd International Conference on Artificial Intelligence in Education, AIED 2022, Part 1, Durham, UK, July 27–31, 2022, Springer, pp. 743-748.
• Oli, P., Rus, V., Banjade, R., Narayanan, A. L., and Brusilovsky, P. (2023) When is reading more effective than tutoring? An analysis through the lens of students' self-efficacy among novices in computer science. . In: Proceedings of 7th Educational Data Mining in Computer Science Education (CSEDM) Workshop at LAK 2023, Arlington, TX, March 13, 2023.
• Chapagain, J., Risha, Z., Banjade, R., Oli, P., Tamang, L., Brusilovsky, P., and Rus, V. (2023) SelfCode: An Annotated Corpus and a Model for Automated Assessment of Self-Explanation During Source Code Comprehension. In: Proceedings of The Thirty-Sixth International Florida Artificial Intelligence Research Society Conference (FLAIRS-36), Clearwater Beach, FL, May 14-17, 2022, Association for the Advancement of Artificial Intelligence.
• Oli, P., Banjade, R., Lekshmi Narayanan, A. B., Chapagain, J., Tamang, L. J., Brusilovsky, P., and Rus, V. (2023) Improving Code Comprehension Through Scaffolded Self-explanations In: N. Wang, G. Rebolledo-Mendez, V. Dimitrova, N. Matsuda and O. C. Santos (eds.) Proceedings of 24th International Conference on Artificial Intelligence in Education, AIED 2023, Part 2, Tokyo, Japan, July 3–7, 2023, Springer, pp. 478-483.
• Lekshmi-Narayanan, A.-B., Oli, P., Chapagain, J., Hassany, M., Banjade, R., Brusilovsky, P., and Rus, V. (2024) Explaining Code Examples in Introductory Programming Courses: LLM vs Humans. In: Proceedings of Workshop on AI for Education - Bridging Innovation and Responsibility at AAAI 2024, , Vancouver, Canada, February 26, 2024.
• Oli, P., Banjade, R., Lekshmi Narayanan, A. B., Brusilovsky, P., and Rus, V. (2024) Exploring The Effectiveness of Reading vs. Tutoring For Enhancing Code Comprehension For Novices. In: Proceedings of ACM/SIGAPP Symposium on Applied Computing, SAC 2024, Avila, Spain, April 8-12, 2024.
• Oli, P., Banjade, R., Chapagain, J., and Rus, V. (2023) The Behavior of Large Language Models When Prompted to Generate Code Explanations. In: Proceedings of Proceedings of the workshop on Generative AI for Education (GAIED) at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, December 2023.