Blogs (3) >>
Sat 23 Mar 2024 11:10 - 11:35 at Meeting Room E146 - LLM - Online Chair(s): Christopher Nitta

Teachers often utilize automatic program repair methods to provide feedback on submitted student code using model answer code. A state-of-the-art tool is Refactory, which achieves a high repair success rate and small patch size (less code repair) by refactoring code to expand the variety of correct code samples that can be referenced. However, Refactory has two major limitations. First, it cannot fix code with syntax errors. Second, it has difficulty fixing code when there are few correct submissions. Herein we propose a new method that combines Refactory and OpenAI’s GPT models to address these issues and conduct a performance measurement experiment. The experiment uses a dataset consisting of 5 programming assignment problems and almost 1,800 real-life incorrect Python program submissions from 361 students for an introductory programming course at a large public university. The proposed method improves the repair success rate by 1–21% when the set of correct code samples is sufficient and the patch size is smaller than Refactory alone in 16-45% of the cases. When there was no set of correct code samples at all (only the model answer code was used as a reference for repair), method improves the repair success rate by 1-43% and the patch size is smaller than Refactory alone in 42-68% of the cases.

Sat 23 Mar

Displayed time zone: Pacific Time (US & Canada) change

10:45 - 12:00
LLM - OnlinePapers at Meeting Room E146
Chair(s): Christopher Nitta University of California, Davis
10:45
25m
Talk
A Self-Regulated Learning Framework using Generative AI and its Application in CS Educational Intervention DesignOnlineGlobalIn-Person
Papers
Prajish Prasad FLAME University, Aamod Sane FLAME University
DOI
11:10
25m
Talk
Improvement in Program Repair Methods using Refactoring with GPT ModelsOnlineGlobalIn-Person
Papers
Ryosuke Ishizue NTT DATA Group Corporation / Waseda University, Kazunori Sakamoto WillBooster Inc. / Tokyo Online Unicersity / Waseda University, Hironori Washizaki Waseda University, Yoshiaki Fukazawa Waseda University
DOI
11:35
25m
Talk
Using GPT-4 to Provide Tiered, Formative Code FeedbackOnlineIn-Person
Papers
Ha Nguyen Utah State University, Vicki Allan Utah State University
DOI