Blogs (4) >>
Fri 22 Mar 2024 11:35 - 12:00 at Oregon Ballroom 204 - LLM - tools Chair(s): Geoffrey Herman

As the prominence of Large Language Models (LLMs) grows in various sectors, their potential in the educational realm warrants exploration. This study investigates the feasibility of employing GPT-3.5, an LLM developed by OpenAI, as a virtual teaching assistant (TA) in computer science courses. The primary objective is to enhance the accessibility of computer science education while maintaining academic integrity by refraining from providing direct solutions to assignment questions. To achieve this goal, a virtual TA was developed using LangChain, a sophisticated and data-aware framework for applications powered by language models, known for its ability to interact with diverse data sources and environments. We selected GPT-3.5 for its exceptional contextual understanding and capability to generate nuanced responses, ensuring its suitability for the task. The study focused on the Foundations of Programming undergraduate course (CS1), where the virtual TA was designed to provide coding feedback, address student inquiries, and clarify intricate concepts. Our findings revealed that the virtual TA outperformed human TAs in providing more detailed responses to sample student questions. The system successfully abstained from disclosing current semester assignment solutions. However, in cases where specific assignment requirements were queried, human TAs proved more reliable. Our research demonstrates that virtual TAs hold great promise for enriching computer science education, but their optimal use necessitates human supervision. By leveraging the capabilities of LLMs within the LangChain framework, we can pave the way for a more accessible and effective educational experience in the digital era.

Fri 22 Mar

Displayed time zone: Pacific Time (US & Canada) change

10:45 - 12:00
LLM - toolsPapers at Oregon Ballroom 204
Chair(s): Geoffrey Herman University of Illinois at Urbana-Champaign
10:45
25m
Talk
Evaluating Automatically Generated Contextualised Programming ExercisesGlobal
Papers
Andre del Carpio Gutierrez The University of Auckland, Paul Denny The University of Auckland, Andrew Luxton-Reilly The University of Auckland
DOI
11:10
25m
Talk
A Fast and Accurate Machine Learning Autograder for the Breakout Assignment
Papers
Evan Liu Stanford University, David Yuan Stanford University, Syed Ahmed Oakland University, Elyse Cornwall Stanford University, Juliette Woodrow Stanford University, Kaylee Burns Stanford University, Allen Nie Stanford University, Emma Brunskill Stanford University, Chris Piech Stanford University, Chelsea Finn Stanford University
DOI
11:35
25m
Talk
Beyond Traditional Teaching: Designing a virtual teaching assistant using LLMs for CS educationGlobal
Papers
Mengqi Liu Mcgill university, Faten M'Hiri Mcgill university
DOI