TY - GEN
T1 - Optimizing Large Language Models for Auto-Generation of Programming Quizzes
AU - Kumar, Yulia
AU - Manikandan, Anjana
AU - Li, J. Jenny
AU - Morreale, Patricia
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - This study analyzes the use of Large Language Models (LLMs) like ChatGPT in creating quizzes for Java programming courses, specifically Object-Oriented Programming (CS1) and Data Structures (CS2). It aims to evaluate the accuracy of LLM-generated assessments, understand the benefits and drawbacks of using LLMs in CS education from educators' viewpoints, and identify effective prompt engineering strategies to enhance the quality of educational materials. The research compares quizzes made by LLMs against human-created content to assess their consistency with Java programming principles, alignment with CS1 and CS2 learning goals, and their impact on student engagement and comprehension, providing insights into LLMs' effectiveness in academic assessment creation for computer science education.
AB - This study analyzes the use of Large Language Models (LLMs) like ChatGPT in creating quizzes for Java programming courses, specifically Object-Oriented Programming (CS1) and Data Structures (CS2). It aims to evaluate the accuracy of LLM-generated assessments, understand the benefits and drawbacks of using LLMs in CS education from educators' viewpoints, and identify effective prompt engineering strategies to enhance the quality of educational materials. The research compares quizzes made by LLMs against human-created content to assess their consistency with Java programming principles, alignment with CS1 and CS2 learning goals, and their impact on student engagement and comprehension, providing insights into LLMs' effectiveness in academic assessment creation for computer science education.
KW - AI-Supplemental Instructor (AI-SI)
KW - Java programming instruction
KW - use of LLMs in CS education
UR - http://www.scopus.com/inward/record.url?scp=85205549035&partnerID=8YFLogxK
U2 - 10.1109/ISEC61299.2024.10665141
DO - 10.1109/ISEC61299.2024.10665141
M3 - Conference contribution
AN - SCOPUS:85205549035
T3 - 2024 IEEE Integrated STEM Education Conference, ISEC 2024
BT - 2024 IEEE Integrated STEM Education Conference, ISEC 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 14th IEEE Integrated STEM Education Conference, ISEC 2024
Y2 - 9 March 2024
ER -