Spotting AI Missteps: Students Take on LLM Errors in CS1 (SIGCSE TS 2025 - Posters)

Who

Samantha Smith, Heather Wei, Abby O'Neill, Aneesh Durai, John DeNero, J.D. Zamfirescu-Pereira, Narges Norouzi

Track

SIGCSE TS 2025 Posters

This program is tentative and subject to change.

Time Zone

The program is currently displayed in (GMT-05:00) Eastern Time (US & Canada).

Use conference time zone: (GMT-05:00) Eastern Time (US & Canada)Select other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

By setting a time band, the program will dim events that are outside this time window. This is useful for (virtual) conferences with a continuous program (with repeated sessions).
The time band will also limit the events that are included in the personal iCalendar subscription service.

Display full programSpecify a time band

Save

When

Thu 27 Feb 2025 10:00 - 12:00 at Exhibit Hall C - Posters - Posters #1

Abstract

Large Language Model (LLM)-based assistance has seen rapid adoption in CS1 courses, promising quick and always-available support for homework help. However, a significant challenge lies in ensuring that the hints and guidance provided by these models are accurate and not based on hallucinated solutions. In this work, we study and categorize LLM behavior in cases where students believe an LLM-powered homework bot has given an inaccurate hint. We then demonstrate how certain student behaviors are correlated with our suggested bot-behavior categories.

Samantha Smith

UC Berkeley

United States

Heather Wei

University of California, Berkeley

United States

Abby O'Neill

University of California, Berkeley

United States

Aneesh Durai

UC Berkeley

United States

John DeNero

UC Berkeley

United States

J.D. Zamfirescu-Pereira

UC Berkeley

United States

Narges Norouzi

University of California, Berkeley

United States

This program is tentative and subject to change.

Time Zone

The program is currently displayed in (GMT-05:00) Eastern Time (US & Canada).

Use conference time zone: (GMT-05:00) Eastern Time (US & Canada)Select other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

Display full programSpecify a time band

Save

Session Program

Thu 27 Feb
Displayed time zone: Eastern Time (US & Canada) change

10:00 - 12:00	Posters #1Posters at Exhibit Hall C - Posters

10:00 2h Talk		Overcoming Illusionary Difficulty in Novice Programming through Metacognitive Skill Enhancement Posters Zhen Jiang West Chester University, Jie Wu Temple University, Nora Jiang Swarthmore College
10:00 2h Talk		An Interactive Tool for Randomized Autogradable Graph Assessments Posters Eldar Hasanov UC Berkeley, Dev Ahluwalia University of California, Berkeley, Daniel Garcia University of California Berkeley, Narges Norouzi University of California, Berkeley, Armando Fox UC Berkeley
10:00 2h Talk		Comparing Students' and Teachers' Assessments of Office Hours Posters Zhikai Gao North Carolina State University, Saminur Islam North Carolina State University, Caleb Scott North Carolina State University, Collin Lynch North Carolina State University, Sarah Heckman North Carolina State University
10:00 2h Talk		Simulating Requirement Elicitation: Development and Evaluation of a Persona-Based Tool Posters Ildar Akhmetov Northeastern University, Mirjana Prpa Northeastern University
10:00 2h Talk		A Research-Oriented Course in Developing Tools to Teach AI Posters Fred Martin University of Texas at San Antonio, Deepti Tagare University of Texas at San Antonio, Ismaila Temitayo Sanusi University of Texas at San Antonio
10:00 2h Talk		How Should We Measure Race, Ethnicity, and Gender Equity in Undergraduate CS Programs? Posters Julie Smith Institute for Advancing Computing Education, Monica McGill Institute for Advancing Computing Education
10:00 2h Talk		Managing Project Teams in an Online Class of 1000+ Students Posters Nazanin Tabatabaei Anaraki Georgia Institute of Technology, Taneisha Ng Georgia Institute of Technology, Gaurav Verma Georgia Institute of Technology, Yu Fu Georgia Institute of Technology, Martin Oconnell Georgia Institute of Technology, Matthew Hull Georgia Institute of Technology, Susanta Routray Georgia Institute of Technology, Max Mahdi Roozbahani Georgia Institute of Technology, Duen Horng Chau Georgia Institute of Technology
10:00 2h Talk		Personalized Parsons Puzzles as Scaffolding Enhance Practice Engagement Over Just Showing LLM-Powered Solutions Posters Xinying Hou University of Michigan, Zihan Wu University of Michigan, Xu Wang University of Michigan, Barbara Ericson University of Michigan
10:00 2h Talk		How Good are Large Language Models at Generating Subgoal Labels? Posters Samiha Marwan University of Virginia, Mohamed Ibrahim Arab Academy for Science and Technology, Briana B. Morrison University of Virginia
10:00 2h Talk		Teaching Sustainable Computing Through Repair: Case Studies on Curriculum Design Posters Esther Roorda University of British Columbia, Sathish Gopalakrishnan University of British Columbia, Emily Shilton University of British Columbia
10:00 2h Talk		CS Student Reflections on Equity-Focused Classroom Practices Posters Nadia Najjar University of North Carolina at Charlotte, Debarati Basu Embry-Riddle Aeronautical University, Liam Custer University of North Carolina at Charlotte
10:00 2h Talk		Scaffolding Collaborative Software Design with Serious Games Posters Serena Caraco University of California, Berkeley, Melissa Fabros University of California, Berkeley, Nelson Lojo University of California, Berkeley, Armando Fox UC Berkeley
10:00 2h Talk		Student Retention through Peer-led CS1 Review Sessions: Motivation, Sense of Belonging, and Learning Gains Posters Dee Weikle James Madison University, Michael Stewart James Madison University, A. Carter Gilbert James Madison University, Cory Longenecker James Madison University
10:00 2h Talk		Exploring CS Education Policy Through the Lens of State Governance Models: Access, Accountability and Authority Posters Janice Mak Arizona State University, Carolina Torrejon Capurro Arizona State University, Megean Garvin UMBC
10:00 2h Talk		Analyzing Computational Thinking Gameplay: Identifying Struggles and the Role of Experience Posters Sotheara Veng University of Delaware, Yifan Zhang University of Nottingham Ningbo China, Ekaterina Bergwall University of Delaware, Teomara Rutherford University of Delaware
10:00 2h Talk		Sociotechnical AI Education Course Design for CS Majors and Non-Majors Posters Sri Yash Tadimalla UNC Charlotte, Mary Lou Maher University of North Carolina
10:00 2h Talk		Supporting Students at Scale: Profiling Student Behaviors on Usage and Impact of the Student Support System Posters Teresa Luo University of California, Berkeley, Chenkun Sheng University of California, Berkeley, Lisa Yan UC Berkeley
10:00 2h Talk		Checkpoint: A Tool for Supporting Terminal-Based Capture-the-Flag Assessments Posters Connor Bernard University of California, Berkeley, Melissa Fabros University of California, Berkeley, Zhifei Li Renmin University of China, Narges Norouzi University of California, Berkeley, Daniel Garcia University of California Berkeley, Armando Fox UC Berkeley
10:00 2h Talk		How Do Learners Use Scratch Paper When Working On Dynamic Programming Problems? Posters Zihan Wu University of Michigan, Jonathan Liu University of Chicago, Erica Goodwin University of Chicago, Diana Franklin University of Chicago
10:00 2h Talk		Reducing Isolation through Peer-Modeled Posts Posters Naaz Sibia University of Toronto Mississauga, Angela Zavaleta Bernuy University of Toronto, Amber Richardson University of Toronto Mississauga, Khushi Malik University of Toronto, Prajna Pendharkar University of Toronto Mississauga, Carolina Nobre University of Toronto, Michael Liut University of Toronto Mississauga, Andrew Petersen University of Toronto
10:00 2h Talk		Assessing Course Policy Impacts: Late Course Enrollment and Its Effects on Student Performance and Incomplete Grades Posters Victoria Phelps University of California, Berkeley, Anneliese Galler UC Berkeley, Jordan Schwartz UC Berkeley, Stacey Yoo UC Berkeley, Aananya Lakhani UC Berkeley, Daniel Garcia University of California Berkeley
10:00 2h Talk		Three-stage Learning with Portable Online Hands-on Labware for Quantum-based Machine Learning Development Posters Yong Shi Kennesaw State University, Dan Lo Kennesaw State University, Hongmei Chi Florida A&M University, Andrew Polisetty Kennesaw State University, Kun Suo Kennesaw State University, Tu Nguyen Kennesaw State University
10:00 2h Talk		Leveraging Various Social Media Platforms to Mentor Black Computing Students on Graduate School Opportunities Posters Kaylah Mackroy Morehouse College, Whitney Nelson Morehouse College, Kinnis Gosha Morehouse College
10:00 2h Talk		Crafting Opportunities: Establishing a Micro-Internship Program for Computing Students Posters Nimmi Arunachalam Florida International University, Stephanie Lunn Florida International University, Ashmita Thapaliya Florida International University, Giri Narasimhan Florida International University, Jason Liu Florida International University, Mark Weiss Florida International University
10:00 2h Talk		Co-designing Curriculum to Discuss Environmental Disparities Using Data Science Posters Dana Saito-Stehberger University of California, Irvine, Mark Warschauer University of California, Irvine
10:00 2h Talk		No Average User: Covering Accessibility and Inclusion in an HCI Course Posters Andrea Salgian The College of New Jersey
10:00 2h Talk		Broadening Participation in CS Research with Scalable Undergraduate Research Mini-Projects Posters Bridget Agyare UC Berkeley, Manooshree Patel UC Berkeley, Alicia Matsumoto UC Berkeley, Gireeja Ranade University of California, Berkeley
10:00 2h Talk		An LLM-Based Framework for Simulating, Classifying, and Correcting Students’ Programming Knowledge with the SOLO Taxonomy Posters Shan Zhang University of Florida, Pragati Meshram University of Illinois Urbana-Champaign, Priyadharshini Ganapathy Prasad University of Florida, Maya Israel University of Florida, Suma Bhat University of Illinois Urbana-Champaign
10:00 2h Talk		Is ACM-W Still Relevant After 30 Years? Exploring the Ongoing Needs of Women and URMs in Computing Posters Shelly Engelman Custom EduEval LLC, Rachelle Hippler Baldwin Wallace University, Andrew Watkins Case Western Reserve University, Patricia Opong Columbus State Community College, Natalie Nurse Cuyahoga Community College, Dave Reed Creighton University
10:00 2h Talk		A Systematic Literature Mapping of Early Generative AI Research is CS Education Posters Brian Harrington University of Toronto Scarborough, Ahmad Zubair Alnoor University of Toronto Scarborough, Pedram Haqiqi University of Toronto Scarborough, Zahra Hoseininia University of Toronto Scarborough, Kai Lin University of Toronto Scarborough, Maliha Lodi University of Toronto Scarborough, Asad Mirza University of Toronto Scarborough, Leah Wolfe University of Toronto Scarborough, Kevin Zhang University of Waterloo
10:00 2h Talk		Designing Experiential Learning with Beehive Sensor Data into a Non-Relational Database Course Posters Dean Ganskop Rochester Institute of Technology
10:00 2h Talk		Self-Disclosure and Beyond: Takeaways from an Online and In-Person Computing Ethics Course Posters Helen Weixu Chen University of Waterloo, Maura R. Grossman University of Waterloo, Daniel G. Brown University of Waterloo
10:00 2h Talk		Quack! Configuring Large Language Models to Serve as Rubber Duck Coding Assistants Posters Elias Gonzalez University of Maryland, Joel Chan University of Maryland, David Weintrop University of Maryland
10:00 2h Talk		One-on-One Review Intervention for Students Struggling in Discrete Mathematics Posters Allie Pfleger University of Washington, Robbie Weber University of Washington
10:00 2h Talk		Presenting Computing Accessibility Education Using Experiential Labs Posters Heather Moses Rochester Institute of Technology, Elaina Trapatsos Rochester Institute of Technology, Farzana Rahman Syracuse University, Samuel Malachowsky Rochester Institute of Technology, Daniel Krutz Rochester Institute of Technology
10:00 2h Talk		Persistence in and Change to Computing Majors among Students with Disabilities Posters Jinyoung Hur University of Illinois Urbana-Champaign, Michael Kang University of Illinois Urbana-Champaign, Kathryn Cunningham University of Illinois Urbana-Champaign
10:00 2h Talk		Assessing the Influence of ChatGPT on Student Outcomes in a Models of Computing Course Posters Iñaki Blasco UBC, Karina Mochetti UBC
10:00 2h Talk		Spotting AI Missteps: Students Take on LLM Errors in CS1 Posters Samantha Smith UC Berkeley, Heather Wei University of California, Berkeley, Abby O'Neill University of California, Berkeley, Aneesh Durai UC Berkeley, John DeNero UC Berkeley, J.D. Zamfirescu-Pereira UC Berkeley, Narges Norouzi University of California, Berkeley
10:00 2h Talk		RVR: a versatile RISC-V simulator for education Posters Noah Krim University of California, Davis, Joël Porquet-Lupine University of California, Davis
10:00 2h Talk		Implementation of Technical Interviews as an Alternative Assessment in a Large Introductory CS Course Posters Giulia Alberini McGill University, Elena Bai McGill University
10:00 2h Talk		Nudging the Field Towards Justice-Oriented Goals: Committing to CS Education Posters Alexis Cobo CSforALL, Aman Yadav Michigan State University