Using LLMs to Detect the Presence of Learning Outcomes in Submitted Work Within Computing Ethics Courses (SIGCSE TS 2025 - Posters)

Who

Jedidiah Tsang, Carol Li, Su Min Park, Lisa Yan

Track

SIGCSE TS 2025 Posters

This program is tentative and subject to change.

Time Zone

The program is currently displayed in (GMT-05:00) Eastern Time (US & Canada).

Use conference time zone: (GMT-05:00) Eastern Time (US & Canada)Select other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

By setting a time band, the program will dim events that are outside this time window. This is useful for (virtual) conferences with a continuous program (with repeated sessions).
The time band will also limit the events that are included in the personal iCalendar subscription service.

Display full programSpecify a time band

Save

When

Fri 28 Feb 2025 15:00 - 17:00 at Exhibit Hall C - Posters - Posters #3

Abstract

This study investigates the role of large language models (LLMs) in identifying learning outcomes from student submitted work in a computing ethics course. We use grounded theory to craft a codebook to spot key learning outcomes, such as the usage of critical reasoning and awareness of various social issues. Using Cohen’s kappa to assess interrater reliability, we compare human reviewers’ coding to outputs from models like GPT-4o and GPT-3.5- turbo, finding that GPT-4o performed just as well as the agreement between human reviewers. We use the model outputs to identify specific course readings that students engaged particularly deeply with to better inform our computing ethics instruction.

Jedidiah Tsang

University of California, Berkeley

United States

Carol Li

University of California, Berkeley

United States

Su Min Park

University of California, Berkeley

United States

Lisa Yan

UC Berkeley

United States

This program is tentative and subject to change.

Time Zone

The program is currently displayed in (GMT-05:00) Eastern Time (US & Canada).

Use conference time zone: (GMT-05:00) Eastern Time (US & Canada)Select other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

Display full programSpecify a time band

Save

Session Program

Fri 28 Feb
Displayed time zone: Eastern Time (US & Canada) change

15:00 - 17:00	Posters #3Posters at Exhibit Hall C - Posters

15:00 2h Talk		Student Engagement with Metacognition-based Interventions in Computing and its Relationship with Learning Outcomes Posters Jiaen Yu University of California, San Diego, Anshul Shah University of California, San Diego, John Driscoll University of California, San Diego, Yandong Xiang University of California, San Diego, Xingyin Xu University of California, San Diego, Sophia Krause-Levy University of California, San Diego, Soohyun Liao University of California in San Diego
15:00 2h Talk		From Typing to Insights: An Interactive Code Visualization for Enhanced Student Support Using Keystroke Data Posters Friday James Kansas State University, Russell Feldhausen Kansas State University, Nathan H. Bean Kansas State University, Joshua Weese Kansas State University, David Allen Kansas State University, Michelle Friend University of Nebraska Omaha
15:00 2h Talk		Crafting for Career Agility: An Outcome-Based Redesign of a Machine Learning Curriculum within a Program Bundle Posters En-Shiun Annie Lee University of Toronto, Sean Woodhead York University, Karthik Kuber York University, Hashmat Rohian York University, Stacey Koornneef University of Ontario Institute of Technology
15:00 2h Talk		Facilitators and Barriers for Integrating Technology Education into K-5 Classrooms Posters Miriam Jacobson The University of Texas at Austin, Zhuoying Wang The University of Texas at Austin, Yiwen Yang The University of Texas at Austin, Karanjot Kaur The University of Texas at Austin
15:00 2h Talk		What CS Teachers Want in a CS Professional Development Passport Posters Joseph Tise Institute for Advancing Computing Education, Laycee Thigpen Institute for Advancing Computing Education, Monica McGill Institute for Advancing Computing Education, Robert Schwarzhaupt American Institutes for Research
15:00 2h Talk		ChartCode: A Flowchart-Based Tool for Introductory Programming Courses Posters Guangming Xing Western Kentucky University, Tawfiq Salem Purdue University, Gongbo Liang Texas A & M University - San Antonio
15:00 2h Talk		Detecting Programming Plans in Open-ended Code Submissions Posters Mehmet Arif Demirtas University of Illinois Urbana-Champaign, Claire Zheng University of Illinois Urbana-Champaign, Kathryn Cunningham University of Illinois Urbana-Champaign
15:00 2h Talk		Collaborative CS Professional Development Across Three Cohorts Posters Jina Wilde University of Texas at San Antonio, Michael Zawatski University of Texas at San Antonio, Darean Wilde University of Texas at San Antonio
15:00 2h Talk		Exploring LLMs Impact on Student-Created User Stories and Acceptance Testing in Software Development Posters Allan Brockenbrough Salem State University, Henry Feild Endicott College, Dominic Salinas Salem State University
15:00 2h Talk		Cultivating Sense of Belonging in Computing Education: Identifying Servingness Traits Among Community College Population Posters Christian Servin El Paso Community College, Brett Becker University College Dublin, Emiliano Garcia El Paso Community College
15:00 2h Talk		Experiences teaching a course on Algorithms, Ethics, and Society Posters Nicole Eikmeier Grinnell College, Leah Perlmutter Grinnell College
15:00 2h Talk		The Role of Virtual Reality in Enhancing Computer Science Education Posters Zubair Zabir Rollins College, Sirazum Munira Tisha Rollins College
15:00 2h Talk		Using Peer Tutoring to Bolster Retention Rates and Student Performance in CS1 courses Posters Sharanya Jayaraman Florida State University, Ameya Kolarkar Florida State University
15:00 2h Talk		On a Time Crunch: Examining Learning Outcomes Within a One Unit Computing Ethics Course Posters Jedidiah Tsang University of California, Berkeley, Carol Li University of California, Berkeley, Su Min Park University of California, Berkeley, Lisa Yan UC Berkeley
15:00 2h Talk		Describing the Experience of Instructional Coaches in Supporting Elementary Coding Posters Peter Rich Brigham Young University, Heather Cunningham Boot Up Professional Development, Lien Diaz BootUp PD
15:00 2h Talk		Hacking Student Leadership: Peer Mentorship and Leadership Skill Development Among Hackathon Organizers Posters Kush Patel Temple University, Andrew Tran Temple University, Christine Kapp Temple University, Daniel Bicalho Temple University, Yatri Patel Temple University, Chiku Okechukwu Temple University, Egi Rama Temple University, Stephen MacNeil Temple University
15:00 2h Talk		Empowering CS1 Educators: Enhancing Automated Feedback Instruction with Cognitive Load Theory Posters Justin Gonzaga UNSW, Yuchao Jiang UNSW, Alexandra Vassar UNSW
15:00 2h Talk		Iterative Development of an Evaluative Checklist for Integrated Computer Science Lessons in K-8 Posters Justin Cannady University of Minnesota - Twin Cities
15:00 2h Talk		Computer Science Faculty Perspectives on Undergraduate Student Mental Health Posters Christian Murphy Swarthmore College, Van Nguyen Bryn Mawr College, Asli Yuksel Swarthmore College, Sahasra Nandela Green Hope High School
15:00 2h Talk		Day of AI Australia: Teacher Insights from a Nation-Wide AI Literacy Program for K-12 Students Posters Natasha Banks Day of AI Australia, Jake Renzella University of New South Wales, Sydney
15:00 2h Talk		Raising the Bar: Automating Consistent and Equitable Student Support with LLMs Posters Meenakshi Mittal UC Berkeley, Azalea Bailey UC Berkeley, Victoria Phelps University of California, Berkeley, Mihran Miroyan UC Berkeley, Chancharik Mitra University of California, Berkeley, Rishi Jain UC Berkeley, Rose Niousha University of California, Berkeley, Gireeja Vishnu Ranade UC Berkeley, Narges Norouzi University of California, Berkeley
15:00 2h Talk		Ethics in Motion Interactive Software for Ethical Decision-Making in Autonomous Vehicles Posters Tony Krystofiak College of Saint Benedict and Saint John’s University, Anna Byron College of Saint Benedict and Saint John’s University, Peter Ohmann College of St. Benedict / St. John's University, Heather Amthauer College of Saint Benedict and Saint John’s University
15:00 2h Talk		An Undergraduate Course on Computer Architecture Posters Przemyslaw Nowak Lodz University of Technology, Dariusz Puchala Lodz University of Technology, Kamil Stokfiszewski Lodz University of Technology
15:00 2h Talk		Using LLMs to Detect the Presence of Learning Outcomes in Submitted Work Within Computing Ethics Courses Posters Jedidiah Tsang University of California, Berkeley, Carol Li University of California, Berkeley, Su Min Park University of California, Berkeley, Lisa Yan UC Berkeley
15:00 2h Talk		Scaling Academic Decision-Making with NLP: Automating Transfer Credit Evaluations Posters Nimisha Roy Georgia Institute of Technology, Fisayo Omojokun Georgia Institute of Technology, Huaijin Tu Georgia Institute of Technology
15:00 2h Talk		The Evolving Usage of GenAI by Computing Students Posters Irene Hou Temple University, Hannah Nguyen Temple University, Owen Man Temple University, Stephen MacNeil Temple University
15:00 2h Talk		Rightsizing: Understanding Novice, Casual Learners of Programming Posters Gabriela Ciolacu Karlsruhe Institute of Technology, Christian Haas University of Nebraska at Omaha, Margeret Hall University of Nebraska at Omaha
15:00 2h Talk		Generating AI Literacy MCQs: A Multi-Agent LLM Approach Posters Jiayi Wang Northwestern University, Ruiwei Xiao Carnegie Mellon University, Ying Jui Tseng Carnegie Mellon University
15:00 2h Talk		EclipseMonitor: A Real-Time Student Programming Environment Data Collection Tool Posters Saminur Islam North Carolina State University, Zhikai Gao North Carolina State University, John Bacher North Carolina State University, Gabriel Silva de Oliveira North Carolina State University, Varad Patwardhan North Carolina State University, Sarah Heckman North Carolina State University, Collin Lynch North Carolina State University
15:00 2h Talk		Pencil Puzzles as a Context in Upper-level Core Computing Courses at Multiple Institutions Posters Angelina Brilliantova Rochester Institute of Technology (RIT), Asya Vitko Rochester Institute of Technology, Ivona Bezakova Rochester Institute of Technology, Zack Butler Rochester Institute of Technology
15:00 2h Talk		Evaluating GenAI’s Effectiveness for Students with Varied Programming Backgrounds in a Software Development Course Posters Houda Bouamor Carnegie Mellon University, Gabriela Gongora-Svartzman Carnegie Mellon University, Larry Heimann Carnegie Mellon University, Shihong Huang Carnegie Mellon University
15:00 2h Talk		Scaffolding Mock Conference Projects in Theory of Computing Courses Posters Ryan Dougherty United States Military Academy
15:00 2h Talk		Enabling Professional Growth and Identity of High School CS Teachers through One-on-One Coaching Posters Jennifer Rosato University of Minnesota, Fatima Brunson Institute for Advancing Computing Education, Joseph Tise Institute for Advancing Computing Education, Laycee Thigpen Institute for Advancing Computing Education, Monica McGill Institute for Advancing Computing Education
15:00 2h Talk		Designing an Multi-Modal Community for Black Undergraduate Women in Computing at a Predominantly-White University Posters Luce-Melissa Kouaho University of North Carolina at Charlotte, Noah Q. Cowit University of Colorado, Boulder, Dr. Marlon Mejias University of North Carolina at Charlotte, Rest Johnson University of North Carolina at Charlotte
15:00 2h Talk		A Scaffolding-based Approach for Addressing Challenges of Service Learning Adoption Posters Stan Kurkovsky Central Connecticut State University, Michael Goldweber Xavier University, Chad Williams Central Connecticut State University, Nathan Sommer Xavier University
15:00 2h Talk		Exploring K-12 In-Service Teachers’ Process of Plan Monitor Evaluation in Java programming. Posters Latoya Chandler University of Florida, Rui Huang University of Florida, Maya Israel University of Florida, Priyadharshini Ganapathy Prasad University of Florida
15:00 2h Talk		Introducing Theoretical Computer Science in High School Physics Courses from Impossibility and Undecidability Problems Posters Rafael del Vado Vírseda Universidad Complutense de Madrid
15:00 2h Talk		Glitch Defense: A Computer Game to Teach Cybersecurity Concepts and Broaden Participation in Computing Posters Julio Bahamon UNC Charlotte, Hallie Johnson UNC Charlotte
15:00 2h Talk		Snapgram: matrix multiplications made fun Posters Joël Porquet-Lupine University of California, Davis
15:00 2h Talk		Collaborative design of a week-long physiological computing summer camp with elementary teachers Posters Feiya Luo , Amy Hutchison University of Alabama, Chris Crawford University of Alabama, Fatema Nasrin University of Alabama, Idowu David Awoyemi University of Alabama
15:00 2h Talk		Examining Student Interest and Motivations in Graduate Computer Science Research Posters Bobbie Eicher Georgia Institute of Technology, Alex Duncan Georgia Institute of Technology, Dante Ciolfi Georgia Institute of Technology, Maria Konte Georgia Institute of Technology, Nicholas Lytle Georgia Institute of Technology