Revisiting the Performance of Generative Artificial Intelligence on Introductory Object-Oriented Programming Assessments: Insights from 2026

This study investigates how contemporary Generative Artificial Intelligence (GenAI) systems perform on authentic university-level object-oriented programming (OOP) assessments. Specifically, the authors address two research questions: how GenAI systems perform on authentic introductory OOP assessments (RQ1), and what recurring error patterns and limitations these systems encounter (RQ2). The research context is a 16-week, Java-based introductory OOP course at the University of Tartu, enrolling 270 to 330 students annually, which utilizes a flipped-classroom model, programming tests, group projects, and a computer-based final exam.

Based on a student survey indicating that 87.8% of respondents used AI assistants, the authors selected five widely used GenAI systems for evaluation: ChatGPT-5.2, DeepSeek-V3, Gemini 2.5 Flash, Claude Sonnet 4.5, and M365 Copilot. The models were provided with the exact, unedited Estonian text of two programming tests and a final exam from the course, without any prompt engineering, translation, or iterative refinement. Only the first response generated by each system was evaluated. The grading followed the strict course criteria applied to human students, including the rule that non-compiling code receives zero points.

All evaluated GenAI systems outperformed the average student cohort, frequently earning full marks on longer programming test tasks. Compared to evaluations from the previous year, the systems exhibited noticeable overall improvements. However, performance was not uniform across all topics. The models continued to struggle with advanced object-oriented paradigms, particularly interfaces, abstract classes, and complex inheritance hierarchies. Furthermore, tasks requiring image interpretation and graphics-related questions presented notable limitations, echoing broader challenges in handling non-textual data.