Using AI to evaluate feedback and improve practice in a multi-marker module

This project evaluated marker feedback on the formative individual coursework assessment in Lab Pod 2, a Year 2 core module in the BSc Medical Biosciences programme. Students produce a scientific report structured as a primary research article, including an abstract, introduction, methods, results and discussion. This formative task directly prepares students for a linked summative scientific report based on their own research project. It was therefore an important opportunity to examine whether feedback was helping students understand their strengths, identify areas for development and take effective action before the summative assessment.

The evaluation compared feedback from the 2025 and 2026 academic years, analysing both in-text and overall comments. It aimed to identify effective practice and areas requiring improvement, using the findings to refine marker guidance and support a more consistent student experience.

A key motivation was the challenge of maintaining high-quality feedback across a marking team of approximately nine to ten staff. Although moderation can support consistency in grades, feedback may still vary in quantity, tone, clarity and developmental value. Students with similar marks may consequently receive different messages about expectations and how to improve, potentially creating perceptions of inequity. High marking volumes and strict turnaround deadlines can also encourage brief, generic or grade-justifying comments rather than constructive, actionable feedforward.

The project therefore examined consistency between markers; the presence of strengths, improvement points and actionable guidance; tone and constructiveness; and the balance of evaluative, descriptive, corrective and feedforward comments. AI-assisted thematic categorisation was used because it enabled systematic analysis of a large volume of qualitative feedback more efficiently and consistently than manual review alone. The approach was intended to strengthen quality assurance, inform marker calibration and create a scalable model for evaluating feedback across the wider programme.

AI was not used to mark student work or generate feedback. Instead, it supported quality enhancement by analysing feedback already written by academic staff. OpenAI GPT models helped develop Python scripts that extracted anonymised comments from Turnitin PDF reports, converted them into structured spreadsheets and submitted them in batches through the OpenAI API. The API then classified comments against predefined educational criteria, enabling analysis of feedback quality, consistency and patterns across the marking team. Results identified effective practice and areas for improvement, informing marker training and revised guidance. Using the API, rather than a chatbot, automated large-scale processing efficiently, consistently and reproducibly

AI classification was based on predefined educational criteria and existing marker guidance, ensuring alignment with evidence-based feedback practice. Structured prompts defined categories, decision rules and output formats. The process was refined through testing small batches of anonymised comments, manually reviewing results, and revising prompts before analysing the full dataset. Python scripts handled local extraction and preprocessing, while only anonymised comments were sent to the OpenAI API. Academic judgement remained central to validating results and interpreting findings. Risks of misclassifying nuanced comments were mitigated through iterative testing and human review. Initial development required time, but the resulting workflow is efficient, reproducible and scalable.

The 2025 evaluation found generally strong feedback practice: 98% of overarching comments identified a strength, 99% identified an area for improvement, and 93% included actionable advice to support future performance. However, two areas for development emerged. First, 30% of comments used hedging language, such as “perhaps” or “you could,” which can reduce clarity, particularly for international and multilingual students. Revised guidance encouraged direct but constructive communication, reducing hedged comments to 24% in 2026. Second, 7% of comments focused on the student rather than their submitted work, for example by referring to effort or ability. Guidance was amended to emphasise work-focused feedback, and by 2026 all comments were work-focused. Overall, AI-supported evaluation enabled systematic reflection on feedback quality, strengthened consistency across the marking team, and informed targeted improvements. Continued annual monitoring will establish whether these changes are sustained and further enhance students’ engagement with feedback.

• Feedback practices were broadly aligned across the module, but variations existed despite shared instructions, rubrics and criteria, highlighting the need for ongoing calibration, shared discussion and regular review to ensure consistency and equity.
• AI-assisted analysis strengthens quality assurance by surfacing patterns in consistency, tone and feedback type that manual review may miss but it should support, not replace, academic judgement.
• When repeating the same approach define clear evaluation criteria based on recognised feedback principles beforehand; anonymise data where appropriate; be transparent about AI use.
• Approach findings with an open mind, communicate rationale clearly, and ensure marker engagement for effective adoption.
• Pedagogical expertise mattered more than technical skill; human oversight remained essential to refine criteria and catch AI misclassifications of nuanced feedback.

• Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315-1325. https://doi.org/10.1080/02602938.2018.1463354
• Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487
• Mandouit,L., & Hattie,J. (2023). Revisiting “The Power of Feedback” from the perspective of the learner. Learning and Instruction,84. https://doi.org/10.1016/j.learninstruc.2022.101718.
• Ryoo, S. M. (2023). Efficacy of hedges in formative feedback on L2 writing. International Journal on Studies in Education, 5(4), 550-567. https://doi.org/10.46328/ijonse.171