Approaches to TBL assessment in Medicine

Assessment overview

based-learning (TBL). TBL assessments consist of iRAT (individual Readiness Assurance Test), tRAT (team Readiness Assurance Test), and team Application (tAPP)  exercises respectively. TBL assessments follow on from in class case-based learning sessions.  The assessments are invigilated and delivered in-person electronically. Students in groups of five or six are given a number of  single best answer questions relating to content delivered during the case based learning sessions and answer questions individually, then review the same questions as a team. They are then given a more challenging task to complete as a team. These assessments are known as iRAT (individual Readiness Assurance Test), tRAT (team Readiness Assurance Test), and team Application (tAPP) respectively. 

There are a total of four cases each term. Teaching is delivered for each individual case while the iRAT and tRAT assessments are delivered in pairs of cases. So, the first iRAT/tRAT of the term is based on the first and second taught cases, and the second iRAT/tRAT is based on the third and fourth taught cases. The tAPP relates to content taught across the four cases of the term but can incorporate new material that students need to process during the assessment.

Design Decisions

Rationale for TBL Assessments

The choice of delivering the module as TBL came from the need to assess team-working skills more effectively. Many high achieving students, while well-versed in medical knowledge, would fall short of working effectively in a team, which is an essential skill for doctors in a clinical environment. TBL had also been historically successfully used to encourage teamwork in BSc Biomedical Science and BSc Pharmacology students.  

Rationale for electronic delivery and software used

Electronic delivery of assessments using Learning Activity Management System (LAMS) software provides real-time data as students go through the assessment. Electronic assessments also allow for automated marking of the iRAT and tRAT segment, which is important considering the huge undergraduate cohorts in Medicine of about 400 students per academic year. 

LAMS provides the ability to receive live data from student submissions, individually (iRAT) or as a group (tRAT, tAPP). The teacher or examiner has full control over the session and how students progress through the sequence of tasks. One further use of this data is to help mend and shape future sessions; if a question is only answered correctly by 20% of students, for example, then the data may suggest the question is misleading or too difficult. Examiners can then investigate this particular question, taking actions as necessary, as well as write future questions avoiding the same shortcomings. 

Question design

The iRAT and tRAT assessments consist of Single Best Answer (SBA) questions. In the iRAT, questions are designed to link more closely to the actual clinical and scientific teaching that students receive in the module. The tRAT, whilst also clinically relevant, is designed to encourage group communication as well. The tAPP focuses largely on developing team-working skills and important skills outside of clinical and scientific knowledge. The question design is therefore much broader and can be condensed into three main categories: data interpretation, infographics, and more complex clinical cases (see an example of a case study here). 

Alignment with other assessments and the programme/module 

The module is constructed as a collaborative process between other modules. It is intended to connect content-wise with other parts of the curriculum so that students have a consistent knowledge base and understand what and why they are being taught the content in a team-working scenario.  

Some of the program level Intended Learning Outcomes have a heavy team-working aspect. The ability to work in a team is heavily weaved into this module’s specification and thus lends itself to these ILOs well.  

The module and module assessment continues over the three years of Phase 1. This means that students will develop a sense of consistency and familiarity with the assessment type, allowing for long-term progression of integrated knowledge from case-based learning in a team-based environment. 

Practicalities

Preparing students for assessment

To prepare students, there are formative TBL assessments in other modules that serve as preparatory work for CSI. Within CSI, a number of formative cases are constructed and then delivered via a Teams call to allow students to practice in their groups and to facilitate the process of the real assessment in-person. These formative cases are predominantly tAPPS, most likely due to their complexity for both students to complete and for staff to construct and mark. 

Marking arrangements

The iRAT and tRAT are auto-marked. The mark allocation is set up in a way that is as safe as possible for students; even if an individual student performed very poorly in the iRAT, the team-based component would protect them, making it virtually impossible for the student to fail the module.  That being said, for the tRAT, the marking system is quite punitive, with students being allowed four attempts to select the correct answer. 

 

Correct Answer on ' ' AttemptMarks
First +4
Second +1
Third -2
Fourth -5

The tAPP segment is marked by staff and is just as complex to mark as the questions themselves. There are usually more than eight individuals marking tAPPS, and they are all double marked.  

Feedback arrangements

For the iRAT and tRAT, students have a post-assessment 24-hour window where they are given the opportunity to challenge questions for any reason. Once all challenges are accrued, the module staff will sit down, work through the challenges and collectively decide on their outcome. 

The tAPP feedback, being marked manually, generally takes longer. It is typically a 2 week turnaround.  

Management of the process

The module is managed by leads and fellows with one Science Lead (interviewee), one Clinical Lead, one Science Fellow and one Clinical Fellow. In a typical in-person session, there are five to six people who assist in delivering the content to students. These will be clinicians (i.e. medical practitioners), scientists (i.e., staff members involved in scientific research) etc. to encourage a range of perspectives for better delivery of content to students. Having clinicians and scientists in the room to assist in delivering sessions means that most questions asked by students can be answered.

Online adaptations

During Covid the module and assessments were delivered online. Student groups were placed in breakout rooms with tutors invigilating by moving in and out of breakout rooms. Overall, student performance did not change, but student behaviour did change. Teams allowed students to turn off their cameras and microphones, which many of them did, allowing for student disengagement in groups.  

Student Perspective

The student view of TBL is generally positive. There is a recognition of this way of learning being linked to real life practice in MDT (multidisciplinary) meetings where you are expected to express your opinion which is then discussed further with others. The students recognised that the varied tAPPs were a lot more difficult to prepare for. 

The actual style of the CSI questions were thought to be quite difficult.  There was a feeling of questions lacking clarity and that they were not really testing the knowledge, but testing students’ ability to pick up on those tricks.  

There also seemed to be a lot of disparity between the groups with some groups having 5 members and some seven. When it came to doing the tAPP tasks, smaller groups would often struggle because of less people working in a time pressure environment. 

Hear what the Imperial experts have to say...

What is TBL?

What is TBL?

What is TBL?

Kate Ippolito, Centre for Higher Education Research and Scholarship

Pros and cons of group work

Pros and cons of group work

Dr Iro Ntonia, Centre for Higher Education Research and Scholarship

What to consider when introducing TBL

What to consider when introducing TBL

Kate Ippolito, Centre for Higher Education Research and Scholarship

What is summative and formative assessment

What is summative and formative assessment

Kate Ippolito, Centre for Higher Education Research and Scholarship

Disadvantages of formative assessment

Disadvantages of formative assessment

Kate Ippolito, Centre for Higher Education Research and Scholarship

The value of attaching credit to formative assessments

The value of attaching credit to formative assessments

Kate Ippolito, Centre for Higher Education Research and Scholarship

Preparing students for peer assessment

Preparing students for peer assessment

Kate Ippolito, Centre for Higher Education Research and Scholarship