Publications

BibTex format

@article{Davies:2022:10.1007/s10459-021-10076-5,
author = {Davies, DJ and McLean, PF and Kemp, PR and Liddle, AD and Morrell, MJ and Halse, O and Martin, NM and Sam, AH},
doi = {10.1007/s10459-021-10076-5},
journal = {Advances in Health Sciences Education},
pages = {147--165},
title = {Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination},
url = {http://dx.doi.org/10.1007/s10459-021-10076-5},
volume = {27},
year = {2022}
}

Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom’s taxonomy (‘Remember’, versus ‘Understand/Apply’). We predicted higher performance on the OBE, driven by higher performance on ‘Remember’ questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms ‘Open-Book, ‘Bloom Category’ and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14–2.39), and higher on ‘Remember’ than ‘Understand/Apply’ questions (OR 1.13, 95% CI: 1.09–1.19). The difference in performance between ‘Remember’ and ‘Understand/Apply’ questions was greater in the OBE than the CBE (‘Open-Book’  ‘Bloom Category’ interaction: OR 1.2, 95% CI: 1.19–1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge.
AU  - Davies,DJ
AU  - McLean,PF
AU  - Kemp,PR
AU  - Liddle,AD
AU  - Morrell,MJ
AU  - Halse,O
AU  - Martin,NM
AU  - Sam,AH
DO  - 10.1007/s10459-021-10076-5
EP  - 165
PY  - 2022///
SN  - 1382-4996
SP  - 147
TI  - Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination
T2  - Advances in Health Sciences Education
UR  - http://dx.doi.org/10.1007/s10459-021-10076-5
UR  - http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000710599800001&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
UR  - http://hdl.handle.net/10044/1/92472
VL  - 27
ER  -

Download

DrPaulKemp

Contact

Location

Summary

Citation

BibTex format

RIS format (EndNote, RefMan)