Imperial College London

DrOlgaKostopoulou

Faculty of MedicineDepartment of Surgery & Cancer

Reader in Medical Decision Making
 
 
 
//

Contact

 

o.kostopoulou Website

 
 
//

Location

 

5.07Medical SchoolSt Mary's Campus

//

Summary

 

Publications

Publication Type
Year
to

60 results found

Kostopoulou O, Schwartz A, 2021, To unpack or not? Testing public health messaging about COVID-19, Journal of Experimental Psychology: Applied, ISSN: 1076-898X

Support theory suggests that the judged probability of events depends on the explicitness of their description. We tested whether risk communication messages that specify risks involved are associated with increased intentions to comply with public health advice during a pandemic. We conducted an anonymous online survey of the U.K. and U.S. public between April 24 and May 12, 2020. Participants (N = 2087) rated 14 COVID-related symptoms in terms of perceived severity and induced worry. They were then asked about their intention to practise social distancing in response to three public health messages: the standard U.K. government message: “Most people will experience only mild symptoms”; the standard message “unpacked” by listing six of those symptoms as examples; and “Most people will not require hospitalisation.” The unpacked message resulted in the highest intention to comply with social distancing (b = .22 [.04, .40], p = .02) and there was no interaction with country. Worry about symptoms was an independent predictor of intention to comply (b = .02 [.01, .03], p < .001). In the days before lockdown amidst a raging pandemic, the U.K. and U.S. governments sought to reassure the public. Had their messaging been more detailed, it might have been less reassuring but more effective in promoting social distancing.

Journal article

Nurek M, Delaney B, Kostopoulou O, 2021, GENERAL PRACTITIONERS' RISK ASSESSMENTS AND ANTIBIOTIC PRESCRIBING DECISIONS IN CHILDREN WITH COUGH: A VIGNETTE STUDY, Publisher: SAGE PUBLICATIONS INC, Pages: E51-E52, ISSN: 0272-989X

Conference paper

Kourtidis P, Nurek M, Delaney B, Kostopoulou Oet al., 2021, INFLUENCES OF DIAGNOSTIC SUGGESTIONS ON CLINICAL REASONING, Publisher: SAGE PUBLICATIONS INC, Pages: E262-E264, ISSN: 0272-989X

Conference paper

Kostopoulou O, Tracey C, Delaney B, 2021, Can decision support combat incompleteness and bias in routine primary care data?, Journal of the American Medical Informatics Association, ISSN: 1067-5027

Objective: Routine primary care data may be used for the derivation of clinical prediction rules and risk scores. We sought to measure the impact of a decision support system (DSS) on data completeness and freedom from bias.Materials and Methods: We used the clinical documentation of 34 UK general practitioners who took part in a previous study evaluating the DSS. They consulted with 12 standardized patients. In addition to suggesting di- agnoses, the DSS facilitates data coding. We compared the documentation from consultations with the elec- tronic health record (EHR) (baseline consultations) vs consultations with the EHR-integrated DSS (supported consultations). We measured the proportion of EHR data items related to the physician’s final diagnosis. We expected that in baseline consultations, physicians would document only or predominantly observations re- lated to their diagnosis, while in supported consultations, they would also document other observations as a re- sult of exploring more diagnoses and/or ease of coding.Results: Supported documentation contained significantly more codes (incidence rate ratio [IRR] 1⁄4 5.76 [4.31, 7.70] P < .001) and less free text (IRR 1⁄4 0.32 [0.27, 0.40] P < .001) than baseline documentation. As expected, the proportion of diagnosis-related data was significantly lower (b 1⁄4 􏰀0.08 [􏰀0.11, 􏰀0.05] P < .001) in the supported consultations, and this was the case for both codes and free text.Conclusions: We provide evidence that data entry in the EHR is incomplete and reflects physicians’ cognitive biases. This has serious implications for epidemiological research that uses routine data. A DSS that facilitates and motivates data entry during the consultation can improve routine documentation.

Journal article

Kostopoulou O, Nurek M, Delaney B, 2020, Disentangling the relationship between physician and organizational performance: a signal detection approach, Medical Decision Making, Vol: 40, Pages: 746-755, ISSN: 0272-989X

Background. In previous research, we employed a signal detection approach to measure the performance of general practitioners (GPs) when deciding about urgent referral for suspected lung cancer. We also explored associations between provider and organizational performance. We found that GPs from practices with higher referral positive predictive value (PPV; chance of referrals identifying cancer) were more reluctant to refer than those from practices with lower PPV. Here, we test the generalizability of our findings to a different cancer. Methods. A total of 252 GPs responded to 48 vignettes describing patients with possible colorectal cancer. For each vignette, respondents decided whether urgent referral to a specialist was needed. They then completed the 8-item Stress from Uncertainty scale. We measured GPs’ discrimination (d′) and response bias (criterion; c) and their associations with organizational performance and GP demographics. We also measured correlations of d′ and c between the 2 studies for the 165 GPs who participated in both. Results. As in the lung study, organizational PPV was associated with response bias: in practices with higher PPV, GPs had higher criterion (b = 0.05 [0.03 to 0.07]; P < 0.001), that is, they were less inclined to refer. As in the lung study, female GPs were more inclined to refer than males (b = −0.17 [−0.30 to −0.105]; P = 0.005). In a mediation model, stress from uncertainty did not explain the gender difference. Only response bias correlated between the 2 studies (r = 0.39, P < 0.001). Conclusions. This study confirms our previous findings regarding the relationship between provider and organizational performance and strengthens the finding of gender differences in referral decision making. It also provides evidence that response bias is a relatively stable feature of GP referral decision making.

Journal article

Nurek M, Delaney BC, Kostopoulou O, 2020, Risk assessment and antibiotic prescribing decisions in children presenting to UK primary care with cough: a vignette study, BMJ Open, Vol: 10, ISSN: 2044-6055

Objectives: The validated “STARWAVe” clinical prediction rule (CPR) uses seven variables to guide risk assessment and antimicrobial stewardship in children presenting with cough(Short illness duration, Temperature, Age, Recession, Wheeze, Asthma,Vomiting). We aimed to compare General Practitioners’ (GPs) risk assessments and prescribing decisions to those of STARWAVe, and assess the influence of the CPR’s clinical variables. Setting: Primary care. Participants: 252 GPs, currently practising in the UK. Design: GPs were randomly assigned to view four (of a possible eight) clinical vignettes online. Each vignette depicted a child presenting with cough, who was described in terms of the seven STARWAVe variables. Systematically, we manipulated patient age (20 months vs. 5 years), illness duration (3 vs. 6 days),vomiting (present vs. absent) and wheeze (present vs. absent), holding the remaining STARWAVe variables constant. Outcome measures:Per vignette, GPs assessed risk of hospitalisation and indicated whether they would prescribe antibiotics or not. Results: GPs overestimated risk of hospitalisationin 9% of vignette presentations (88/1008) and underestimated it in 46% (459/1008). Despite underestimating risk, they overprescribed: 78% of prescriptions were unnecessary relative to GPs’ own risk assessments (121/156), while 83% were unnecessary relativeto STARWAVe risk assessments (130/156). All four of the manipulated variables influenced risk assessments, but only three influenced prescribing decisions: a shorter illness duration reduced prescribing odds (OR 0.14, 95% CI 0.08-0.27, p<0.001), while vomiting and wheeze increased them (ORvomit2.17, 95% CI 1.32-3.57, p=0.002; ORwheeze8.98, 95% CI 4.99-16.15, p<0.001). Conclusions: Relative to STARWAVe, GPs underestimated riskof hospitalisation, overprescribed, and appeared to

Journal article

Ramtale S, Delaney B, Kostopoulou O, 2020, USING A DIAGNOSTIC AID CHANGES PHYSICIAN BEHAVIOR IN THE CONSULTATION, Publisher: SAGE PUBLICATIONS INC, Pages: E270-E271, ISSN: 0272-989X

Conference paper

Tracey C, Delaney B, Kostopoulou O, 2020, THE USE OF DIAGNOSTIC DECISION SUPPORT CAN REDUCE BIAS IN CLINICAL DOCUMENTATION, Publisher: SAGE PUBLICATIONS INC, Pages: E270-E270, ISSN: 0272-989X

Conference paper

Kostopoulou O, Nurek M, Cantarella S, Okoli G, Fiorentino F, Delaney Bet al., 2019, Referral decision making of General Practitioners: a signal detection study, Medical Decision Making, Vol: 39, Pages: 21-31, ISSN: 0272-989X

Background. Signal detection theory (SDT) describes how respondents categorize ambiguous stimuli over repeated trials. It measures separately “discrimination” (ability to recognize a signal amid noise) and “criterion” (inclination to respond “signal” v. “noise”). This is important because respondents may produce the same accuracy rate for different reasons. We employed SDT to measure the referral decision making of general practitioners (GPs) in cases of possible lung cancer. Methods. We constructed 44 vignettes of patients for whom lung cancer could be considered and estimated their 1-year risk. Under UK risk-based guidelines, half of the vignettes required urgent referral. We recruited 216 GPs from practices across England. Practices differed in the positive predictive value (PPV) of their urgent referrals (chance of referrals identifying cancer) and the sensitivity (chance of cancer patients being picked up via urgent referral from their practice). Participants saw the vignettes online and indicated whether they would refer each patient urgently or not. We calculated each GP’s discrimination (d ′) and criterion (c) and regressed these on practice PPV and sensitivity, as well as on GP experience and gender. Results. Criterion was associated with practice PPV: as PPV increased, GPs’c also increased, indicating lower inclination to refer (b = 0.06 [0.02–0.09]; P = 0.001). Female GPs were more inclined to refer than male GPs (b = −0.20 [−0.40 to −0.001]; P = 0.049). Average discrimination was modest (d′ = 0.77), highly variable (range, −0.28 to 1.91), and not associated with practice referral performance. Conclusions. High referral PPV at the organizational level indicates GPs’ inclination to avoid false positives, not better discrimination. Rather than bluntly mandating increases in practice PPV via more referrals, it is necessary to increase discrimina

Journal article

Okoli GN, Kostopoulou O, Delaney BC, 2018, Is symptom-based diagnosis of lung cancer possible? A systematic review and meta-analysis of symptomatic lung cancer prior to diagnosis for comparison with real-time data from routine general practice, PLoS ONE, Vol: 13, ISSN: 1932-6203

BackgroundLung cancer is a good example of the potential benefit of symptom-based diagnosis, as it is the commonest cancer worldwide, with the highest mortality from late diagnosis and poor symptom recognition. The diagnosis and risk assessment tools currently available have been shown to require further validation. In this study, we determine the symptoms associated with lung cancer prior to diagnosis and demonstrate that by separating prior risk based on factors such as smoking history and age, from presenting symptoms and combining them at the individual patient level, we can make greater use of this knowledge to create a practical framework for the symptomatic diagnosis of individual patients presenting in primary care.AimTo provide an evidence-based analysis of symptoms observed in lung cancer patients prior to diagnosis.Design and settingSystematic review and meta-analysis of primary and secondary care data.MethodSeven databases were searched (MEDLINE, Embase, Cumulative Index to Nursing and Allied Health Literature, Health Management Information Consortium, Web of Science, British Nursing Index and Cochrane Library). Thirteen studies were selected based on predetermined eligibility and quality criteria for diagnostic assessment to establish the value of symptom-based diagnosis using diagnosistic odds ratio (DOR) and summary receiver operating characteristic (SROC) curve. In addition, routinely collated real-time data from primary care electronic health records (EHR), TransHis, was analysed to compare with our findings.ResultsHaemoptysis was found to have the greatest diagnostic value for lung cancer, diagnostic odds ratio (DOR) 6.39 (3.32–12.28), followed by dyspnoea 2.73 (1.54–4.85) then cough 2.64 (1.24–5.64) and lastly chest pain 2.02 (0.88–4.60). The use of symptom-based diagnosis to accurately diagnose lung cancer cases from non-cases was determined using the summary receiver operating characteristic (SROC) curve, the area under t

Journal article

Petrova D, Kostopoulou O, Delaney BD, Cokely ET, Garcia-Retamero Ret al., 2018, Strengths and gaps in physicians’ risk communication: a scenario study of the influence of numeracy on cancer screening communication, Medical Decision Making, Vol: 38, Pages: 355-365, ISSN: 0272-989X

Objective. Many patients have low numeracy, which impedes their understanding of important information about health (e.g., benefits and harms of screening). We investigated whether physicians adapt their risk communication to accommodate the needs of patients with low numeracy, and how physicians’ own numeracy influences their understanding and communication of screening statistics. Methods. UK family physicians (N = 151) read a description of a patient seeking advice on cancer screening. We manipulated the level of numeracy of the patient (low v. high v. unspecified) and measured physicians’ risk communication, recommendation to the patient, understanding of screening statistics, and numeracy. Results. Consistent with best practices, family physicians generally preferred to use visual aids rather than numbers when communicating information to a patient with low (v. high) numeracy. A substantial proportion of physicians (44%) offered high quality (i.e., complete and meaningful) risk communication to the patient. This was more often the case for physicians with higher (v. lower) numeracy who were more likely to mention mortality rates, OR=1.43 [1.10, 1.86], and harms from overdiagnosis, OR=1.44 [1.05, 1.98]. Physicians with higher numeracy were also more likely to understand that increased detection or survival rates do not demonstrate screening effectiveness, OR=1.61 [1.26, 2.06]. Conclusions. Most physicians know how to appropriately tailor risk communication for patients with low numeracy (i.e., with visual aids). However, physicians who themselves have low numeracy are likely to misunderstand the risks and unintentionally mislead patients by communicating incomplete information. High-quality risk communication and shared decision making can depend critically on factors that improve the risk literacy of physicians.

Journal article

Delaney BC, Kostopoulou O, 2017, Decision support for diagnosis should become routine in 21st century primary care, British Journal of General Practice, ISSN: 0960-1643

Journal article

Porat T, Delaney BC, Kostopoulou, 2017, The impact of a diagnostic decision support system on the consultation: perceptions of GPs and patients, BMC Medical Informatics and Decision Making, Vol: 17, ISSN: 1472-6947

BackgroundClinical decision support systems (DSS) aimed at supporting diagnosis are not widely used. This is mainly due to usability issues and lack of integration into clinical work and the electronic health record (EHR). In this study we examined the usability and acceptability of a diagnostic DSS prototype integrated with the EHR and in comparison with the EHR alone.MethodsThirty-four General Practitioners (GPs) consulted with 6 standardised patients (SPs) using only their EHR system (baseline session); on another day, they consulted with 6 different but matched for difficulty SPs, using the EHR with the integrated DSS prototype (DSS session). GPs were interviewed twice (at the end of each session), and completed the Post-Study System Usability Questionnaire at the end of the DSS session. The SPs completed the Consultation Satisfaction Questionnaire after each consultation.ResultsThe majority of GPs (74%) found the DSS useful: it helped them consider more diagnoses and ask more targeted questions. They considered three user interface features to be the most useful: (1) integration with the EHR; (2) suggested diagnoses to consider at the start of the consultation and; (3) the checklist of symptoms and signs in relation to each suggested diagnosis. There were also criticisms: half of the GPs felt that the DSS changed their consultation style, by requiring them to code symptoms and signs while interacting with the patient. SPs sometimes commented that GPs were looking at their computer more than at them; this comment was made more often in the DSS session (15%) than in the baseline session (3%). Nevertheless, SP ratings on the satisfaction questionnaire did not differ between the two sessions.ConclusionsTo use the DSS effectively, GPs would need to adapt their consultation style, so that they code more information during rather than at the end of the consultation. This presents a potential barrier to adoption. Training GPs to use the system in a patient-centred way

Journal article

Corrigan D, Munelley G, Kazienko P, Kajdanowcz T, Soler J-K, Mahmoud S, Porat T, Kostopoulou O, Curcin V, Delaney BCet al., 2017, Requirements and validation of a prototype learning health system for clinical diagnosis, Learning Health Systems, Vol: 1, ISSN: 2379-6146

IntroductionDiagnostic error is a major threat to patient safety in the context of family practice. The patient safety implications are severe for both patient and clinician. Traditional approaches to diagnostic decision support have lacked broad acceptance for a number of well-documented reasons: poor integration with electronic health records and clinician workflow, static evidence that lacks transparency and trust, and use of proprietary technical standards hindering wider interoperability. The learning health system (LHS) provides a suitable infrastructure for development of a new breed of learning decision support tools. These tools exploit the potential for appropriate use of the growing volumes of aggregated sources of electronic health records.MethodsWe describe the experiences of the TRANSFoRm project developing a diagnostic decision support infrastructure consistent with the wider goals of the LHS. We describe an architecture that is model driven, service oriented, constructed using open standards, and supports evidence derived from electronic sources of patient data. We describe the architecture and implementation of 2 critical aspects for a successful LHS: the model representation and translation of clinical evidence into effective practice and the generation of curated clinical evidence that can be used to populate those models, thus closing the LHS loop.Results/ConclusionsSix core design requirements for implementing a diagnostic LHS are identified and successfully implemented as part of this research work. A number of significant technical and policy challenges are identified for the LHS community to consider, and these are discussed in the context of evaluating this work: medico-legal responsibility for generated diagnostic evidence, developing trust in the LHS (particularly important from the perspective of decision support), and constraints imposed by clinical terminologies on evidence generation.

Journal article

Sirota M, Kostopoulou O, Round T, Samaranayaka Set al., 2017, Prevalence and alternative explanations influence cancer diagnosis: an experimental study with physicians, Health Psychology, Vol: 36, Pages: 477-485, ISSN: 1930-7810

Objective: Cancer causes death to millions of people worldwide. Early detection of cancer in primary care may enhance patients’ chances of survival. However, physicians often miss early cancers, which tend to present with undifferentiated symptoms. Within a theoretical framework of the hypothesis generation (HyGene) model, together with psychological literature, we studied how 2 factors—cancer prevalence and an alternative explanation for the patient’s symptoms—impede early cancer detection, as well as prompt patient management. Method: Three hundred family physicians diagnosed and managed 2 patient cases, where cancer was a possible diagnosis (one colorectal cancer, the other lung cancer). We employed a 2 (cancer prevalence: low vs. high) × 2 (alternative explanation: present vs. absent) between-subjects design. Cancer prevalence was manipulated by changing either patient age or sex; the alternative explanation for the symptoms was manipulated by adding or removing a relevant clinical history. Each patient consulted twice. Results: In a series of random-intercept logistic models, both higher prevalence (OR = 1.92, 95% confidence interval [CI 1.27, 2.92]) and absence of an alternative explanation (OR = 1.70, 95% CI [1.11, 2.59]) increased the likelihood of a cancer diagnosis, which, in turn, increased the likelihood of prompt referral (OR = 22.84, 95% CI [16.14, 32.32]). Conclusions: These findings confirm the probabilistic nature of the diagnosis generation process and validate the application of the HyGene model to early cancer detection. Increasing the salience of cancer—such as listing cancer as a diagnostic possibility—during the initial hypothesis generation phase may improve early cancer detection. (PsycINFO Database Record).

Journal article

Sirota M, Round T, Samaranayaka S, Kostopoulou Oet al., 2017, Expectations for antibiotics increase their prescribing: causal evidence about localized impact, Health Psychology, Vol: 36, Pages: 402-409, ISSN: 1930-7810

Objective: Clinically irrelevant but psychologically important factors such as patients’ expectations for antibiotics encourage overprescribing. We aimed to (a) provide missing causal evidence of this effect, (b) identify whether the expectations distort the perceived probability of a bacterial infection either in a pre- or postdecisional distortions pathway, and (c) detect possible moderators of this effect. Method: Family physicians expressed their willingness to prescribe antibiotics (Experiment 1, n₁ = 305) or their decision to prescribe (Experiment 2, n₂ = 131) and assessed the probability of a bacterial infection in hypothetical patients with infections either with low or high expectations for antibiotics. Response order of prescribing/probability was manipulated in Experiment 1. Results: Overall, the expectations for antibiotics increased intention to prescribe (Experiment 1, F(1, 301) = 25.32, p< .001, η p² = .08, regardless of the response order; Experiment 2, odds ratio [OR] = 2.31, and OR = 0.75, Vignettes 1 and 2, respectively). Expectations for antibiotics did not change the perceived probability of a bacterial infection (Experiment 1, F(1, 301) = 1.86, p = .173, ηp² = .01, regardless of the response order; Experiment 2, d = −0.03, and d = +0.25, Vignettes 1 and 2, respectively). Physicians’ experience was positively associated with prescribing, but it did not moderate the expectations effect on prescribing. Conclusions: Patients’ and their parents’ expectations increase antibiotics prescribing, but their effect is localized—it does not leak into the perceived probability of a bacterial infection. Interventions reducing the overprescribing of antibiotics should target also psychological factors. (PsycINFO Database Record (c) 2017 APA, all rights reserved)

Journal article

Kostopoulou O, Porat T, Corrigan D, Mahmoud S, Delaney BCet al., 2017, Diagnostic accuracy of GPs when using an early-intervention decision support system: a high-fidelity simulation, British Journal of General Practice, Vol: 67, Pages: e201-e208, ISSN: 1478-5242

Background Observational and experimental studies of the diagnostic task have demonstrated the importance of the first hypotheses that come to mind for accurate diagnosis. A prototype decision support system (DSS) designed to support GPs’ first impressions has been integrated with a commercial electronic health record (EHR) system.Aim To evaluate the prototype DSS in a high-fidelity simulation.Design and setting Within-participant design: 34 GPs consulted with six standardised patients (actors) using their usual EHR. On a different day, GPs used the EHR with the integrated DSS to consult with six other patients, matched for difficulty and counterbalanced.Method Entering the reason for encounter triggered the DSS, which provided a patient-specific list of potential diagnoses, and supported coding of symptoms during the consultation. At each consultation, GPs recorded their diagnosis and management. At the end, they completed a usability questionnaire. The actors completed a satisfaction questionnaire after each consultation.Results There was an 8–9% absolute improvement in diagnostic accuracy when the DSS was used. This improvement was significant (odds ratio [OR] 1.41, 95% confidence interval [CI] = 1.13 to 1.77, P<0.01). There was no associated increase of investigations ordered or consultation length. GPs coded significantly more data when using the DSS (mean 12.35 with the DSS versus 1.64 without), and were generally satisfied with its usability. Patient satisfaction ratings were the same for consultations with and without the DSS.Conclusion The DSS prototype was successfully employed in simulated consultations of high fidelity, with no measurable influences on patient satisfaction. The substantially increased data coding can operate as motivation for future DSS adoption.

Journal article

Nurek M, Kostopoulou O, 2016, What You Find Depends on How You Measure It: Reactivity of Response Scales Measuring Predecisional Information Distortion in Medical Diagnosis, PLOS One, Vol: 11, ISSN: 1932-6203

“Predecisional information distortion” occurs when decision makers evaluate new information in a way that is biased towards their leading option. The phenomenon is well established, as is the method typically used to measure it, termed “stepwise evolution of preference” (SEP). An inadequacy of this method has recently come to the fore: it measures distortion as the total advantage afforded a leading option over its competitor, and therefore it cannot differentiate between distortion to strengthen a leading option (“proleader” distortion) and distortion to weaken a trailing option (“antitrailer” distortion). To address this, recent research introduced new response scales to SEP. We explore whether and how these new response scales might influence the very proleader and antitrailer processes that they were designed to capture (“reactivity”). We used the SEP method with concurrent verbal reporting: fifty family physicians verbalized their thoughts as they evaluated patient symptoms and signs (“cues”) in relation to two competing diagnostic hypotheses. Twenty-five physicians evaluated each cue using the response scale traditional to SEP (a single response scale, returning a single measure of distortion); the other twenty-five did so using the response scales introduced in recent studies (two separate response scales, returning two separate measures of distortion: proleader and antitrailer). We measured proleader and antitrailer processes in verbalizations, and compared verbalizations in the single-scale and separate-scales groups. Response scales did not appear to affect proleader processes: the two groups of physicians were equally likely to bolster their leading diagnosis verbally. Response scales did, however, appear to affect antitrailer processes: the two groups denigrated their trailing diagnosis verbally to differing degrees. Our findings suggest that the response scales used to measure infor

Journal article

Kostopoulou O, Sirota M, Round T, Samaranayaka S, Delaney BCet al., 2016, The role of physicians’ first impressions in the diagnosis of possible cancers without alarm symptoms, Medical Decision Making, Vol: 37, Pages: 9-16, ISSN: 1552-681X

Background. First impressions are thought to exert a disproportionate influence on subsequent judgments; however, their role in medical diagnosis has not been systematically studied. We aimed to elicit and measure the association between first impressions and subsequent diagnoses in common presentations with subtle indications of cancer. Methods. Ninety UK family physicians conducted interactive simulated consultations online, while on the phone with a researcher. They saw 6 patient cases, 3 of which could be cancers. Each cancer case included 2 consultations, whereby each patient consulted again with nonimproving and some new symptoms. After reading an introduction (patient description and presenting problem), physicians could request more information, which the researcher displayed online. In 2 of the possible cancers, physicians thought aloud. Two raters coded independently the physicians’ first utterances (after reading the introduction but before requesting more information) as either acknowledging the possibility of cancer or not. We measured the association of these first impressions with the final diagnoses and management decisions. Results. The raters coded 297 verbalizations with high interrater agreement (Kappa = 0.89). When the possibility of cancer was initially verbalized, the odds of subsequently diagnosing it were on average 5 times higher (odds ratio 4.90 [95% CI 2.72 to 8.84], P < 0.001), while the odds of appropriate referral doubled (OR 1.98 [1.10 to 3.57], P = 0.002). The number of cancer-related questions physicians asked mediated the relationship between first impressions and subsequent diagnosis, explaining 29% of the total effect. Conclusion. We measured a strong association between family physicians’ first diagnostic impressions and subsequent diagnoses and decisions. We suggest that interventions to influence and support the diagnostic process should target its early stage of hypothesis generation.

Journal article

Porat T, Kostopoulou O, Woolley A, Delaney BCet al., 2015, Eliciting user decision requirements for designing computerized diagnostic support for family physicians, Journal of Cognitive Engineering and Decision Making, Vol: 10, Pages: 57-73, ISSN: 1555-3434

Despite its 40-year history, computerized diagnostic support is not used in routine clinical practice. As part of a European project to develop computerized diagnostic support for family physicians, we identified user decision requirements and made design recommendations. To this end, we employed multiple data types and sources. All data were elicited from U.K. family physicians and pertained to consultations with patients, either real or simulated. To elicit user requirements, we conducted in situ observations and interviews with eight physicians and performed a hierarchical task analysis of the diagnostic task. We also analyzed 34 think-aloud transcripts of 17 family physicians diagnosing detailed patient scenarios on a computer and 24 interview transcripts of 18 family physicians describing past cases of intuitive diagnoses from their experience. All transcripts were coded using the situation assessment record (SAR) method. We report our methods and results using the decision-centered design framework. Studies employing multiple human factors techniques and data types in order to elicit user requirements are rare. Our approach enabled us to propose interface design recommendations that go beyond existing “differential diagnosis generators,” with the aim to improve physicians’ performance and acceptance of the resulting tool.

Journal article

Nurek M, Kostopoulou O, Delaney BC, Esmail Aet al., 2015, Reducing diagnostic errors in primary care. A systematic meta-review of computerized diagnostic decision support systems by the LINNEAUS collaboration on patient safety in primary care, European Journal of General Practice, Vol: 21, Pages: 8-13, ISSN: 1751-1402

BACKGROUND: Computerized diagnostic decision support systems (CDDSS) have the potential to support the cognitive task of diagnosis, which is one of the areas where general practitioners have greatest difficulty and which accounts for a significant proportion of adverse events recorded in the primary care setting. OBJECTIVE: To determine the extent to which CDDSS may meet the requirements of supporting the cognitive task of diagnosis, and the currently perceived barriers that prevent the integration of CDDSS with electronic health record (EHR) systems. METHODS: We conducted a meta-review of existing systematic reviews published in English, searching MEDLINE, Embase, PsycINFO and Web of Knowledge for articles on the features and effectiveness of CDDSS for medical diagnosis published since 2004. Eligibility criteria included systematic reviews where individual clinicians were primary end users. Outcomes we were interested in were the effectiveness and identification of specific features of CDDSS on diagnostic performance. RESULTS: We identified 1970 studies and excluded 1938 because they did not fit our inclusion criteria. A total of 45 articles were identified and 12 were found suitable for meta-review. Extraction of high-level requirements identified that a more standardized computable approach is needed to knowledge representation, one that can be readily updated as new knowledge is gained. In addition, a deep integration with the EHR is needed in order to trigger at appropriate points in cognitive workflow. CONCLUSION: Developing a CDDSS that is able to utilize dynamic vocabulary tools to quickly capture and code relevant diagnostic findings, and coupling these with individualized diagnostic suggestions based on the best-available evidence has the potential to improve diagnostic accuracy, but requires evaluation.

Journal article

Kostopoulou O, Lionis C, Angelaki A, Ayis S, Durbaba S, Delaney BCet al., 2015, Early diagnostic suggestions improve accuracy of family physicians: a randomized controlled trial in Greece., Family Practice, Vol: 32, Pages: 323-328, ISSN: 1460-2229

BACKGROUND: In a recent randomized controlled trial, providing UK family physicians with 'early support' (possible diagnoses to consider before any information gathering) was associated with diagnosing hypothetical patients on computer more accurately than control. Another group of physicians, who gathered information, gave a diagnosis, and subsequently received a list of possible diagnoses to consider ('late support'), were no more accurate than control, despite being able to change their initial diagnoses. OBJECTIVE: To replicate the UK study findings in another country with a different primary health care system. METHODS: All study materials were translated into Greek. Greek family physicians were randomly allocated to one of three groups: control, early support and late support. Participants saw nine scenarios in random order. After reading some information about the patient and the reason for encounter, they requested more information to diagnose. The main outcome measure was diagnostic accuracy. RESULTS: One hundred fifty Greek family physicians participated. The early support group was more accurate than control [odds ratio (OR): 1.67 (1.21-2.31)]. Like their UK counterparts, physicians in the late support group rarely changed their initial diagnoses after receiving support. The pooled OR for the early support versus control comparison from the meta-analysis of the UK and Greek data was 1.40 (1.13-1.67). CONCLUSION: Using the same methodology with a different sample of family physicians in a different country, we found that suggesting diagnoses to consider before physicians start gathering information was associated with more accurate diagnoses. This constitutes further supportive evidence of a generalizable effect of early support.

Journal article

Delaney BC, Curcin V, Andreasson A, Arvanitis TN, Bastiaens H, Corrigan D, Ethier JF, Kostopoulou O, Kuchinke W, McGilchrist M, van Royen P, Wagner Pet al., 2015, Translational Medicine and Patient Safety in Europe: TRANSFoRm-Architecture for the Learning Health System in Europe., Biomed Research International, Vol: 2015, ISSN: 2314-6133

The Learning Health System (LHS) describes linking routine healthcare systems directly with both research translation and knowledge translation as an extension of the evidence-based medicine paradigm, taking advantage of the ubiquitous use of electronic health record (EHR) systems. TRANSFoRm is an EU FP7 project that seeks to develop an infrastructure for the LHS in European primary care. Methods. The project is based on three clinical use cases, a genotype-phenotype study in diabetes, a randomised controlled trial with gastroesophageal reflux disease, and a diagnostic decision support system for chest pain, abdominal pain, and shortness of breath. Results. Four models were developed (clinical research, clinical data, provenance, and diagnosis) that form the basis of the projects approach to interoperability. These models are maintained as ontologies with binding of terms to define precise data elements. CDISC ODM and SDM standards are extended using an archetype approach to enable a two-level model of individual data elements, representing both research content and clinical content. Separate configurations of the TRANSFoRm tools serve each use case. Conclusions. The project has been successful in using ontologies and archetypes to develop a highly flexible solution to the problem of heterogeneity of data sources presented by the LHS.

Journal article

Vadillo MA, Kostopoulou O, Shanks DR, 2015, A critical review and meta-analysis of the unconscious thought effect in medical decision making, FRONTIERS IN PSYCHOLOGY, Vol: 6, ISSN: 1664-1078

Journal article

Woolley A, Kostopoulou O, Delaney BC, 2015, Can medical diagnosis benefit from "unconscious thought"?, Medical Decision Making, Vol: 36, Pages: 541-549, ISSN: 1552-681X

The unconscious thought theory argues that making complex decisions after a period of distraction can lead to better decision quality than deciding either immediately or after conscious deliberation. Two studies have tested this unconscious thought effect (UTE) in clinical diagnosis with conflicting results. The studies used different methodologies and had methodological weaknesses. We attempted to replicate the UTE in medical diagnosis by providing favorable conditions for the effect while maintaining ecological validity. Family physicians (N= 116) diagnosed 3 complex cases in 1 of 3 thinking modes: immediate, unconscious (UT), and conscious (CT). Cases were divided into short sentences, which were presented briefly and sequentially on computer. After each case presentation, the immediate response group gave a diagnosis, the UT group performed a 2-back distraction task for 3 min before giving a diagnosis, and the CT group could take as long as necessary before giving a diagnosis. We found no differences in diagnostic accuracy between groups (P= 0.95). The CT group took a median of 7 s to diagnose, which suggests that physicians were able to diagnose "online," as information was being presented. The lack of a difference between the immediate and UT groups suggests that the distraction had no additional effect on performance. To assess the decisiveness of the evidence of this null result, we computed a Bayes factor (BF01) for the 2 comparisons of interest. We found a BF01of 5.76 for the UT versus immediate comparison and of 3.61 for the UT versus CT comparison. Both BFs provide substantial evidence in favor of the null hypothesis: physicians' diagnoses made after distraction are no better than diagnoses made either immediately or after self-paced deliberation.

Journal article

Kostopoulou O, Rosen A, Round T, Wright E, Douiri A, Delaney Bet al., 2015, Early diagnostic suggestions improve accuracy of GPs: a randomised controlled trial using computer-simulated patients, BRITISH JOURNAL OF GENERAL PRACTICE, Vol: 65, Pages: E49-E54, ISSN: 0960-1643

Journal article

Nurek M, Kostopoulou O, Hagmayer Y, 2014, Predecisional information distortion in physicians’ diagnostic judgments: Strengthening a leading hypothesis or weakening its competitor?, Judgment and Decision Making, Vol: 9, Pages: 572-585, ISSN: 1930-2975

Decision makers have been found to bias their interpretation of incoming information to support an emerging judgment (predecisional information distortion). This is a robust finding in human judgment, and was recently also established and measured in physicians’ diagnostic judgments (Kostopoulou et al. 2012). The two studies reported here extend this work by addressing the constituent modes of distortion in physicians. Specifically, we studied whether and to what extent physicians distort information to strengthen their leading diagnosis and/or to weaken a competing diagnosis. We used the “stepwise evolution of preference” method with three clinical scenarios, and measured distortion on separate rating scales, one for each of the two competing diagnoses per scenario.In Study 1, distortion in an experimental group was measured against the responses of a separate control group. In Study 2, distortion in a new experimental group was measured against participants’ own, personal responses provided under control conditions, with the two response conditions separated by a month. The two studies produced consistent results. On average, we found considerable distortion of information to weaken the trailing diagnosis but little distortion to strengthen the leading diagnosis. We also found individual differences in the tendency to engage in either mode of distortion. Given that two recent studies found both modes of distortion in lay preference (Blanchard, Carlson & Meloy, 2014; DeKay, Miller, Schley & Erford, 2014), we suggest that predecisional information distortion is affected by participant and task characteristics. Our findings contribute to the growing research on the different modes of predecisional distortion and their stability to methodological variation.

Journal article

Kostopoulou O, Sirota M, Round T, Samaranayaka S, Delaney Bet al., 2014, The role of information gathering and physician experience in detecting early presentations of cancer in primary care, Publisher: WILEY-BLACKWELL, Pages: 29-29, ISSN: 0961-5423

Conference paper

Sirota M, Juanchich M, Kostopoulou O, Hanak Ret al., 2014, Decisive Evidence on a Smaller-Than-YouThink Phenomenon: Revisiting the "1-in-X'' Effect on Subjective Medical Probabilities, MEDICAL DECISION MAKING, Vol: 34, Pages: 419-429, ISSN: 0272-989X

Journal article

This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.

Request URL: http://wlsprd.imperial.ac.uk:80/respub/WEB-INF/jsp/search-html.jsp Request URI: /respub/WEB-INF/jsp/search-html.jsp Query String: respub-action=search.html&id=00568581&limit=30&person=true