Publications
8 results found
Khanbhai M, Symons J, Flott K, et al., 2022, Enriching the value of patient experience feedback: interactive dashboard development using co-design and heuristic evaluation, JMIR Human Factors, Vol: 9, Pages: 1-14, ISSN: 2292-9495
Background:There is an abundance of patient experience data held within healthcare organisations but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualised data to summarise key patient experience feedback, have the potential to address these issues.Objective:The aim of this study was to develop a patient experience dashboard with an emphasis on FFT reporting as per the national policy drive. An iterative process involving co-design involving key stakeholders was used to develop the dashboard, followed by heuristic usability testing.Methods:A two staged approach was employed; participatory co-design involving 20 co-designers to develop a dashboard prototype followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the Friends and Family Test (FFT) and the themes and sentiment generated was used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score (SUS).Results:Qualitative analysis from the co-design process informed development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. Heuristic evaluation revelated the majority of cumulative scores had no usability problem (n=18), cosmetic problem only (n=7), or minor usability problem (n= 5). Mean SUS was 89.7 (SD 7.9) suggesting an excellent rating.Conclusions:The growing capacity to collect and process patient experience data suggests that data visualisation will be increasingly important in turning the feedback into improvements to care. Through heuristic usability we demonstrated that very large FFT data can be presented into a thematically driven, simple visual display without loss of the nuances and still allow for exploration of the original free-text comments. T
Khanbhai M, Warren L, Symons J, et al., 2022, Using natural language processing to understand, facilitate and maintain continuity in patient experience across transitions of care, International Journal of Medical Informatics, Vol: 157, Pages: 1-7, ISSN: 1386-5056
BackgroundPatient centred care necessitates that healthcare experiences and perceived outcomes be considered across all transitions of care. Information encoded within free-text patient experience comments relating to transitions of care are not captured in a systematic way due to the manual resource required. We demonstrate the use of natural language processing (NLP) to extract meaningful information from the Friends and Family Test (FFT).MethodsFree-text fields identifying favourable service (“What did we do well?”) and areas requiring improvement (“What could we do better?”) were extracted from 69,285 FFT reports across four care settings at a secondary care National Health Service (NHS) hospital. Sentiment and patient experience themes were coded by three independent coders to produce a training dataset. The textual data was standardised with a series of pre-processing techniques and the performance of six machine learning (ML) models was obtained. The best performing ML model was applied to predict the themes and sentiment from the remaining reports. Comments relating to transitions of care were extracted, categorised by sentiment, and care setting to identify the most frequent words/combinations presented as tri-grams and word clouds.ResultsThe support vector machine (SVM) ML model produced the highest accuracy in predicting themes and sentiment. The most frequent single words relating to transition and continuity with a negative sentiment were “discharge” in inpatients and Accident and Emergency, “appointment” in outpatients, and “home’ in maternity. Tri-grams identified from the negative sentiments such as ‘seeing different doctor’, ‘information aftercare lacking’, ‘improve discharge process’ and ‘timing discharge letter’ have highlighted some of the problems with care transitions. None of this information was available from the quantitative data.Conc
Zhang J, Symons J, Agapow P, et al., 2022, Best practices in the real-world data life cycle., PLOS Digit Health, Vol: 1
With increasing digitization of healthcare, real-world data (RWD) are available in greater quantity and scope than ever before. Since the 2016 United States 21st Century Cures Act, innovations in the RWD life cycle have taken tremendous strides forward, largely driven by demand for regulatory-grade real-world evidence from the biopharmaceutical sector. However, use cases for RWD continue to grow in number, moving beyond drug development, to population health and direct clinical applications pertinent to payors, providers, and health systems. Effective RWD utilization requires disparate data sources to be turned into high-quality datasets. To harness the potential of RWD for emerging use cases, providers and organizations must accelerate life cycle improvements that support this process. We build on examples obtained from the academic literature and author experience of data curation practices across a diverse range of sectors to describe a standardized RWD life cycle containing key steps in production of useful data for analysis and insights. We delineate best practices that will add value to current data pipelines. Seven themes are highlighted that ensure sustainability and scalability for RWD life cycles: data standards adherence, tailored quality assurance, data entry incentivization, deploying natural language processing, data platform solutions, RWD governance, and ensuring equity and representation in data.
Philpott-Morgan S, Thakrar DB, Symons J, et al., 2021, Characterising the nationwide burden and predictors of unkept outpatient appointments in the National Health Service in England: A cohort study using a machine learning approach, PLOS MEDICINE, Vol: 18, ISSN: 1549-1277
Khanbhai M, Anyadi P, Symons J, et al., 2021, Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review, BMJ Health & Care Informatics, Vol: 28, ISSN: 2632-1009
Objectives Unstructured free-text patient feedback contains rich information, and analysing these data manually would require a lot of personnel resources which are not available in most healthcare organisations.To undertake a systematic review of the literature on the use of natural language processing (NLP) and machine learning (ML) to process and analyse free-text patient experience data.Methods Databases were systematically searched to identify articles published between January 2000 and December 2019 examining NLP to analyse free-text patient feedback. Due to the heterogeneous nature of the studies, a narrative synthesis was deemed most appropriate. Data related to the study purpose, corpus, methodology, performance metrics and indicators of quality were recorded.Results Nineteen articles were included. The majority (80%) of studies applied language analysis techniques on patient feedback from social media sites (unsolicited) followed by structured surveys (solicited). Supervised learning was frequently used (n=9), followed by unsupervised (n=6) and semisupervised (n=3). Comments extracted from social media were analysed using an unsupervised approach, and free-text comments held within structured surveys were analysed using a supervised approach. Reported performance metrics included the precision, recall and F-measure, with support vector machine and Naïve Bayes being the best performing ML classifiers.Conclusion NLP and ML have emerged as an important tool for processing unstructured free text. Both supervised and unsupervised approaches have their role depending on the data source. With the advancement of data analysis tools, these techniques may be useful to healthcare organisations to generate insight from the volumes of unstructured free-text data.
Sounderajah V, Patel V, Varatharajan L, et al., 2020, Are disruptive innovations recognised in the healthcare literature? A systematic review, BMJ Innovations, Vol: 7, Pages: 208-216, ISSN: 2055-8074
The study aims to conduct a systematic review to characterise the spread and use of the concept of ‘disruptive innovation’ within the healthcare sector. We aim to categorise references to the concept over time, across geographical regions and across prespecified healthcare domains. From this, we further aim to critique and challenge the sector-specific use of the concept. PubMed, Medline, Embase, Global Health, PsycINFO, Maternity and Infant Care, and Health Management Information Consortium were searched from inception to August 2019 for references pertaining to disruptive innovations within the healthcare industry. The heterogeneity of the articles precluded a meta-analysis, and neither quality scoring of articles nor risk of bias analyses were required. 245 articles that detailed perceived disruptive innovations within the health sector were identified. The disruptive innovations were categorised into seven domains: basic science (19.2%), device (12.2%), diagnostics (4.9%), digital health (21.6%), education (5.3%), processes (17.6%) and technique (19.2%). The term has been used with increasing frequency annually and is predominantly cited in North American (78.4%) and European (15.2%) articles. The five most cited disruptive innovations in healthcare are ‘omics’ technologies, mobile health applications, telemedicine, health informatics and retail clinics. The concept ‘disruptive innovation’ has diffused into the healthcare industry. However, its use remains inconsistent and the recognition of disruption is obscured by other types of innovation. The current definition does not accommodate for prospective scouting of disruptive innovations, a likely hindrance to policy makers. Redefining disruptive innovation within the healthcare sector is therefore crucial for prospectively identifying cost-effective innovations.
Symons J, Ashrafian H, Dunscombe R, et al., 2019, From EHR to PHR: Let’s get the UK record straight, BMJ Open, Vol: 9, Pages: 1-5, ISSN: 2044-6055
This article reflects on the changing nature of health information access and the transition of focus from electronic health records (EHRs) to personal health records (PHRs) along with the challenges and need for alignment of national initiatives for EHR and PHR in the National Health Service (NHS) of the United Kingdom (UK).The importance of implementing integrated EHRs as a route to enhance the quality of health delivery has been increasingly understood. EHRs however carry several limitations that include major fragmentation through multiple providers and protocols throughout the NHS. Questions over ownership and control of data further complicate the potential for fully utilising records. Analysing the previous initiatives and the current landscape, we identify that adopting a patient health record system can empower patients and allow better harmonisation of clinical data at a national level. We propose regional PHR “hubs” to provide a universal interface that integrates digital heath data at a regional level with further integration at a national level.We propose that these PHR “hubs” will reduce the complexity of connections, decrease governance challenges and interoperability issues while also providing a safe platform for high-quality scalable and sustainable digital solutions, including artificial intelligence (AI) across the UK NHS, serving as an exemplar for other countries which wish to realise the full value of healthcare records.
Martin G, Ghafur S, Cingolani I, et al., 2019, The effects and preventability of 2627 patient safety incidents related to health information technology failures: a retrospective analysis of 10 years of incident reporting in England and Wales, The Lancet Digital Health, Vol: 1, Pages: e127-e135, ISSN: 2589-7500
BackgroundThe use of health information technology (IT) is rapidly increasing to support improvements in the delivery of care. Although health IT is delivering huge benefits, new technology can also introduce unique risks. Despite these risks, evidence on the preventability and effects of health IT failures on patients is scarce. In our study we therefore sought to evaluate the preventability and effects of health IT failures by examining patient safety incidents in England and Wales.MethodsWe designed our study as a retrospective analysis of 10 years of incident reporting in England and Wales. We used text mining with the words “computer”, “system”, “workstation”, and “network” to explore free-text incident descriptors to identify incidents related to health IT failures following a previously described approach. We then applied an n-gram model of searching to identify contiguous sequences of words and provide spatial context. We examined incident details, recorded harm, and preventability. Standard descriptive statistics were applied. Degree of harm was identified according to standardised definitions and preventability was assessed by two independent reviewers.FindingsWe identified 2627 incidents related to health IT failures. 2557 (97%) of 2627 incidents were assessed for harm (70 incidents were excluded). 2106 (82%) of 2557 health IT failures caused no harm to patients, 331 (13%) caused low harm, 102 (4%) caused moderate harm, 14 (1%) caused severe harm, and four (<1%) contributed to the death of a patient. 1964 (75%) of 2627 incidents were deemed to be preventable.InterpretationHealth IT is fundamental to the delivery of high-quality care, yet there is a poor understanding of the effects of IT failures on patient safety and whether they can be prevented. Failures are complex and involve interlinked aspects of technology, people, and the environment. Health IT failures are undoubtedly a potential source of subst
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.