Publications
66 results found
Borsci S, Prati E, Malizia A, et al., 2023, Ciao AI: the Italian adaptation and validation of the Chatbot Usability Scale, Personal and Ubiquitous Computing, Vol: 27, Pages: 2161-2170, ISSN: 1617-4909
Chatbot-based tools are becoming pervasive in multiple domains from commercial websites to rehabilitation applications. Only recently, an eleven-item satisfaction inventory was developed (the ChatBot Usability Scale, BUS-11) to help designers in the assessment process of their systems. The BUS-11 has been validated in multiple contexts and languages, i.e., English, German, Dutch, and Spanish. This scale forms a solid platform enabling designers to rapidly assess chatbots both during and after the design process. The present work aims to adapt and validate the BUS-11 inventory in Italian. A total of 1360 questionnaires were collected which related to a total of 10 Italian chatbot-based systems using the BUS-11 inventory and also using the lite version of the Usability Metrics for User eXperience for convergent validity purposes. The Italian version of the BUS-11 was adapted in terms of the wording of one item, and a Multi-Group Confirmatory Factorial Analysis was performed to establish the factorial structure of the scale and compare the effects of the wording adaptation. Results indicate that the adapted Italian version of the scale matches the expected factorial structure of the original scale. The Italian BUS-11 is highly reliable (Cronbach alpha: 0.921), and it correlates to other measures of satisfaction (e.g., UMUX-Lite, τb = 0.67; p <.001) by also offering specific insights regarding the chatbots’ characteristics. The Italian BUS-11 can be confidently used by chatbot designers to assess the satisfaction of their users during formative or summative tests.
Di Bidino R, Piaggio D, Andellini M, et al., 2023, Scoping Meta-Review of Methods Used to Assess Artificial Intelligence-Based Medical Devices for Heart Failure., Bioengineering (Basel), Vol: 10, ISSN: 2306-5354
Artificial intelligence and machine learning (AI/ML) are playing increasingly important roles, permeating the field of medical devices (MDs). This rapid progress has not yet been matched by the Health Technology Assessment (HTA) process, which still needs to define a common methodology for assessing AI/ML-based MDs. To collect existing evidence from the literature about the methods used to assess AI-based MDs, with a specific focus on those used for the management of heart failure (HF), the International Federation of Medical and Biological Engineering (IFMBE) conducted a scoping meta-review. This manuscript presents the results of this search, which covered the period from January 1974 to October 2022. After careful independent screening, 21 reviews, mainly conducted in North America and Europe, were retained and included. Among the findings were that deep learning is the most commonly utilised method and that electronic health records and registries are among the most prevalent sources of data for AI/ML algorithms. Out of the 21 included reviews, 19 focused on risk prediction and/or the early diagnosis of HF. Furthermore, 10 reviews provided evidence of the impact on the incidence/progression of HF, and 13 on the length of stay. From an HTA perspective, the main areas requiring improvement are the quality assessment of studies on AI/ML (included in 11 out of 21 reviews) and their data sources, as well as the definition of the criteria used to assess the selection of the most appropriate AI/ML algorithm.
Borsci S, Lehtola VV, Nex F, et al., 2023, Embedding artificial intelligence in society: looking beyond the EU AI master plan using the culture cycle, AI & SOCIETY, Vol: 38, Pages: 1465-1484, ISSN: 0951-5666
- Author Web Link
- Cite
- Citations: 5
Federici S, Bifolchi G, Bracalenti M, et al., 2023, eGLU-Box Mobile: A Smartphone App for Usability Testing by Italian Public Administration Webmasters, Pages: 64-79, ISSN: 0302-9743
Smartphones and tablets now offer consumers unique advantages such as portability and accessibility. Developers are also working with a mobile-first approach, and are prioritizing mobile applications over desktop versions. This study introduces eGLU-box Mobile, an application for performing a drive usability test directly from a smartphone. An experimental study was conducted in which the participants were divided into two groups: an experimental group, which used the new mobile application from a smartphone, and a control group, which used the desktop application from a computer. The participants’ behavior was assessed using explicit (self-report questionnaires) and implicit measures (eye movement data). The results were encouraging, and showed that both the mobile and desktop versions of eGLU-box enabled participants to test the usability with a similar level of UX, despite some minimal (although significant) differences in terms of satisfaction of use.
Lehtola VV, Koeva M, Elberink SO, et al., 2022, Digital twin of a city: Review of technology serving city needs, INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION, Vol: 114, ISSN: 1569-8432
- Author Web Link
- Cite
- Citations: 16
Borsci S, Prati E, Federici S, et al., 2022, “Ciao AI”: The Italian adaptation and validation of the Chatbot Usability Scale
<p>Chatbot-based tools becoming pervasive in multiple domains from commercial websites to rehabilitation applications. Only recently an eleven-item satisfaction inventory was developed (the chatBot Usbaliy Scale, BUS-11) to help designers in the assessment process of their systems. The BUS-11 has been validated in multiple contexts and languages i.e., English, German, Dutch, and Spanish. This scale forms a solid platform enabling designers to rapidly assess chatbots both during and after the design process. The present work aims to adapt and validate the BUS-11 inventory in Italian. A total of 1360 questionnaires were collected which related to a total of 10 Italian chatbot-based systems using the BUS-11 inventory and also using the lite version of the Usability Metrics for User eXperience for convergent validity purposes. The Italian version of the BUS-11 was adapted in terms of the wording of one item, and a Multi-Group Confirmatory Factorial Analysis was performed to establish the factorial structure of the scale and compare the effects of the wording adaptation. Results indicate that the adapted Italian version of the scale matches the expected factorial structure of the original scale. The Italian BUS-11 is highly reliable (Cronbach alpha: 0.921), and it correlates to other measures of satisfaction (e.g., UMUX-Lite, τb=0.67; p&lt; .001 ) by also offering specific insights regarding the chatbots’ characteristics. The Italian BUS-11 can be confidently used by chatbot designers to assess the satisfaction of their users during formative or summative tests.</p>
Federici S, De Filippis ML, Mele ML, et al., 2022, Approaches Adopted by Researchers to Measure the Quality of the Experience of People Working from Home: a Scoping Review., J Technol Behav Sci, Vol: 7, Pages: 451-467, ISSN: 2366-5963
UNLABELLED: Working from home (WFH) remotely is a modality of working that requires the careful design of systems of rules and tools to enable people to exchange information and perform actions. WFH is expected to expand after the COVID-19 pandemic. How to assess and compare in a reliable way the experience of workers with different (sociotechnical) systems of WFH is a central point to supporting the diffusion of acceptable modalities of working. However, the concept of experience and how it can be measured in the domain in WFH is yet to be clearly characterized. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology for scoping reviews, we systematically map the approaches used by researchers to assess WFH, identify which aspects are usually investigated, and examine how such aspects are usually measured in terms of questions and tools. Literature is collected using Scopus and Web of Science. Thirty-four records out of 323 focusing either on validating a scale, presenting theoretically the experience of workers or testing this empirically are included in the qualitative synthesis. The results highlight a lack of unified terminology and tools, with assessments of workers' experience mainly characterized by survey approaches and qualitative questions. Clustering together the most investigated aspects in the literature and reviewing how these aspects are assessed, we propose a list of 10 relevant overarching dimensions and attempt to define workers' experience in the domain of WFH remotely. This definition can be used as a tool by researchers aiming to assess the experience of workers in order to inform the design or redesign of the sociotechnical systems that enable WFH. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s41347-022-00264-4.
Federici S, Bifolchi G, Mele ML, et al., 2022, Remote Working: A Way to Foster Greater Inclusion and Accessibility?, Editors: Miesenberger, Kouroupetroglou, Mavrou, Manduchi, Rodriguez, Penaz, Publisher: SPRINGER INTERNATIONAL PUBLISHING AG, Pages: 192-199, ISBN: 978-3-031-08644-1
Federici S, De Filippis ML, Mele ML, et al., 2021, Measuring the experience of remote home workers: A scoping review
<p>Introduction: Working from home (WFH) remotely is a modality of working that requires the careful design of systems of rules and tools to enable people to exchange information and perform actions. WFH is expected to expand after the COVID-19 pandemic, and how best to reliably assess and compare the experience of workers with different (sociotechnical) systems of WFH is central to the diffusion of acceptable modalities of remote working. However, the concept of experience and how it can be measured in the domain in WFH is yet to be clearly characterized.Methods: Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology for scoping reviews, we systematically map the approaches used by researchers to assess WFH, identify which aspects are usually investigated, and examine how such aspects are usually measured in terms of questions and tools. Literature is collected using Scopus and Web of Science.Results: Thirty-four records out of 323 focusing either on validating a scale, presenting theoretically the experience of workers or testing this empirically are included in the qualitative synthesis. The results highlight a lack of unified terminology and tools, with assessments of workers’ experience mainly characterized by survey approaches and qualitative questions. Conclusion: Clustering together the most investigated aspects in the literature and reviewing how these aspects are assessed, we propose a list of 10 relevant overarching dimensions and attempt to define workers’ experience in the domain of WFH remotely. This definition can be used as a tool by researchers aiming to assess the experience of workers in order to inform the design or redesign of the sociotechnical systems that enable WFH.</p>
Borsci S, Malizia A, Schmettow M, et al., 2021, The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents, PERSONAL AND UBIQUITOUS COMPUTING, Vol: 26, Pages: 95-119, ISSN: 1617-4909
- Author Web Link
- Cite
- Citations: 1
Micocci M, Borsci S, Thakerar V, et al., 2021, Attitudes towards trusting artificial intelligence insights and factors to prevent the passive adherence of GPs: a pilot study, Journal of Clinical Medicine, Vol: 10, Pages: 1-11, ISSN: 2077-0383
Artificial Intelligence (AI) systems could improve system efficiency by supporting clinicians in making appropriate referrals. However, they are imperfect by nature and misdiagnoses, if not correctly identified, can have consequences for patient care. In this paper, findings from an online survey are presented to understand the aptitude of GPs (n = 50) in appropriately trusting or not trusting the output of a fictitious AI-based decision support tool when assessing skin lesions, and to identify which individual characteristics could make GPs less prone to adhere to erroneous diagnostics results. The findings suggest that, when the AI was correct, the GPs’ ability to correctly diagnose a skin lesion significantly improved after receiving correct AI information, from 73.6% to 86.8% (X2 (1, N = 50) = 21.787, p < 0.001), with significant effects for both the benign (X2 (1, N = 50) = 21, p < 0.001) and malignant cases (X2 (1, N = 50) = 4.654, p = 0.031). However, when the AI provided erroneous information, only 10% of the GPs were able to correctly disagree with the indication of the AI in terms of diagnosis (d-AIW M: 0.12, SD: 0.37), and only 14% of participants were able to correctly decide the management plan despite the AI insights (d-AIW M:0.12, SD: 0.32). The analysis of the difference between groups in terms of individual characteristics suggested that GPs with domain knowledge in dermatology were better at rejecting the wrong insights from AI. View Full-Text
Polisena J, Andellini M, Salerno P, et al., 2021, Case Studies on the Use of Sentiment Analysis to Assess the Effectiveness and Safety of Health Technologies: A Scoping Review, IEEE ACCESS, Vol: 9, Pages: 66043-66051, ISSN: 2169-3536
- Author Web Link
- Cite
- Citations: 4
Federici S, Mele ML, Bracalenti M, et al., 2021, A Chatbot Solution for eGLU-Box Pro: The Usability Evaluation Platform for Italian Public Administrations, Editors: Kurosu, Publisher: SPRINGER INTERNATIONAL PUBLISHING AG, Pages: 268-279, ISBN: 978-3-030-78461-4
- Author Web Link
- Cite
- Citations: 3
Federici S, de Filippis ML, Mele ML, et al., 2020, Inside pandora's box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs, DISABILITY AND REHABILITATION-ASSISTIVE TECHNOLOGY, Vol: 15, Pages: 832-837, ISSN: 1748-3107
- Author Web Link
- Cite
- Citations: 9
Vijayasingam A, Frost E, Wilkins J, et al., 2020, Tablet and web-based audiometry to screen for hearing loss in adults with cystic fibrosis, Thorax, Vol: 75, Pages: 632-639, ISSN: 0040-6376
INTRODUCTION: Individuals with chronic lung disease (eg, cystic fibrosis (CF)) often receive antimicrobial therapy including aminoglycosides resulting in ototoxicity. Extended high-frequency audiometry has increased sensitivity for ototoxicity detection, but diagnostic audiometry in a sound-booth is costly, time-consuming and requires a trained audiologist. This cross-sectional study analysed tablet-based audiometry (Shoebox MD) performed by non-audiologists in an outpatient setting, alongside home web-based audiometry (3D Tune-In) to screen for hearing loss in adults with CF. METHODS: Hearing was analysed in 126 CF adults using validated questionnaires, a web self-hearing test (0.5 to 4 kHz), tablet (0.25 to 12 kHz) and sound-booth audiometry (0.25 to 12 kHz). A threshold of ≥25 dB hearing loss at ≥1 audiometric frequency was considered abnormal. Demographics and mitochondrial DNA sequencing were used to analyse risk factors, and accuracy and usability of hearing tests determined. RESULTS: Prevalence of hearing loss within any frequency band tested was 48%. Multivariate analysis showed age (OR 1.127; (95% CI: 1.07 to 1.18; p value<0.0001) per year older) and total intravenous antibiotic days over 10 years (OR 1.006; (95% CI: 1.002 to 1.010; p value=0.004) per further intravenous day) were significantly associated with increased risk of hearing loss. Tablet audiometry had good usability, was 93% sensitive, 88% specific with 94% negative predictive value to screen for hearing loss compared with web self-test audiometry and questionnaires which had poor sensitivity (17% and 13%, respectively). Intraclass correlation (ICC) of tablet versus sound-booth audiometry showed high correlation (ICC >0.9) at all frequencies ≥4 kHz. CONCLUSIONS: Adults with CF have a high prevalence of drug-related hearing loss and tablet-based audiometry can be a practical, accurate screening tool within integrated ototoxicity monitoring programmes for early detection.
Borsci S, Buckle P, Walne S, 2020, Is the LITE version of the usability metric for user experience (UMUX-LITE) a reliable tool to support rapid assessment of new healthcare technology?, Applied Ergonomics: human factors in technology and society, Vol: 84, ISSN: 0003-6870
ObjectiveTo ascertain the reliability of a standardised, short-scale measure of satisfaction in the use of new healthcare technology i.e., the LITE version of the usability metric for user experience (UMUX-LITE). Whilst previous studies have demonstrated the reliability of UMUX-LITE, and its relationship with measures of likelihood to recommend a product, such as the Net Promoter Score (NPS) in other sectors no such testing has been undertaken with healthcare technology.Materials and methodsSix point-of-care products at different stages of development were assessed by 120 healthcare professionals. UMUX-LITE was used to gather their satisfaction in use, and NPS to declare their intention to promote the product. Inferential statistics were used to: i) ascertain the reliability of UMUX-LITE, and ii) assess the relationship between UMUX-LITE and NPS at different stages of products development.ResultsUMUX-LITE showed an acceptable reliability (α = 0.7) and a strong positive correlation with NPS (r = 0.455, p < .001). This is similar to findings in other fields of application. The level of product development did not affect the UMUX-LITE scores, while the stage of development was a significant predictor (R2 = 0.49) of the intention to promote.Discussion and conclusionPractitioners may apply UMUX-LITE alone, or in combination with the NPS, to complement interview and ‘homemade’ scales to investigate the quality of new products at different stages of development. This shortened scale is appropriate for use in the context of healthcare in which busy professionals have a minimal amount of time to support innovation.
Ni M, Borsci S, Walne S, et al., 2020, The Lean and Agile Multi-dimensional Process (LAMP) - a new framework for rapid and iterative evidence generation to support health-care technology design and development, Expert Review of Medical Devices, Vol: 17, Pages: 277-288, ISSN: 1743-4440
Introduction: Health technology assessments (HTA) are tools for policymaking and resource allocation. Early HTAs are increasingly used in design and development of new technologies. Conducting early HTAs is challenging, due to a lack of evidence and significant uncertainties in the technology and the market. A multi-disciplinary approach is considered essential. However, an operational framework that can enable the integration of multi-dimensional evidence into commercialization remains lacking.Areas covered: We developed the Lean and Agile Multi-dimensional Process (LAMP), an early HTA framework, for embedding commercial decision-making in structured evidence generation activities, divided into phases. Diverse evidence in unmet needs, user acceptance, cost-effectiveness, and market competitiveness are being generated in increasing depth. This supports the emergence of design and value propositions that align technology capabilities and clinical and user needs.Expert opinion: We have been applying LAMP to working with medical device and diagnostic industry in the UK. The framework can be adapted to suit different technologies, decision needs, time scales, and resources. LAMP offers a practical solution to the multi-disciplinary approach. Methodologists drive the process by performing evidence generation and synthesis as and by enabling interactions between manufacturers, designers, clinicians, and other key stakeholders.
de Filippis ML, Federici S, Mele ML, et al., 2020, Preliminary results of a systematic review: Quality assessment of conversational agents (chatbots) for people with disabilities or special needs, Pages: 250-257, ISBN: 9783030587956
People with disabilities or special needs can benefit from AI-based conversational agents, which are used in competence training and well-being management. Assessment of the quality of interactions with these chatbots is key to being able to reduce dissatisfaction with them and to understand their potential long-term benefits. This will in turn help to increase adherence to their use, thereby improving the quality of life of the large population of end-users that they are able to serve. We systematically reviewed the literature on methods of assessing the perceived quality of interactions with chatbots, and identified only 15 of 192 papers on this topic that included people with disabilities or special needs in their assessments. The results also highlighted the lack of a shared theoretical framework for assessing the perceived quality of interactions with chatbots. Systematic procedures based on reliable and valid methodologies continue to be needed in this field. The current lack of reliable tools and systematic methods for assessing chatbots for people with disabilities and special needs is concerning, and may lead to unreliable systems entering the market with disruptive consequences for users. Three major conclusions can be drawn from this systematic analysis: (i) researchers should adopt consolidated and comparable methodologies to rule out risks in use; (ii) the constructs of satisfaction and acceptability are different, and should be measured separately; (iii) dedicated tools and methods for assessing the quality of interaction with chatbots should be developed and used to enable the generation of comparable evidence.
Borsci S, Federici S, Malizia A, et al., 2019, Shaking the usability tree: why usability is not a dead end, and a constructive way forward, BEHAVIOUR & INFORMATION TECHNOLOGY, Vol: 38, Pages: 519-532, ISSN: 0144-929X
- Author Web Link
- Cite
- Citations: 17
Borsci S, Buckle P, Walne S, et al., 2019, Trust and Human Factors in the Design of Healthcare Technology, 20th Congress of the International-Ergonomics-Association (IEA), Publisher: SPRINGER INTERNATIONAL PUBLISHING AG, Pages: 218-226, ISSN: 2194-5357
Hummel JM, Borsci S, Fico G, 2019, Multicriteria decision aiding for early health technology assessment of medical devices, Clinical Engineering Handbook, Second Edition, Pages: 807-811, ISBN: 9780128134672
The value of technology to health care is not dependent on clinical and economic considerations alone. In this chapter we describe health related as well as economic, social, legal, ethical, and organizational criteria to include in early health technology assessment. Moreover, we provide recommendations on methods to collect data on these criteria, and how to integrate these finding by means of multicriteria decision analysis. Such early assessments of biomedical innovations can provide developers with recommendations on which technological innovation to invest in, and how to optimize the value of this innovation.
Borsci S, David LZ, 2019, Human factors and system thinking for medical device, Clinical Engineering Handbook, Second Edition, Pages: 829-831, ISBN: 9780128134672
The main role of human factors and ergonomics professionals within health care is to generate evidence to support the development and the redesign of services and products. To achieve safe and resilient health system environments, both technology and people should collaborate in a resonant way by ensuring an optimal experience to patients. Human factors and ergonomics methods of investigation aim to generate evidence to inform decisions about appropriate selection of a device, processes resilience, human reliability, and safety of healthcare processes and products. Clinical settings are continuously changing and evolving. Human factors and ergonomics bring in the clinical engineering team a systems thinking approach that could be used to map the context of use and to inform about the operational costs and barriers associated to the use of device in the field-e.g., time to perform, time to obtain the results, time for decision-making, people involved in the process and stakeholders, etc. These analyses may add nuances to gray areas which are usually only partially informed by premarket research and add to health economics evaluation. In line with that, human factors and ergonomics methods are essential tools to define, model, and evaluate the value added by a new product or service in the health environment and to engineer, during the development, the value of new solutions.
Roberts HW, Wagh VK, Mullens IJM, et al., 2018, Evaluation of a hub-and-spoke model for the delivery of femtosecond laser-assisted cataract surgery within the context of a large randomised controlled trial, British Journal of Ophthalmology, Vol: 102, Pages: 1556-1563, ISSN: 0007-1161
AIMS: To test a hypothesis that cataract operating room (OR) productivity can be improved with a femtosecond laser (FL) using a hub-and-spoke model and whether any increase in productivity can offset additional costs relating to the FL. METHODS: 400 eyes of 400 patients were enrolled in a randomised-controlled trial comparing FL-assisted cataract surgery (FLACS) with conventional phacoemulsification surgery (CPS). 299 of 400 operations were performed on designated high-volume theatre lists (FLACS=134, CPS=165), where a hub-and-spoke FLACS model (1×FL, 2×ORs=2:1) was compared with independent CPS theatre lists. Details of operative timings and OR utilisation were recorded. Differences in productivity between hub-and-spoke FLACS and CPS sessions were compared using an economic model including testing hypothetical 3:1 and 4:1 models. RESULTS: The duration of the operation itself was 12.04±4.89 min for FLACS compared with CPS of 14.54±6.1 min (P<0.001). Total patient time in the OR was reduced from 23.39±6.89 min with CPS to 20.34±5.82 min with FLACS (P<0.001)(reduction of 3.05 min per case). There was no difference in OR turnaround time between the models. Average number of patients treated per theatre list was 9 for FLACS and 8 for CPS. OR utilisation was 92.08% for FLACS and 95.83% for CPS (P<0.001). Using a previously established economic model, the FLACS service cost £144.60 more than CPS per case. This difference would be £131 and £125 for 3:1 and 4:1 models, respectively. CONCLUSION: The FLACS hub-and-spoke model was significantly faster than CPS, with patients spending less time in the OR. This enabled an improvement in productivity, but insufficient to meaningfully offset the additional costs relating to FLACS.
Roberts HW, Myerscough J, Borsci S, et al., 2018, Time and motion studies of National Health Service cataract theatre lists to determine strategies to improve efficiency, British Journal of Ophthalmology, Vol: 102, Pages: 1259-1267, ISSN: 0007-1161
Aim To provide a quantitative assessment of cataract theatre lists focusing on productivity and staffing levels/tasks using time and motion studies.Methods National Health Service (NHS) cataract theatre lists were prospectively observed in five different institutions (four NHS hospitals and one private hospital). Individual tasks and their timings of every member of staff were recorded. Multiple linear regression analyses were performed to investigate possible associations between individual timings and tasks.Results 140 operations were studied over 18 theatre sessions. The median number of scheduled cataract operations was 7 (range: 5–14). The average duration of an operation was 10.3 min±(SD 4.11 min). The average time to complete one case including patient turnaround was 19.97 min (SD 8.77 min). The proportion of the surgeons’ time occupied on total duties or operating ranged from 65.2% to 76.1% and from 42.4% to 56.7%, respectively. The correlations of the surgical time to patient time in theatre was R2=0.95. A multiple linear regression model found a significant association (F(3,111)=32.86, P<0.001) with R2=0.47 between the duration of one operation and the number of allied healthcare professionals (AHPs), the number of AHP key tasks and the time taken to perform these key tasks by the AHPs.Conclusions Significant variability in the number of cases performed and the efficiency of patient flow were found between different institutions. Time and motion studies identified requirements for high-volume models and factors relating to performance. Supporting the surgeon with sufficient AHPs and tasks performed by AHPs could improve surgical efficiency up to approximately double productivity over conventional theatre models.
Polisena J, Castaldo R, Ciani O, et al., 2018, HEALTH TECHNOLOGY ASSESSMENT METHODS GUIDELINES FOR MEDICAL DEVICES: HOW CAN WE ADDRESS THE GAPS? THE INTERNATIONAL FEDERATION OF MEDICAL AND BIOLOGICAL ENGINEERING PERSPECTIVE, INTERNATIONAL JOURNAL OF TECHNOLOGY ASSESSMENT IN HEALTH CARE, Vol: 34, Pages: 276-289, ISSN: 0266-4623
- Author Web Link
- Cite
- Citations: 22
Borsci S, Uchegbu I, Buckle P, et al., 2017, Designing medical technology for resilience: Integrating health economics and human factors approaches, Expert Review of Medical Devices, Vol: 15, Pages: 15-26, ISSN: 1743-4440
INTRODUCTION: The slow adoption of innovation into healthcare calls into question the manner of evidence generation for medical technology. This paper identifies potential reasons for this including a lack of attention to human factors, poor evaluation of economic benefits, lack of understanding of the existing healthcare system and a failure to recognise the need to generate resilient products. Areas covered. Recognising a cross-disciplinary need to enhance evidence generation early in a technology's life cycle, the present paper proposes a new approach that integrates human factors and health economic evaluation as part of a wider systems approach to the design of technology. This approach (Human and Economic Resilience Design for Medical Technology or HERD MedTech) supports early stages of product development and is based on the recent experiences of the National Institute for Health Research London Diagnostic Evidence Co-operative in the UK. Expert commentary. HERD MedTech i) proposes a shift from design for usability to design for resilience, ii) aspires to reduce the need for service adaptation to technological constraints iii) ensures value of innovation at the time of product development, and iv) aims to stimulate discussion around the integration of pre- and post-market methods of assessment of medical technology.
Borsci S, Buckle P, Huddy J, et al., 2017, Usability study of pH strips for nasogastric tube placement, PLoS ONE, Vol: 12, Pages: 1-14, ISSN: 1932-6203
Aims(1) To model the process of use and usability of pH strips (2) to identify, through simulation studies, the likelihood of misreading pH strips, and to assess professional’s acceptance, trust and perceived usability of pH strips.MethodsThis study was undertaken in four phases and used a mixed method approach (an audit, a semi-structured interview, a survey and simulation study). The three months audit was of 24 patients, the semi-structured interview was performed with 19 health professionals and informed the process of use of pH strips. A survey of 134 professionals and novices explored the likelihood of misinterpreting pH strips. Standardised questionnaires were used to assess professionals perceived usability, trust and acceptance of pH strip use in a simulated study.ResultsThe audit found that in 45.7% of the cases aspiration could not be achieved, and that 54% of the NG-tube insertions required x-ray confirmation. None of those interviewed had received formal training on pH strips use. In the simulated study, participants made up to 11.15% errors in reading the strips with important implications for decision making regarding NG tube placement. No difference was identified between professionals and novices in their likelihood of misinterpreting the pH value of the strips. Whilst the overall experience of usage is poor (47.3%), health professionals gave a positive level of trust in both the interview (62.6%) and the survey (68.7%) and acceptance (interview group 65.1%, survey group 74.7%). They also reported anxiety in the use of strips (interview group 29.7%, survey group 49.7%).ConclusionsSignificant errors occur when using pH strips in a simulated study. Manufacturers should consider developing new pH strips, specifically designed for bedside use, that are more usable and less likely to be misread.
Borsci S, Buckle P, Uchegbu I, et al., 2017, Integrating human factors and health economics to inform the design of medical device: a conceptual framework, EMBEC & NBC 2017: Joint Conference of the European Medical and Biological Engineering Conference (EMBEC) and the Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC)
Federici S, Meloni F, Borsci S, 2016, The abandonment of assistive technology in Italy: a survey of National Health Service users., European Journal of Physical and Rehabilitation Medicine, Vol: 52, Pages: 516-526, ISSN: 1973-9095
BACKGROUND: This study was an extension of research which began in the Umbria region in 2009. AIM: To investigate the extent to which assistive technology (AT) has been abandoned by users of the Italian National Health Service (ULHS) and the reasons for this. DESIGN: Observational study. SETTING: Users who received a hearing device (HD) or mobility device (MD) by ULHS between 2010 and 2013. POPULATION: 749 out of 3,791 ULHS users contacted via telephone completed the interview: 330 (44.06%) had a HD and 419 (55.94%) a MD. METHODS: Data were collected using a specially developed telephone interview questionnaire including the Italian version of the Quebec User Evaluation of Satisfaction with AT (QUEST 2.0) and Assistive Technology Use Follow-up Survey (ATUFS). RESULTS: 134 users (17.9%) were no longer using their assigned AT device within seven months of issue and 40% of this group reported that they had never used the device. Duration of use (for how long the AT device was used before abandonment) and satisfaction with service delivery did not predict AT abandonment. People who received a HD where more likely to abandon their device (22.4%) than those who received a MD (14.4%). CONCLUSIONS: Abandonment may be due to assignment of inappropriate devices or failure to meet user needs and expectations. These findings are consistent with previous data collected by Federici and Borsci in 2009. Utility of AT in use, reasons of abandonment, and importance of device and service satisfaction for the use or non-use of an AT are presented and discussed. CLINICAL REHABILITATION IMPACT: AT abandonment surveys provide useful information for modelling AT assessment and delivery process. The study confirms the relevance of person centredness approach for a successful AT assessment and delivery process.
Borsci S, Lawson G, Salanitri D, et al., 2016, When simulated environments make the difference: the effectiveness of different types of training of car service procedures, Virtual Reality, Vol: 20, Pages: 83-99, ISSN: 1434-9957
An empirical analysis was performed to compare the effectiveness of different approaches to traininga set of procedural skills to a sample of novice trainees. Sixty-five participants were randomlyassigned to one of the following three training groups: i) learning-by-doing in a 3D desktop virtualenvironment, ii) learning-by-observing a video (show-and-tell) explanation of the procedures, and iii)trial-and-error. In each group participants were trained on two car service procedures. Participantswere recalled to perform a procedure either two or four weeks after the training. The results showedthat: i) participants trained through the virtual approach of learning-by-doing performed bothprocedures significantly better (i.e., p <.05 in terms of errors and time) than people of non-virtualgroups. ii) The virtual training group, after a period of non-use, were more effective than non-virtualtraining (i.e., p <.05) in their ability to recover their skills. iii) After a (simulated) long period from thetraining – i.e., up to 12 weeks – people who experienced 3D environments consistently performedbetter than people who received other kinds of training. The results also suggested that independentlyfrom the training group, trainees’ visuospatial abilities were a predictor of performance, at least forthe complex service procedure, Adj R2= .460, and that post-training performances of people trainedthrough virtual learning-by-doing are not affected by learning styles. Finally, a strong relationship(p<..001 R2 =.441) was identified between usability and trust in the use of the virtual training tool –i.e., the more the system was perceived as usable, the more it was perceived as trustable to acquire thecompetences
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.