Applying heuristic evaluation to improve the usability of a telemedicine system.

PubMed

Tang, Zhihua; Johnson, Todd R; Tindall, R Douglas; Zhang, Jiajie

2006-02-01

The development of a telemedicine system should not only take advantage of technological advances but also pay close attention to users and the human issues involved. In this paper we examine the utility of heuristic evaluation in improving the usability of a digital emergency medical services (EMS) system equipped on an ambulance. The digital EMS system used advanced communication technologies to help remotely located trauma specialists gain access to patient data in real-time and direct life-saving measures in a timely fashion. To improve its usability, three experts inspected prototypes of the system according to 14 software usability heuristics. The analyses revealed information on the prevalence, severity, and nature of heuristic violations in the user interface design. The results were subsequently utilized to guide the iterative software design process. A comparison between two consecutive prototypes showed that the second design had only half as many usability violations as the first prototype and had considerable improvement in a number of usability heuristic categories. The validity of heuristic evaluation was examined in an ethnographic study of paramedics using a prototype of the system in their work environment. Users' task performances partially verified heuristic evaluation results. However, they also revealed problems that were not identified in heuristic evaluation but only became prominent during field observation. In conclusion, we argue that usability should be given high priority in the development of a telemedicine system, and that heuristic evaluation can be an effective and efficient way to identify usability problems in the early stage of software development.

Applying usability heuristics to radiotherapy systems.

PubMed

Chan, Alvita J; Islam, Mohammad K; Rosewall, Tara; Jaffray, David A; Easty, Anthony C; Cafazzo, Joseph A

2012-01-01

Heuristic evaluations have been used to evaluate safety of medical devices by identifying and assessing usability issues. Since radiotherapy treatment delivery systems often consist of multiple complex user-interfaces, a heuristic evaluation was conducted to assess the potential safety issues of such a system. A heuristic evaluation was conducted to evaluate the treatment delivery system at Princess Margaret Hospital (Toronto, Canada). Two independent evaluators identified usability issues with the user-interfaces and rated the severity of each issue. The evaluators identified 75 usability issues in total. Eighteen of them were rated as high severity, indicating the potential to have a major impact on patient safety. A majority of issues were found on the record and verify system, and many were associated with the patient setup process. While the hospital has processes in place to ensure patient safety, recommendations were developed to further mitigate the risks of potential consequences. Heuristic evaluation is an efficient and inexpensive method that can be successfully applied to radiotherapy delivery systems to identify usability issues and improve patient safety. Although this study was conducted only at one site, the findings may have broad implications for the design of these systems. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Evaluating a Web-Based Interface for Internet Telemedicine

NASA Technical Reports Server (NTRS)

Lathan, Corinna E.; Newman, Dava J.; Sebrechts, Marc M.; Doarn, Charles R.

1997-01-01

The objective is to introduce the usability engineering methodology, heuristic evaluation, to the design and development of a web-based telemedicine system. Using a set of usability criteria, or heuristics, one evaluator examined the Spacebridge to Russia web-site for usability problems. Thirty-four usability problems were found in this preliminary study and all were assigned a severity rating. The value of heuristic analysis in the iterative design of a system is shown because the problems can be fixed before deployment of a system and the problems are of a different nature than those found by actual users of the system. It was therefore determined that there is potential value of heuristic evaluation paired with user testing as a strategy for optimal system performance design.

Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems.

PubMed

Khajouei, Reza; Zahiri Esfahani, Misagh; Jahani, Yunes

2017-04-01

There are several user-based and expert-based usability evaluation methods that may perform differently according to the context in which they are used. The objective of this study was to compare 2 expert-based methods, heuristic evaluation (HE) and cognitive walkthrough (CW), for evaluating usability of health care information systems. Five evaluators independently evaluated a medical office management system using HE and CW. We compared the 2 methods in terms of the number of identified usability problems, their severity, and the coverage of each method. In total, 156 problems were identified using the 2 methods. HE identified a significantly higher number of problems related to the "satisfaction" attribute ( P  = .002). The number of problems identified using CW concerning the "learnability" attribute was significantly higher than those identified using HE ( P  = .005). There was no significant difference between the number of problems identified by HE, based on different usability attributes ( P  = .232). Results of CW showed a significant difference between the number of problems related to usability attributes ( P  < .0001). The average severity of problems identified using CW was significantly higher than that of HE ( P  < .0001). This study showed that HE and CW do not differ significantly in terms of the number of usability problems identified, but they differ based on the severity of problems and the coverage of some usability attributes. The results suggest that CW would be the preferred method for evaluating systems intended for novice users and HE for users who have experience with similar systems. However, more studies are needed to support this finding. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Evaluating Usability in a Distance Digital Systems Laboratory Class

ERIC Educational Resources Information Center

Kostaras, N.; Xenos, M.; Skodras, A. N.

2011-01-01

This paper presents the usability evaluation of a digital systems laboratory class offered to distance-learning students. It details the way in which students can participate remotely in such a laboratory, the methodology employed in the usability assessment of the laboratory infrastructure (hardware and software), and also outlines the main…

Usability and feasibility of health IT interventions to enhance Self-Care for Lymphedema Symptom Management in breast cancer survivors.

PubMed

Fu, Mei R; Axelrod, Deborah; Guth, Amber A; Wang, Yao; Scagliola, Joan; Hiotis, Karen; Rampertaap, Kavita; El-Shammaa, Nardin

2016-09-01

The-Optimal-Lymph-Flow health IT system (TOLF) is a patient-centered, web-and-mobile-based educational and behavioral health IT system focusing on safe, innovative, and pragmatic self-care strategies for lymphedema symptom management. The purpose of this study was to evaluate usability, feasibility, and acceptability of TOLF among the end-user of breast cancer survivors. Two types of usability testing were completed with 30 breast cancer survivors: heuristic evaluation and end-user testing. Each participant was asked to think aloud while completing a set of specified tasks designed to explicate and freely explore the system features. A heuristic evaluation checklist, the Perceived Ease of Use and Usefulness Questionnaire, and The Post Study System Usability Questionnaire were used to evaluate usability of the system. Open-ended questions were used to gather qualitative data. Quantitative data were analyzed using descriptive statistics and qualitative data were summarized thematically. Breast cancer survivors were very satisfied with the system: 90% (n = 27) rated the system having no usability problems; 10% (n = 3) noted minor cosmetic problems: spelling errors or text font size. The majority of participants 96.6% (n = 29) strongly agreed that the system was easy to use and effective in helping to learn about lymphedema, symptoms and self-care strategies. Themes from the qualitative data included empowerment, high quality information, loving avatar simulation videos, easy accessibility, and user-friendliness. This usability study provided evidence on breast cancer survivor's acceptance and highly positive evaluation of TOLF's usability as well as feasibility of using technologically-driven delivery model to enhance self-care strategies for lymphedema symptom management.

Design and validation of a questionnaire to evaluate the usability of computerized critical care information systems.

PubMed

von Dincklage, Falk; Lichtner, Gregor; Suchodolski, Klaudiusz; Ragaller, Maximilian; Friesdorf, Wolfgang; Podtschaske, Beatrice

2017-08-01

The implementation of computerized critical care information systems (CCIS) can improve the quality of clinical care and staff satisfaction, but also holds risks of disrupting the workflow with consecutive negative impacts. The usability of CCIS is one of the key factors determining their benefits and weaknesses. However, no tailored instrument exists to measure the usability of such systems. Therefore, the aim of this study was to design and validate a questionnaire that measures the usability of CCIS. Following a mixed-method design approach, we developed a questionnaire comprising two evaluation models to assess the usability of CCIS: (1) the task-specific model rates the usability individually for several tasks which CCIS could support and which we derived by analyzing work processes in the ICU; (2) the characteristic-specific model rates the different aspects of the usability, as defined by the international standard "ergonomics of human-system interaction". We tested validity and reliability of the digital version of the questionnaire in a sample population. In the sample population of 535 participants both usability evaluation models showed a strong correlation with the overall rating of the system (multiple correlation coefficients ≥0.80) as well as a very high internal consistency (Cronbach's alpha ≥0.93). The novel questionnaire is a valid and reliable instrument to measure the usability of CCIS and can be used to study the influence of the usability on their implementation benefits and weaknesses.

A Modified User-Oriented Heuristic Evaluation of a Mobile Health System for Diabetes Self-management Support

PubMed Central

Georgsson, Mattias; Staggers, Nancy; Weir, Charlene

2016-01-01

Mobile health platforms offer significant opportunities for improving diabetic self-care, but only if adequate usability exists. Expert evaluations such as heuristic evaluation can provide distinct usability information about systems. The purpose of this study was to complete a usability evaluation of a mobile health system for diabetes patients using a modified heuristic evaluation technique of (1) dual-domain experts (healthcare professionals, usability experts), (2) validated scenarios and user tasks related to patients’ self-care, and (3) in-depth severity factor ratings. Experts identified 129 usability problems with 274 heuristic violations for the system. The categories Consistency and Standards dominated at 24.1% (n = 66), followed by Match Between System and Real World at 22.3% (n = 61). Average severity ratings across system views were 2.8 (of 4), with 9.3% (n = 12) rated as catastrophic and 53.5% (n = 69) as major. The large volume of violations with severe ratings indicated clear priorities for redesign. The modified heuristic approach allowed evaluators to identify unique and important issues, including ones related to self-management and patient safety. This article provides a template for one type of expert evaluation adding to the informaticists’ toolbox when needing to conduct a fast, resource-efficient and user-oriented heuristic evaluation. PMID:26657618

A Modified User-Oriented Heuristic Evaluation of a Mobile Health System for Diabetes Self-management Support.

PubMed

Georgsson, Mattias; Staggers, Nancy; Weir, Charlene

2016-02-01

Mobile health platforms offer significant opportunities for improving diabetic self-care, but only if adequate usability exists. Expert evaluations such as heuristic evaluation can provide distinct usability information about systems. The purpose of this study was to complete a usability evaluation of a mobile health system for diabetes patients using a modified heuristic evaluation technique of (1) dual-domain experts (healthcare professionals, usability experts), (2) validated scenarios and user tasks related to patients' self-care, and (3) in-depth severity factor ratings. Experts identified 129 usability problems with 274 heuristic violations for the system. The categories Consistency and Standards dominated at 24.1% (n = 66), followed by Match Between System and Real World at 22.3% (n = 61). Average severity ratings across system views were 2.8 (of 4), with 9.3% (n = 12) rated as catastrophic and 53.5% (n = 69) as major. The large volume of violations with severe ratings indicated clear priorities for redesign. The modified heuristic approach allowed evaluators to identify unique and important issues, including ones related to self-management and patient safety. This article provides a template for one type of expert evaluation adding to the informaticists' toolbox when needing to conduct a fast, resource-efficient and user-oriented heuristic evaluation.

A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care.

PubMed

Bakhshi-Raiez, F; de Keizer, N F; Cornet, R; Dorrepaal, M; Dongelmans, D; Jaspers, M W M

2012-05-01

To evaluate the usability of a large compositional interface terminology based on SNOMED CT and the terminology application for registration of the reasons for intensive care admission in a Patient Data Management System. Observational study with user-based usability evaluations before and 3 months after the system was implemented and routinely used. Usability was defined by five aspects: effectiveness, efficiency, learnability, overall user satisfaction, and experienced usability problems. Qualitative (the Think-Aloud user testing method) and quantitative (the System Usability Scale questionnaire and Time-on-Task analyses) methods were used to examine these usability aspects. The results of the evaluation study revealed that the usability of the interface terminology fell short (SUS scores before and after implementation of 47.2 out of 100 and 37.5 respectively out of 100). The qualitative measurements revealed a high number (n=35) of distinct usability problems, leading to ineffective and inefficient registration of reasons for admission. The effectiveness and efficiency of the system did not change over time. About 14% (n=5) of the revealed usability problems were related to the terminology content based on SNOMED CT, while the remaining 86% (n=30) was related to the terminology application. The problems related to the terminology content were more severe than the problems related to the terminology application. This study provides a detailed insight into how clinicians interact with a controlled compositional terminology through a terminology application. The extensiveness, complexity of the hierarchy, and the language usage of an interface terminology are defining for its usability. Carefully crafted domain-specific subsets and a well-designed terminology application are needed to facilitate the use of a complex compositional interface terminology based on SNOMED CT. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

A literature review about usability evaluation methods for e-learning platforms.

PubMed

Freire, Luciana Lopes; Arezes, Pedro Miguel; Campos, José Creissac

2012-01-01

The usability analysis of information systems has been the target of several research studies over the past thirty years. These studies have highlighted a great diversity of points of view, including researchers from different scientific areas such as Ergonomics, Computer Science, Design and Education. Within the domain of information ergonomics, the study of tools and methods used for usability evaluation dedicated to E-learning presents evidence that there is a continuous and dynamic evolution of E-learning systems, in many different contexts -academics and corporative. These systems, also known as LMS (Learning Management Systems), can be classified according to their educational goals and their technological features. However, in these systems the usability issues are related with the relationship/interactions between user and system in the user's context. This review is a synthesis of research project about Information Ergonomics and embraces three dimensions, namely the methods, models and frameworks that have been applied to evaluate LMS. The study also includes the main usability criteria and heuristics used. The obtained results show a notorious change in the paradigms of usability, with which it will be possible to discuss about the studies carried out by different researchers that were focused on usability ergonomic principles aimed at E-learning.

Usability in the real world: assessing medical information technologies in patients' homes.

PubMed

Kaufman, David R; Patel, Vimla L; Hilliman, Charlyn; Morin, Philip C; Pevzner, Jenia; Weinstock, Ruth S; Goland, Robin; Shea, Steven; Starren, Justin

2003-01-01

This paper presents an approach to usability evaluation of computer-based health care systems designed for patient use in their homes. Although such devices are becoming more prevalent, there is very little known about their usability. The theoretical foundations for the methods are discussed. The approach incorporates a cognitive walkthrough usability evaluation and new methods for usability testing that can be conducted in patient's homes. The method was applied to the IDEATel intervention, a multi-institution randomized controlled trial of the feasibility, acceptability, and clinical utility of a home-based telemedicine system for diabetic Medicare population. The usability study was designed to assess barriers to optimal use of the system. The focus was both on dimensions of the interface and on dimensions of patient skills and competency. The usability field research involved testing 25 patients in their homes using the system. The analysis included a range of video-analytic methods of varying levels of granularity. The usability evaluation revealed aspects of the interface that were sub-optimal and impeded the performance of certain tasks. It also found a range of patient-related factors such as numeracy and psychomotor skills that constituted barriers to productive use. A multifaceted usability approach provided important insight regarding use of technology by an elderly chronic-care patient population and more generally, for understanding how home health initiatives can more effectively use such technology.

Initial Usability and Feasibility Evaluation of a Personal Health Record-Based Self-Management System for Older Adults.

PubMed

Sheehan, Barbara; Lucero, Robert J

2015-01-01

Electronic personal health record-based (ePHR-based) self-management systems can improve patient engagement and have an impact on health outcomes. In order to realize the benefits of these systems, there is a need to develop and evaluate heath information technology from the same theoretical underpinnings. Using an innovative usability approach based in human-centered distributed information design (HCDID), we tested an ePHR-based falls-prevention self-management system-Self-Assessment via a Personal Health Record (i.e., SAPHeR)-designed using HCDID principles in a laboratory. And we later evaluated SAPHeR's use by community-dwelling older adults at home. The innovative approach used in this study supported the analysis of four components: tasks, users, representations, and functions. Tasks were easily learned and features such as text-associated images facilitated task completion. Task performance times were slow, however user satisfaction was high. Nearly seven out of every ten features desired by design participants were evaluated in our usability testing of the SAPHeR system. The in vivo evaluation suggests that older adults could improve their confidence in performing indoor and outdoor activities after using the SAPHeR system. We have applied an innovative consumer-usability evaluation. Our approach addresses the limitations of other usability testing methods that do not utilize consistent theoretically based methods for designing and testing technology. We have successfully demonstrated the utility of testing consumer technology use across multiple components (i.e., task, user, representational, functional) to evaluate the usefulness, usability, and satisfaction of an ePHR-based self-management system.

Usability in telemedicine systems-A literature survey.

PubMed

Klaassen, B; van Beijnum, B J F; Hermens, H J

2016-09-01

The rapid development of sensors and communication technologies enable the growth of new innovative services in healthcare, such as Telemedicine. An essential ingredient in the development of a telemedicine system and its final acceptance by end users are usability studies. The principles of usability engineering, evaluations and telemedicine are well established, and it may contribute to the adoption and eventually deployment of such systems and services. An in-depth usability analysis, including performance and attitude measures, requires knowledge about available usability techniques, and is depending on the amount of resources. Therefore it is worth investigating how usability methods are applied in developing telemedicine systems. Our hypothesis is: with increasing research and development of telemedicine systems, we expect that various usability methods are more equally employed for different end-user groups and applications. A literature survey was conducted to find telemedicine systems that have been evaluated for usability or ease of use. The elements of the PICO framework were used as a basis for the selection criteria in the literature search. The search was not limited by year. Two independent reviewers screened all search results first by title, and then by abstract for inclusion. Articles were included up to May 2015. In total, 127 publications were included in this survey. The number of publications on telemedicine systems significantly increased after 2008. Older adults and end-users with cardiovascular conditions were among largest target end-user groups. Remote monitoring systems were found the most, in 90 publications. Questionnaires are the most common means for evaluating telemedicine systems, and were found in 88 publications. Questionnaires are used frequently in studies focusing on cardiovascular diseases, Parkinson's disease and older adult conditions. Interviews are found the most in publications related to stroke. In total 71% of the publications were trial-orientated and the remaining process orientated. An increase in telemedicine research, development and applications is found worldwide, with the majority of publications conducted in America. Monitoring patients in their homes can lead to better healthcare at lower costs which implies an increased demand of new healthcare strategies like telemedicine. We expected that with the increase in telemedicine research and development, a greater range of usability methods would also be employed in the included publications. This is not the case. Researchers employed questionnaires as a preferred usability method for each type of telemedicine system and most end-users. However, in process-orientated studies a greater range of usability evaluations were applied, with fewer differences found in the amount of publications for each evaluation method. Questionnaires enable researchers to evaluate a system quickly on end users, as it requires less expertise on the evaluation method compared to the other methods. They are easily distributed and are customizable. The use of questionnaires is therefore an evaluation method of choice for a variety of telemedicine systems and end-users. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Developing a Self-Report-Based Sequential Analysis Method for Educational Technology Systems: A Process-Based Usability Evaluation

ERIC Educational Resources Information Center

Lin, Yi-Chun; Hsieh, Ya-Hui; Hou, Huei-Tse

2015-01-01

The development of a usability evaluation method for educational systems or applications, called the self-report-based sequential analysis, is described herein. The method aims to extend the current practice by proposing self-report-based sequential analysis as a new usability method, which integrates the advantages of self-report in survey…

Evaluating Usability of Radiology Information Systems in Hospitals of Tabriz University of Medical Sciences.

PubMed

Rezaei-Hachesu, Peyman; Pesianian, Esmaeil; Mohammadian, Mohsen

2016-02-01

Radiology information system (RIS) in order to reduce workload and improve the quality of services must be well-designed. Heuristic evaluation is one of the methods that understand usability problems with the least time, cost and resources. The aim of present study is to evaluate the usability of RISs in hospitals. This is a cross-sectional descriptive study (2015) that uses heuristic evaluation method to evaluate the usability of RIS used in 3 hospitals of Tabriz city. The data are collected using a standard checklist based on 13 principles of Nielsen Heuristic evaluation method. Usability of RISs was investigated based on the number of components observed from Nielsen principles and problems of usability based on the number of non-observed components as well as non-existent or unrecognizable components. by evaluation of RISs in each of the hospitals 1, 2 and 3, total numbers of observed components were obtained as 173, 202 and 196, respectively. It was concluded that the usability of RISs in the studied population, on average and with observing 190 components of the 291 components related to the 13 principles of Nielsen is 65.41 %. Furthermore, problems of usability were obtained as 26.35%. The established and visible nature of some components such as response time of application, visual feedbacks, colors, view and design and arrangement of software objects cause more attention to these components as principal components in designing UI software. Also, incorrect analysis before system design leads to a lack of attention to secondary needs like Help software and security issues.

Qualitative review of usability problems in health information systems for radiology.

PubMed

Dias, Camila Rodrigues; Pereira, Marluce Rodrigues; Freire, André Pimenta

2017-12-01

Radiology processes are commonly supported by Radiology Information System (RIS), Picture Archiving and Communication System (PACS) and other software for radiology. However, these information technologies can present usability problems that affect the performance of radiologists and physicians, especially considering the complexity of the tasks involved. The purpose of this study was to extract, classify and analyze qualitatively the usability problems in PACS, RIS and other software for radiology. A systematic review was performed to extract usability problems reported in empirical usability studies in the literature. The usability problems were categorized as violations of Nielsen and Molich's usability heuristics. The qualitative analysis indicated the causes and the effects of the identified usability problems. From the 431 papers initially identified, 10 met the study criteria. The analysis of the papers identified 90 instances of usability problems, classified into categories corresponding to established usability heuristics. The five heuristics with the highest number of instances of usability problems were "Flexibility and efficiency of use", "Consistency and standards", "Match between system and the real world", "Recognition rather than recall" and "Help and documentation", respectively. These problems can make the interaction time consuming, causing delays in tasks, dissatisfaction, frustration, preventing users from enjoying all the benefits and functionalities of the system, as well as leading to more errors and difficulties in carrying out clinical analyses. Furthermore, the present paper showed a lack of studies performed on systems for radiology, especially usability evaluations using formal methods of evaluation involving the final users. Copyright © 2017 Elsevier Inc. All rights reserved.

Review of health information technology usability study methodologies

PubMed Central

Bakken, Suzanne

2011-01-01

Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user–task interaction, n=42; stage 2, system–task interaction, n=310; stage 3, user–task–system interaction, n=69; stage 4, user–task–system–environment interaction, n=54; and stage 5, user–task–system–environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation. PMID:21828224

Evidence-based Heuristics for Evaluating Demands on eHealth Literacy and Usability in a Mobile Consumer Health Application.

PubMed

Monkman, Helen; Griffith, Janessa; Kushniruk, Andre W

2015-01-01

Heuristic evaluations have proven to be valuable for identifying usability issues in systems. Commonly used sets of heuritics exist; however, they may not always be the most suitable, given the specific goal of the analysis. One such example is seeking to evaluate the demands on eHealth literacy and usability of consumer health information systems. In this study, eight essential heuristics and three optional heuristics subsumed from the evidence on eHealth/health literacy and usability were tested for their utility in assessing a mobile blood pressure tracking application (app). This evaluation revealed a variety of ways the design of the app could both benefit and impede users with limited eHealth literacy. This study demonstrated the utility of a low-cost, single evaluation approach for identifying both eHealth literacy and usability issues based on existing evidence in the literature.

A health literacy and usability heuristic evaluation of a mobile consumer health application.

PubMed

Monkman, Helen; Kushniruk, Andre

2013-01-01

Usability and health literacy are two critical factors in the design and evaluation of consumer health information systems. However, methods for evaluating these two factors in conjunction remain limited. This study adapted a set of existing guidelines for the design of consumer health Web sites into evidence-based evaluation heuristics tailored specifically for mobile consumer health applications. In order to test the approach, a mobile consumer health application (app) was then evaluated using these heuristics. In addition to revealing ways to improve the usability of the system, this analysis identified opportunities to augment the content to make it more understandable by users with limited health literacy. This study successfully demonstrated the utility of converting existing design guidelines into heuristics for the evaluation of usability and health literacy. The heuristics generated could be applied for assessing and revising other existing consumer health information systems.

'Televaluation' of clinical information systems: an integrative approach to assessing Web-based systems.

PubMed

Kushniruk, A W; Patel, C; Patel, V L; Cimino, J J

2001-04-01

The World Wide Web provides an unprecedented opportunity for widespread access to health-care applications by both patients and providers. The development of new methods for assessing the effectiveness and usability of these systems is becoming a critical issue. This paper describes the distance evaluation (i.e. 'televaluation') of emerging Web-based information technologies. In health informatics evaluation, there is a need for application of new ideas and methods from the fields of cognitive science and usability engineering. A framework is presented for conducting evaluations of health-care information technologies that integrates a number of methods, ranging from deployment of on-line questionnaires (and Web-based forms) to remote video-based usability testing of user interactions with clinical information systems. Examples illustrating application of these techniques are presented for the assessment of a patient clinical information system (PatCIS), as well as an evaluation of use of Web-based clinical guidelines. Issues in designing, prototyping and iteratively refining evaluation components are discussed, along with description of a 'virtual' usability laboratory.

Pre-post evaluation of physicians' satisfaction with a redesigned electronic medical record system.

PubMed

Jaspers, Monique W M; Peute, Linda W P; Lauteslager, Arnaud; Bakker, Piet J M

2008-01-01

Physicians' acceptance of Electronic Medical Record Systems (EMRs) is closely related to their usability. Knowledge about end-users' opinions on usability of an EMR system may contribute to planning for the next phase of the usability cycle of the system. A demand for integration of new functionalities, such as computerized order entry and an electronic patient status led to redesign of our EMR system, which had been in use for over 8 years at the Academic Medical Center of Amsterdam. The aim of this study was to understand whether the redesigned EMR system was an improvement of the earlier EMR and which system aspects accounted for user satisfaction and which did not. We conducted a formative pre- and post usability evaluation of our former and redesigned EMR system. For the assessment of both system versions' usability, we distributed two standardized usability questionnaires among 150 clinicians who routinely had used the older EMR system and had been working with its newer version for 6 weeks. Though overall user satisfaction was relatively high for both EMR systems, screen layout and interaction structure proved less easy to work with in the newer EMR system. The new EMR system however was more appreciated because of its enhanced functionality, capabilities and likeable user-interface. The results point to a number of actions that might be useful in future usability improvement efforts of our EMR system and other EMRs.

Usability evaluation of an emergency department information system prototype designed using cognitive systems engineering techniques.

PubMed

Clark, Lindsey N; Benda, Natalie C; Hegde, Sudeep; McGeorge, Nicolette M; Guarrera-Schick, Theresa K; Hettinger, A Zachary; LaVergne, David T; Perry, Shawna J; Wears, Robert L; Fairbanks, Rollin J; Bisantz, Ann M

2017-04-01

This article presents an evaluation of novel display concepts for an emergency department information system (EDIS) designed using cognitive systems engineering methods. EDISs assist emergency medicine staff with tracking patient care and ED resource allocation. Participants performed patient planning and orientation tasks using the EDIS displays and rated the display's ability to support various cognitive performance objectives along with the usability, usefulness, and predicted frequency of use for 18 system components. Mean ratings were positive for cognitive performance support objectives, usability, usefulness, and frequency of use, demonstrating the successful application of design methods to create useful and usable EDIS concepts that provide cognitive support for emergency medicine staff. Nurse and provider roles had significantly different perceptions of the usability and usefulness of certain EDIS components, suggesting that they have different information needs while working. Copyright © 2016 Elsevier Ltd. All rights reserved.

Usability evaluation of an emergency department information system prototype designed using cognitive systems engineering techniques

PubMed Central

Clark, Lindsey N.; Benda, Natalie C.; Hegde, Sudeep; McGeorge, Nicolette M.; Guarrera-Schick, Theresa K.; Hettinger, A. Zachary; LaVergne, David T.; Perry, Shawna J.; Wears, Robert L.; Fairbanks, Rollin J.; Bisantz, Ann M.

2017-01-01

This article presents an evaluation of novel display concepts for an emergency department information system (EDIS) designed using cognitive systems engineering methods. EDISs assist emergency medicine staff with tracking patient care and ED resource allocation. Participants performed patient planning and orientation tasks using the EDIS displays and rated the display’s ability to support various cognitive performance objectives along with the usability, usefulness, and predicted frequency of use for 18 system components. Mean ratings were positive for cognitive performance support objectives, usability, usefulness, and frequency of use, demonstrating the successful application of design methods to create useful and usable EDIS concepts that provide cognitive support for emergency medicine staff. Nurse and provider roles had significantly different perceptions of the usability and usefulness of certain EDIS components, suggesting that they have different information needs while working. PMID:28166896

Optimizing the user interface of a data entry module for an electronic patient record for cardiac rehabilitation: A mixed method usability approach.

PubMed

van Engen-Verheul, Mariëtte M; Peute, Linda W P; de Keizer, Nicolette F; Peek, Niels; Jaspers, Monique W M

2016-03-01

Cumbersome electronic patient record (EPR) interfaces may complicate data-entry in clinical practice. Completeness of data entered in the EPR determines, among other things, the value of computerized clinical decision support (CCDS). Quantitative usability evaluations can provide insight into mismatches between the system design model of data entry and users' data entry behavior, but not into the underlying causes for these mismatches. Mixed method usability evaluation studies may provide these insights, and thus support generating redesign recommendations for improving an EPR system's data entry interface. To improve the usability of the data entry interface of an EPR system with CCDS in the field of cardiac rehabilitation (CR), and additionally, to assess the value of a mixed method usability approach in this context. Seven CR professionals performed a think-aloud usability evaluation both before (beta-version) and after the redesign of the system. Observed usability problems from both evaluations were analyzed and categorized using Zhang et al.'s heuristic principles of good interface design. We combined the think-aloud usability evaluation of the system's beta-version with the measurement of a new usability construct: users' deviations in action sequence from the system's predefined data entry order sequence. Recommendations for redesign were implemented. We assessed whether the redesign improved CR professionals' (1) task efficacy (with respect to the completeness of data they collected), and (2) task efficiency (with respect to the average number of mouse clicks they needed to complete data entry subtasks). With the system's beta version, 40% of health care professionals' navigation actions through the system deviated from the predefined next system action. The causes for these deviations as revealed by the think-aloud method mostly concerned mismatches between the system design model for data entry action sequences and users expectations of these action sequences, based on their paper-based daily routines. This caused non completion of data entry tasks (31% of main tasks completed), and more navigation actions than minimally required (146% of the minimum required). In the redesigned system the data entry navigational structure was organized in a flexible way around an overview screen to better mimic users' paper-based daily routines of collecting patient data. This redesign resulted in an increased number of completed main tasks (70%) and a decrease in navigation actions (133% of the minimum required). The think-aloud usability evaluation of the redesigned system showed that remaining problems concerned flexibility (e.g., lack of customization options) and consistency (mainly with layout and position of items on the screen). The mixed method usability evaluation was supportive in revealing the magnitude and causes of mismatches between the system design model of data-entry with users' data entry behavior. However, as both task efficacy and efficiency were still not optimal with the redesigned EPR, we advise to perform a cognitive analysis on end users' mental processes and behavior patterns in daily work processes specifically during the requirements analysis phase of development of interactive healthcare information systems. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Assessing the Potential Use of Eye-Tracking Triangulation for Evaluating the Usability of an Online Diabetes Exercise System.

PubMed

Schaarup, Clara; Hartvigsen, Gunnar; Larsen, Lars Bo; Tan, Zheng-Hua; Årsand, Eirik; Hejlesen, Ole Kristian

2015-01-01

The Online Diabetes Exercise System was developed to motivate people with Type 2 diabetes to do a 25 minutes low-volume high-intensity interval training program. In a previous multi-method evaluation of the system, several usability issues were identified and corrected. Despite the thorough testing, it was unclear whether all usability problems had been identified using the multi-method evaluation. Our hypothesis was that adding the eye-tracking triangulation to the multi-method evaluation would increase the accuracy and completeness when testing the usability of the system. The study design was an Eye-tracking Triangulation; conventional eye-tracking with predefined tasks followed by The Post-Experience Eye-Tracked Protocol (PEEP). Six Areas of Interests were the basis for the PEEP-session. The eye-tracking triangulation gave objective and subjective results, which are believed to be highly relevant for designing, implementing, evaluating and optimizing systems in the field of health informatics. Future work should include testing the method on a larger and more representative group of users and apply the method on different system types.

Human Factors in the Large: Experiences from Denmark, Finland and Canada in Moving Towards Regional and National Evaluations of Health Information System Usability. Contribution of the IMIA Human Factors Working Group.

PubMed

Kushniruk, A; Kaipio, J; Nieminen, M; Hyppönen, H; Lääveri, T; Nohr, C; Kanstrup, A M; Berg Christiansen, M; Kuo, M-H; Borycki, E

2014-08-15

The objective of this paper is to explore approaches to understanding the usability of health information systems at regional and national levels. Several different methods are discussed in case studies from Denmark, Finland and Canada. They range from small scale qualitative studies involving usability testing of systems to larger scale national level questionnaire studies aimed at assessing the use and usability of health information systems by entire groups of health professionals. It was found that regional and national usability studies can complement smaller scale usability studies, and that they are needed in order to understand larger trends regarding system usability. Despite adoption of EHRs, many health professionals rate the usability of the systems as low. A range of usability issues have been noted when data is collected on a large scale through use of widely distributed questionnaires and websites designed to monitor user perceptions of usability. As health information systems are deployed on a widespread basis, studies that examine systems used regionally or nationally are required. In addition, collection of large scale data on the usability of specific IT products is needed in order to complement smaller scale studies of specific systems.

Human Factors in the Large: Experiences from Denmark, Finland and Canada in Moving Towards Regional and National Evaluations of Health Information System Usability

PubMed Central

Kaipio, J.; Nieminen, M.; Hyppönen, H.; Lääveri, T.; Nohr, C.; Kanstrup, A. M.; Berg Christiansen, M.; Kuo, M.-H.; Borycki, E.

2014-01-01

Summary Objectives The objective of this paper is to explore approaches to understanding the usability of health information systems at regional and national levels. Methods Several different methods are discussed in case studies from Denmark, Finland and Canada. They range from small scale qualitative studies involving usability testing of systems to larger scale national level questionnaire studies aimed at assessing the use and usability of health information systems by entire groups of health professionals. Results It was found that regional and national usability studies can complement smaller scale usability studies, and that they are needed in order to understand larger trends regarding system usability. Despite adoption of EHRs, many health professionals rate the usability of the systems as low. A range of usability issues have been noted when data is collected on a large scale through use of widely distributed questionnaires and websites designed to monitor user perceptions of usability. Conclusion As health information systems are deployed on a widespread basis, studies that examine systems used regionally or nationally are required. In addition, collection of large scale data on the usability of specific IT products is needed in order to complement smaller scale studies of specific systems. PMID:25123725

Usability Evaluation of a Clinical Decision Support System for Geriatric ED Pain Treatment.

PubMed

Genes, Nicholas; Kim, Min Soon; Thum, Frederick L; Rivera, Laura; Beato, Rosemary; Song, Carolyn; Soriano, Jared; Kannry, Joseph; Baumlin, Kevin; Hwang, Ula

2016-01-01

Older adults are at risk for inadequate emergency department (ED) pain care. Unrelieved acute pain is associated with poor outcomes. Clinical decision support systems (CDSS) hold promise to improve patient care, but CDSS quality varies widely, particularly when usability evaluation is not employed. To conduct an iterative usability and redesign process of a novel geriatric abdominal pain care CDSS. We hypothesized this process would result in the creation of more usable and favorable pain care interventions. Thirteen emergency physicians familiar with the Electronic Health Record (EHR) in use at the study site were recruited. Over a 10-week period, 17 1-hour usability test sessions were conducted across 3 rounds of testing. Participants were given 3 patient scenarios and provided simulated clinical care using the EHR, while interacting with the CDSS interventions. Quantitative System Usability Scores (SUS), favorability scores and qualitative narrative feedback were collected for each session. Using a multi-step review process by an interdisciplinary team, positive and negative usability issues in effectiveness, efficiency, and satisfaction were considered, prioritized and incorporated in the iterative redesign process of the CDSS. Video analysis was used to determine the appropriateness of the CDS appearances during simulated clinical care. Over the 3 rounds of usability evaluations and subsequent redesign processes, mean SUS progressively improved from 74.8 to 81.2 to 88.9; mean favorability scores improved from 3.23 to 4.29 (1 worst, 5 best). Video analysis revealed that, in the course of the iterative redesign processes, rates of physicians' acknowledgment of CDS interventions increased, however most rates of desired actions by physicians (such as more frequent pain score updates) decreased. The iterative usability redesign process was instrumental in improving the usability of the CDSS; if implemented in practice, it could improve geriatric pain care. The usability evaluation process led to improved acknowledgement and favorability. Incorporating usability testing when designing CDSS interventions for studies may be effective to enhance clinician use.

An Iterative, Mixed Usability Approach Applied to the Telekit System from the Danish TeleCare North Trial

PubMed Central

Schaarup, Clara; Hejlesen, Ole Kristian

2016-01-01

Objective. The aim of the present study is to evaluate the usability of the telehealth system, coined Telekit, by using an iterative, mixed usability approach. Materials and Methods. Ten double experts participated in two heuristic evaluations (HE1, HE2), and 11 COPD patients attended two think-aloud tests. The double experts identified usability violations and classified them into Jakob Nielsen's heuristics. These violations were then translated into measurable values on a scale of 0 to 4 indicating degree of severity. In the think-aloud tests, COPD participants were invited to verbalise their thoughts. Results. The double experts identified 86 usability violations in HE1 and 101 usability violations in HE2. The majority of the violations were rated in the 0–2 range. The findings from the think-aloud tests resulted in 12 themes and associated examples regarding the usability of the Telekit system. The use of the iterative, mixed usability approach produced both quantitative and qualitative results. Conclusion. The iterative, mixed usability approach yields a strong result owing to the high number of problems identified in the tests because the double experts and the COPD participants focus on different aspects of Telekit's usability. This trial is registered with Clinicaltrials.gov, NCT01984840, November 14, 2013. PMID:27974888

An Iterative, Mixed Usability Approach Applied to the Telekit System from the Danish TeleCare North Trial.

PubMed

Lilholt, Pernille Heyckendorff; Schaarup, Clara; Hejlesen, Ole Kristian

2016-01-01

Objective . The aim of the present study is to evaluate the usability of the telehealth system, coined Telekit, by using an iterative, mixed usability approach. Materials and Methods . Ten double experts participated in two heuristic evaluations (HE1, HE2), and 11 COPD patients attended two think-aloud tests. The double experts identified usability violations and classified them into Jakob Nielsen's heuristics. These violations were then translated into measurable values on a scale of 0 to 4 indicating degree of severity. In the think-aloud tests, COPD participants were invited to verbalise their thoughts. Results . The double experts identified 86 usability violations in HE1 and 101 usability violations in HE2. The majority of the violations were rated in the 0-2 range. The findings from the think-aloud tests resulted in 12 themes and associated examples regarding the usability of the Telekit system. The use of the iterative, mixed usability approach produced both quantitative and qualitative results. Conclusion . The iterative, mixed usability approach yields a strong result owing to the high number of problems identified in the tests because the double experts and the COPD participants focus on different aspects of Telekit's usability. This trial is registered with Clinicaltrials.gov, NCT01984840, November 14, 2013.

VRUSE--a computerised diagnostic tool: for usability evaluation of virtual/synthetic environment systems.

PubMed

Kalawsky, R S

1999-02-01

A special questionnaire (VRUSE) has been designed to measure the usability of a VR system according to the attitude and perception of its users. Important aspects of VR systems were carefully derived to produce key usability factors for the questionnaire. Unlike questionnaires designed for generic interfaces VRUSE is specifically designed to cater for evaluating virtual environments, being a diagnostic tool providing a wealth of information about a user's viewpoint of the interface. VRUSE can be used to great effect with other evaluation techniques to pinpoint problematical areas of a VR interface. Other applications include bench-marking of competitor VR systems.

Evaluation of Usability Utilizing Markov Models

ERIC Educational Resources Information Center

Penedo, Janaina Rodrigues; Diniz, Morganna; Ferreira, Simone Bacellar Leal; Silveira, Denis S.; Capra, Eliane

2012-01-01

Purpose: The purpose of this paper is to analyze the usability of a remote learning system in its initial development phase, using a quantitative usability evaluation method through Markov models. Design/methodology/approach: The paper opted for an exploratory study. The data of interest of the research correspond to the possible accesses of users…

Development and user evaluation of a virtual rehabilitation system for wobble board balance training.

PubMed

Fitzgerald, Diarmaid; Trakarnratanakul, Nanthana; Dunne, Lucy; Smyth, Barry; Caulfield, Brian

2008-01-01

We have developed a prototype virtual reality-based balance training system using a single inertial orientation sensor attached to the upper surface of a wobble board. This input device has been interfaced with Neverball, an open source computer game to create the balance training platform. Users can exercise with the system by standing on the wobble board and tilting it in different directions to control an on-screen environment. We have also developed a customized instruction manual to use when setting up the system. To evaluate the usability our prototype system we undertook a user evaluation study with twelve healthy novice participants. Participants were required to assemble the system using an instruction manual and then perform balance exercises with the system. Following this period of exercise VRUSE, a usability evaluation questionnaire, was completed by participants. Results indicated a high level of usability in all categories evaluated.

Safety and usability evaluation of a web-based insulin self-titration system for patients with type 2 diabetes mellitus.

PubMed

Simon, Airin C R; Holleman, Frits; Gude, Wouter T; Hoekstra, Joost B L; Peute, Linda W; Jaspers, Monique W M; Peek, Niels

2013-09-01

The rising incidence of type 2 diabetes mellitus (T2DM) induces severe challenges for the health care system. Our research group developed a web-based system named PANDIT that provides T2DM patients with insulin dosing advice using state of the art clinical decision support technology. The PANDIT interface resembles a glucose diary and provides advice through pop-up messages. Diabetes nurses (DNs) also have access to the system, allowing them to intervene when needed. The objective of this study was to establish whether T2DM patients can safely use PANDIT at home. To this end, we assessed whether patients experience usability problems with a high risk of compromising patient safety when interacting with the system, and whether PANDIT's insulin dosing advice are clinically safe. The study population consisted of patients with T2DM (aged 18-80) who used a once daily basal insulin as well as DNs from a university hospital. The usability evaluation consisted of think-aloud sessions with four patients and three DNs. Video data, audio data and verbal utterances were analyzed for usability problems encountered during PANDIT interactions. Usability problems were rated by a physician and a usability expert according to their potential impact on patient safety. The usability evaluation was followed by an implementation with a duration of four weeks. This implementation took place at the patients' homes with ten patients to evaluate clinical safety of PANDIT advice. PANDIT advice were systematically compared with DN advice. Deviating advice were evaluated with respect to patient safety by a panel of experienced physicians, which specialized in diabetes care. We detected seventeen unique usability problems, none of which was judged to have a high risk of compromising patient safety. Most usability problems concerned the lay-out of the diary, which did not clearly indicate which data entry fields had to be entered in order to obtain an advice. 27 out of 74 (36.5%) PANDIT advice differed from those provided by DNs. However, only one of these (1.4%) was considered unsafe by the panel. T2DM patients with no prior experience with the web-based self-management system were capable of consulting the system without encountering significant usability problems. Furthermore, the large majority of PANDIT advice were considered clinically safe according to the expert panel. One advice was considered unsafe. This could however easily be corrected by implementing a small modification to the system's knowledge base. Copyright © 2013 Elsevier B.V. All rights reserved.

From Usability Engineering to Evidence-based Usability in Health IT.

PubMed

Marcilly, Romaric; Peute, Linda; Beuscart-Zephir, Marie-Catherine

2016-01-01

Usability is a critical factor in the acceptance, safe use, and success of health IT. The User-Centred Design process is widely promoted to improve usability. However, this traditional case by case approach that is rooted in the sound understanding of users' needs is not sufficient to improve technologies' usability and prevent usability-induced use-errors that may harm patients. It should be enriched with empirical evidence. This evidence is on design elements (what are the most valuable design principles, and the worst usability mistakes), and on the usability evaluation methods (which combination of methods is most suitable in which context). To achieve this evidence, several steps must be fulfilled and challenges must be overcome. Some attempts to search evidence for designing elements of health IT and for usability evaluation methods exist and are summarized. A concrete instance of evidence-based usability design principles for medication-related alerting systems is briefly described.

Usability Prediction & Ranking of SDLC Models Using Fuzzy Hierarchical Usability Model

NASA Astrophysics Data System (ADS)

Gupta, Deepak; Ahlawat, Anil K.; Sagar, Kalpna

2017-06-01

Evaluation of software quality is an important aspect for controlling and managing the software. By such evaluation, improvements in software process can be made. The software quality is significantly dependent on software usability. Many researchers have proposed numbers of usability models. Each model considers a set of usability factors but do not cover all the usability aspects. Practical implementation of these models is still missing, as there is a lack of precise definition of usability. Also, it is very difficult to integrate these models into current software engineering practices. In order to overcome these challenges, this paper aims to define the term `usability' using the proposed hierarchical usability model with its detailed taxonomy. The taxonomy considers generic evaluation criteria for identifying the quality components, which brings together factors, attributes and characteristics defined in various HCI and software models. For the first time, the usability model is also implemented to predict more accurate usability values. The proposed system is named as fuzzy hierarchical usability model that can be easily integrated into the current software engineering practices. In order to validate the work, a dataset of six software development life cycle models is created and employed. These models are ranked according to their predicted usability values. This research also focuses on the detailed comparison of proposed model with the existing usability models.

Evaluation of a BCMA’s Electronic Medication Administration Record

PubMed Central

Staggers, Nancy; Iribarren, Sarah; Guo, Jia-Wen; Weir, Charlene

2015-01-01

Barcode medication administration (BCMA) systems can reduce medication errors, but sociotechnical issues are quite common. Although crucial to nurses’ work, few usability evaluations are available for electronic medication administration record screens (eMARs). The purpose of this research was to identify current usability problems in the VA’s eMAR/BCMA system and explore how these might impact nurses’ situation awareness. Three expert evaluators used 10 tasks/elements, heuristic evaluation techniques and explored potential impacts using a situation awareness perspective. The results yielded 99 usability problems categorized into 440 heuristic violations with the largest volume in the category of Match with the Real World. Fifteen usability issues were rated as catastrophic with the Administer/Chart medications task having the most. Situational awareness was impacted at all levels, especially at Level 2, Comprehension. Usability problems point to important areas for improvement because these issues have the potential to impact nurses’ situation awareness, “at a glance” information, nurse productivity and patient safety. PMID:25601936

Usability Evaluation of the Student Centered e-Learning Environment

ERIC Educational Resources Information Center

Junus, Inas Sofiyah; Santoso, Harry Budi; Isal, R. Yugo K.; Utomo, Andika Yudha

2015-01-01

Student Centered e-Learning Environment (SCeLE) has substantial roles to support learning activities at Faculty of Computer Science, Universitas Indonesia (Fasilkom UI). Although it has been utilized for about 10 years, the usability aspect of SCeLE as an e-Learning system has not been evaluated. Therefore, the usability aspects of SCeLE Fasilkom…

A Usability Study of Users' Perceptions toward a Multimedia Computer-Assisted Learning Tool for Neuroanatomy

ERIC Educational Resources Information Center

Gould, Douglas J.; Terrell, Mark A.; Fleming, Jo

2008-01-01

This usability study evaluated users' perceptions of a multimedia prototype for a new e-learning tool: Anatomy of the Central Nervous System: A Multimedia Course. Usability testing is a collection of formative evaluation methods that inform the developmental design of e-learning tools to maximize user acceptance, satisfaction, and adoption.…

A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives

PubMed Central

Lee, Yushin; Yun, Myung Hwan

2017-01-01

A new Brain-Computer Interface (BCI) technique, which is called a hybrid BCI, has recently been proposed to address the limitations of conventional single BCI system. Although some hybrid BCI studies have shown promising results, the field of hybrid BCI is still in its infancy and there is much to be done. Especially, since the hybrid BCI systems are so complicated and complex, it is difficult to understand the constituent and role of a hybrid BCI system at a glance. Also, the complicated and complex systems make it difficult to evaluate the usability of the systems. We systematically reviewed and analyzed the current state-of-the-art hybrid BCI studies, and proposed a systematic taxonomy for classifying the types of hybrid BCIs with multiple taxonomic criteria. After reviewing 74 journal articles, hybrid BCIs could be categorized with respect to 1) the source of brain signals, 2) the characteristics of the brain signal, and 3) the characteristics of operation in each system. In addition, we exhaustively reviewed recent literature on usability of BCIs. To identify the key evaluation dimensions of usability, we focused on task and measurement characteristics of BCI usability. We classified and summarized 31 BCI usability journal articles according to task characteristics (type and description of task) and measurement characteristics (subjective and objective measures). Afterwards, we proposed usability dimensions for BCI and hybrid BCI systems according to three core-constructs: Satisfaction, effectiveness, and efficiency with recommendations for further research. This paper can help BCI researchers, even those who are new to the field, can easily understand the complex structure of the hybrid systems at a glance. Recommendations for future research can also be helpful in establishing research directions and gaining insight in how to solve ergonomics and HCI design issues surrounding BCI and hybrid BCI systems by usability evaluation. PMID:28453547

A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives.

PubMed

Choi, Inchul; Rhiu, Ilsun; Lee, Yushin; Yun, Myung Hwan; Nam, Chang S

2017-01-01

A new Brain-Computer Interface (BCI) technique, which is called a hybrid BCI, has recently been proposed to address the limitations of conventional single BCI system. Although some hybrid BCI studies have shown promising results, the field of hybrid BCI is still in its infancy and there is much to be done. Especially, since the hybrid BCI systems are so complicated and complex, it is difficult to understand the constituent and role of a hybrid BCI system at a glance. Also, the complicated and complex systems make it difficult to evaluate the usability of the systems. We systematically reviewed and analyzed the current state-of-the-art hybrid BCI studies, and proposed a systematic taxonomy for classifying the types of hybrid BCIs with multiple taxonomic criteria. After reviewing 74 journal articles, hybrid BCIs could be categorized with respect to 1) the source of brain signals, 2) the characteristics of the brain signal, and 3) the characteristics of operation in each system. In addition, we exhaustively reviewed recent literature on usability of BCIs. To identify the key evaluation dimensions of usability, we focused on task and measurement characteristics of BCI usability. We classified and summarized 31 BCI usability journal articles according to task characteristics (type and description of task) and measurement characteristics (subjective and objective measures). Afterwards, we proposed usability dimensions for BCI and hybrid BCI systems according to three core-constructs: Satisfaction, effectiveness, and efficiency with recommendations for further research. This paper can help BCI researchers, even those who are new to the field, can easily understand the complex structure of the hybrid systems at a glance. Recommendations for future research can also be helpful in establishing research directions and gaining insight in how to solve ergonomics and HCI design issues surrounding BCI and hybrid BCI systems by usability evaluation.

Task-specific usability requirements of electronic medical records systems: Lessons learned from a national survey of end-users.

PubMed

Farzandipour, Mehrdad; Meidani, Zahra; Riazi, Hossein; Sadeqi Jabali, Monireh

2018-09-01

There are various approaches to evaluating the usability of electronic medical record (EMR) systems. User perspectives are an integral part of evaluation. Usability evaluations efficiently and effectively contribute to user-centered design and supports tasks and increase user satisfaction. This study determined the main usability requirements for EMRs by means of an end-user survey. A mixed-method strategy was conducted in three phases. A qualitative approach was employed to collect and formulate EMR usability requirements using the focus group method and the modified Delphi technique. Classic Delphi technique was used to evaluate the proposed requirements among 380 end-users in Iran. The final list of EMR usability requirements was verified and included 163 requirements divided into nine groups. The highest rates of end-user agreement relate to EMR visual clarity (3.65 ± 0.61), fault tolerance (3.58 ± 0.56), and suitability for learning (3.55 ± 0.54). The lowest end-user agreement was for auditory presentation (3.18 ± 0.69). The highest and lowest agreement among end-users was for visual clarity and auditory presentation by EMRs, respectively. This suggests that user priorities in determination of EMR usability and their understanding of the importance of the types of individual tasks and context characteristics differ.

The use of think-aloud and instant data analysis in evaluation research: Exemplar and lessons learned.

PubMed

Joe, Jonathan; Chaudhuri, Shomir; Le, Thai; Thompson, Hilaire; Demiris, George

2015-08-01

While health information technologies have become increasingly popular, many have not been formally tested to ascertain their usability. Traditional rigorous methods take significant amounts of time and manpower to evaluate the usability of a system. In this paper, we evaluate the use of instant data analysis (IDA) as developed by Kjeldskov et al. to perform usability testing on a tool designed for older adults and caregivers. The IDA method is attractive because it takes significantly less time and manpower than the traditional usability testing methods. In this paper we demonstrate how IDA was used to evaluate usability of a multifunctional wellness tool, discuss study results and lessons learned while using this method. We also present findings from an extension of the method which allows the grouping of similar usability problems in an efficient manner. We found that the IDA method is a quick, relatively easy approach to identifying and ranking usability issues among health information technologies. Copyright © 2015 Elsevier Inc. All rights reserved.

Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.

PubMed

Brown, Benjamin; Balatsoukas, Panos; Williams, Richard; Sperrin, Matthew; Buchan, Iain

2016-10-01

Audit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians' clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn improving patient safety. To evaluate the usability of a novel theoretically-informed and research-led e-A&F system for primary care (the Performance Improvement plaN GeneratoR: PINGR). (1) Describe PINGR's design, rationale and theoretical basis; (2) Identify usability issues with PINGR; (3) Understand how these issues may interfere with the cognitive goals of end-users; (4) Translate the issues into recommendations for the user-centred design of e-A&F systems. Eight experienced health system evaluators performed a usability inspection using an innovative hybrid approach consisting of five stages: (1) Development of representative user tasks, Goals, and Actions; (2) Combining Heuristic Evaluation and Cognitive Walkthrough methods into a single protocol to identify usability issues; (3) Consolidation of issues; (4) Severity rating of consolidated issues; (5) Analysis of issues according to usability heuristics, interface components, and Goal-Action structure. A final list of 47 issues were categorised into 8 heuristic themes. The most error-prone heuristics were 'Consistency and standards' (13 usability issues; 28% of the total) and 'Match between system and real world' (n=10, 21%). The recommended actions component of the PINGR interface had the most usability issues (n=21, 45%), followed by patient-level data (n=5, 11%), patient lists (n=4, 9%), and summaries of clinical performance (n=4, 9%). The most error-prone Actions across all user Goals were: (1) Patient selection from a list; (2) Data identification from a figure (both population-level and patient-level); (3) Disagreement with a system recommendation. By contextualising our findings within the wider literature on health information system usability, we provide recommendations for the design of e-A&F system interfaces relating to their four key components, in addition to how they may be integrated within a system. Copyright © 2016. Published by Elsevier Ireland Ltd.

An appraisal of published usability evaluations of electronic health records via systematic review.

PubMed

Ellsworth, Marc A; Dziadzko, Mikhail; O'Horo, John C; Farrell, Ann M; Zhang, Jiajie; Herasevich, Vitaly

2017-01-01

In this systematic review, we aimed to evaluate methodological and reporting trends present in the current literature by investigating published usability studies of electronic health records (EHRs). A literature search was conducted for articles published through January 2015 using MEDLINE (Ovid), EMBASE, Scopus, and Web of Science, supplemented by citation and reference list reviews. Studies were included if they tested the usability of hospital and clinic EHR systems in the inpatient, outpatient, emergency department, or operating room setting. A total of 4848 references were identified for title and abstract screening. Full text screening was performed for 197 articles, with 120 meeting the criteria for study inclusion. A review of the literature demonstrates a paucity of quality published studies describing scientifically valid and reproducible usability evaluations at various stages of EHR system development. A lack of formal and standardized reporting of EHR usability evaluation results is a major contributor to this knowledge gap, and efforts to improve this deficiency will be one step of moving the field of usability engineering forward. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

A usability evaluation of four commercial dental computer-based patient record systems

PubMed Central

Thyvalikakath, Thankam P.; Monaco, Valerie; Thambuganipalle, Hima Bindu; Schleyer, Titus

2008-01-01

Background The usability of dental computer-based patient record (CPR) systems has not been studied, despite early evidence that poor usability is a problem for dental CPR system users at multiple levels. Methods The authors conducted formal usability tests of four dental CPR systems by using a purposive sample of four groups of five novice users. The authors measured task outcomes (correctly completed, incorrectly completed and incomplete) in each CPR system while the participants performed nine clinical documentation tasks, as well as the number of usability problems identified in each CPR system and their potential relationship to task outcomes. The authors reviewed the software application design aspects responsible for these usability problems. Results The range for correctly completed tasks was 16 to 64 percent, for incorrectly completed tasks 18 to 38 percent and for incomplete tasks 9 to 47 percent. The authors identified 286 usability problems. The main types were three unsuccessful attempts, negative affect and task incorrectly completed. They also identified six problematic interface and interaction designs that led to usability problems. Conclusion The four dental CPR systems studied have significant usability problems for novice users, resulting in a steep learning curve and potentially reduced system adoption. Clinical Implications The significant number of data entry errors raises concerns about the quality of documentation in clinical practice. PMID:19047669

Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches. Contribution of the IMIA Working Group Health Informatics for Patient Safety.

PubMed

Borycki, E; Kushniruk, A; Nohr, C; Takeda, H; Kuwata, S; Carvalho, C; Bainbridge, M; Kannry, J

2013-01-01

Issues related to lack of system usability and potential safety hazards continue to be reported in the health information technology (HIT) literature. Usability engineering methods are increasingly used to ensure improved system usability and they are also beginning to be applied more widely for ensuring the safety of HIT applications. These methods are being used in the design and implementation of many HIT systems. In this paper we describe evidence-based approaches to applying usability engineering methods. A multi-phased approach to ensuring system usability and safety in healthcare is described. Usability inspection methods are first described including the development of evidence-based safety heuristics for HIT. Laboratory-based usability testing is then conducted under artificial conditions to test if a system has any base level usability problems that need to be corrected. Usability problems that are detected are corrected and then a new phase is initiated where the system is tested under more realistic conditions using clinical simulations. This phase may involve testing the system with simulated patients. Finally, an additional phase may be conducted, involving a naturalistic study of system use under real-world clinical conditions. The methods described have been employed in the analysis of the usability and safety of a wide range of HIT applications, including electronic health record systems, decision support systems and consumer health applications. It has been found that at least usability inspection and usability testing should be applied prior to the widespread release of HIT. However, wherever possible, additional layers of testing involving clinical simulations and a naturalistic evaluation will likely detect usability and safety issues that may not otherwise be detected prior to widespread system release. The framework presented in the paper can be applied in order to develop more usable and safer HIT, based on multiple layers of evidence.

To twist or poke? A method for identifying usability issues with the rotary controller and touch screen for control of in-vehicle information systems.

PubMed

Harvey, Catherine; Stanton, Neville A; Pickering, Carl A; McDonald, Mike; Zheng, Pengjun

2011-07-01

In-vehicle information systems (IVIS) can be controlled by the user via direct or indirect input devices. In order to develop the next generation of usable IVIS, designers need to be able to evaluate and understand the usability issues associated with these two input types. The aim of this study was to investigate the effectiveness of a set of empirical usability evaluation methods for identifying important usability issues and distinguishing between the IVIS input devices. A number of usability issues were identified and their causal factors have been explored. These were related to the input type, the structure of the menu/tasks and hardware issues. In particular, the translation between inputs and on-screen actions and a lack of visual feedback for menu navigation resulted in lower levels of usability for the indirect device. This information will be useful in informing the design of new IVIS, with improved usability. STATEMENT OF RELEVANCE: This paper examines the use of empirical methods for distinguishing between direct and indirect IVIS input devices and identifying usability issues. Results have shown that the characteristics of indirect input devices produce more serious usability issues, compared with direct devices and can have a negative effect on the driver-vehicle interaction.

Usability engineering: domain analysis activities for augmented-reality systems

NASA Astrophysics Data System (ADS)

Gabbard, Joseph; Swan, J. E., II; Hix, Deborah; Lanzagorta, Marco O.; Livingston, Mark; Brown, Dennis B.; Julier, Simon J.

2002-05-01

This paper discusses our usability engineering process for the Battlefield Augmented Reality System (BARS). Usability engineering is a structured, iterative, stepwise development process. Like the related disciplines of software and systems engineering, usability engineering is a combination of management principals and techniques, formal and semi- formal evaluation techniques, and computerized tools. BARS is an outdoor augmented reality system that displays heads- up battlefield intelligence information to a dismounted warrior. The paper discusses our general usability engineering process. We originally developed the process in the context of virtual reality applications, but in this work we are adapting the procedures to an augmented reality system. The focus of this paper is our work on domain analysis, the first activity of the usability engineering process. We describe our plans for and our progress to date on our domain analysis for BARS. We give results in terms of a specific urban battlefield use case we have designed.

Developing an Android-Based Patient Decision Aid Based on Ottawa Standards for Patients After Kidney Transplant and Its Usability Evaluation.

PubMed

Zare Moayedi, Mahboobeh; Aslani, Azam; Fakhrahmad, Mostafa; Ezzatzadegan J, Shahrokh

2018-01-01

This study was conducted to develop an android based patient decision aid (PDA) as a self-care instrument for patients after kidney transplant and its usability evaluation. In this study, the systematic development process of Android-based self-care application for patients after kidney transplant based on Ottawa standard was included: scoping, assemble steering group, analysis of requirements, designing, develop of a prototype and system evaluation. The PDA is a self-triage system that will help early identification of risk symptoms in patients, and help manage them. System recommendations for risk signs are: Refer to the nearest hospital or healthcare center without delay, refer to the doctor and tell your doctor in the next visit. To identify patient care needs, a semi-structured interview with members of steering group, including patients and clinical experts, was conducted by the researchers. A prototype of the decision aid was made according to identified needs in the previous step. Finally, in order to evaluate its usability rate by using the System Usability Scale (SUS) questionnaire, it was used by exerts and patients. This study identified information needs, risk signs and steps that patients need to make appropriate decisions about them. The main capabilities of the decision aid are features such as reminders for appointment/test, time of taking medication, registration of symptoms, weight, blood pressure, body temperature, advising to patient in case of signs of risk, weight, blood pressure, body temperature and test results which were reported in the diagram. The mean score of system's usability evaluated by medical informatics specialists, clinicians, and patients were 88.33, 95, and 91. PDAs was usable and desirable from the point of view of medical informatics specialists, clinicians and patients.

Usability Study of a Wireless Monitoring System among Alzheimer's Disease Elderly Population

PubMed Central

Avvenuti, Marco

2014-01-01

Healthcare technologies are slowly entering into our daily lives, replacing old devices and techniques with newer intelligent ones. Although they are meant to help people, the reaction and willingness to use such new devices by the people can be unexpected, especially among the elderly. We conducted a usability study of a fall monitoring system in a long-term nursing home. The subjects were the elderly with advanced Alzheimer's disease. The study presented here highlights some of the challenges faced in the use of wearable devices and the lessons learned. The results gave us useful insights, leading to ergonomics and aesthetics modifications to our wearable systems that significantly improved their usability and acceptance. New evaluating metrics were designed for the performance evaluation of usability and acceptability. PMID:24963289

An evaluation of patients' experienced usability of a diabetes mHealth system using a multi-method approach.

PubMed

Georgsson, Mattias; Staggers, Nancy

2016-02-01

mHealth systems are becoming more common to aid patients in their diabetes self-management, but recent studies indicate a need for thorough evaluation of patients' experienced usability. Current evaluations lack a multi-method design for data collection and structured methods for data analyses. The purpose of this study was to provide a feasibility test of a multi-method approach for both data collection and data analyses for patients' experienced usability of a mHealth system for diabetes type 2 self-management. A random sample of 10 users was selected from a larger clinical trial. Data collection methods included user testing with eight representative tasks and Think Aloud protocol, a semi-structured interview and a questionnaire on patients' experiences using the system. The Framework Analysis (FA) method and Usability Problem Taxonomy (UPT) were used to structure, code and analyze the results. A usability severity rating was assigned after classification. The combined methods resulted in a total of 117 problems condensed into 19 usability issues with an average severity rating of 2.47 or serious. The usability test detected 50% of the initial usability problems, followed by the post-interview at 29%. The usability test found 18 of 19 consolidated usability problems while the questionnaire uncovered one unique issue. Patients experienced most usability problems (8) in the Glucose Readings View when performing complex tasks such as adding, deleting, and exporting glucose measurements. The severity ratings were the highest for the Glucose Diary View, Glucose Readings View, and Blood Pressure View with an average severity rating of 3 (serious). Most of the issues were classified under the artifact component of the UPT and primary categories of Visualness (7) and Manipulation (6). In the UPT task component, most issues were in the primary category Task-mapping (12). Multiple data collection methods yielded a more comprehensive set of usability issues. Usability testing uncovered the largest volume of usability issues, followed by interviewing and then the questionnaire. The interview did not surface any unique consolidated usability issues while the questionnaire surfaced one. The FA and UPT were valuable in structuring and classifying problems. The resulting descriptions serve as a communication tool in problem solving and programming. We recommend the usage of multiple methods in data collection and employing the FA and UPT in data analyses for future usability testing. Copyright © 2015 The Authors. Published by Elsevier Inc. All rights reserved.

Usability Evaluation at the Point-of-Care: A Method to Identify User Information Needs in CPOE Applications

PubMed Central

Washburn, Jeff; Fiol, Guilherme Del; Rocha, Roberto A.

2006-01-01

Point of care usability evaluation may help identify information needs that occur during the process of providing care. We describe the process of using usability-specific recording software to record Computerized Physician Order Entry (CPOE) ordering sessions on admitted adult and pediatric patients at two urban tertiary hospitals in the Intermountain Healthcare system of hospitals. PMID:17238756

Usability Evaluation and Implementation of a Health Information Technology Dashboard of Evidence-Based Quality Indicators.

PubMed

Schall, Mark Christopher; Cullen, Laura; Pennathur, Priyadarshini; Chen, Howard; Burrell, Keith; Matthews, Grace

2017-06-01

Health information technology dashboards that integrate evidence-based quality indicators can efficiently and accurately display patient risk information to promote early intervention and improve overall quality of patient care. We describe the process of developing, evaluating, and implementing a dashboard designed to promote quality care through display of evidence-based quality indicators within an electronic health record. Clinician feedback was sought throughout the process. Usability evaluations were provided by three nurse pairs and one physician from medical-surgical areas. Task completion times, error rates, and ratings of system usability were collected to compare the use of quality indicators displayed on the dashboard to the indicators displayed in a conventional electronic health record across eight experimental scenarios. Participants rated the dashboard as "highly usable" following System Usability Scale (mean, 87.5 [SD, 9.6]) and Poststudy System Usability Questionnaire (mean, 1.7 [SD, 0.5]) criteria. Use of the dashboard led to reduced task completion times and error rates in comparison to the conventional electronic health record for quality indicator-related tasks. Clinician responses to the dashboard display capabilities were positive, and a multifaceted implementation plan has been used. Results suggest application of the dashboard in the care environment may lead to improved patient care.

Usability of clinical decision support system as a facilitator for learning the assistive technology adaptation process.

PubMed

Danial-Saad, Alexandra; Kuflik, Tsvi; Weiss, Patrice L Tamar; Schreuer, Naomi

2016-01-01

The aim of this study was to evaluate the usability of Ontology Supported Computerized Assistive Technology Recommender (OSCAR), a Clinical Decision Support System (CDSS) for the assistive technology adaptation process, its impact on learning the matching process, and to determine the relationship between its usability and learnability. Two groups of expert and novice clinicians (total, n = 26) took part in this study. Each group filled out system usability scale (SUS) to evaluate OSCAR's usability. The novice group completed a learning questionnaire to assess OSCAR's effect on their ability to learn the matching process. Both groups rated OSCAR's usability as "very good", (M [SUS] = 80.7, SD = 11.6, median = 83.7) by the novices, and (M [SUS] = 81.2, SD = 6.8, median = 81.2) by the experts. The Mann-Whitney results indicated that no significant differences were found between the expert and novice groups in terms of OSCAR's usability. A significant positive correlation existed between the usability of OSCAR and the ability to learn the adaptation process (rs = 0.46, p = 0.04). Usability is an important factor in the acceptance of a system. The successful application of user-centered design principles during the development of OSCAR may serve as a case study that models the significant elements to be considered, theoretically and practically in developing other systems. Implications for Rehabilitation Creating a CDSS with a focus on its usability is an important factor for its acceptance by its users. Successful usability outcomes can impact the learning process of the subject matter in general, and the AT prescription process in particular. The successful application of User-Centered Design principles during the development of OSCAR may serve as a case study that models the significant elements to be considered, theoretically and practically. The study emphasizes the importance of close collaboration between the developers and the end users.

Usability of Low-Cost Android Data Collection System for Community-Based Participatory Research.

PubMed

Salihu, Hamisu M; Salinas-Miranda, Abraham; Turner, DeAnne; King, Lindsey; Paothong, Arnut; Austin, Deborah; Berry, Estrellita Lo

2016-01-01

Android tablet computers can be valuable tools for data collection, but their usability has not been evaluated in community-based participatory research (CBPR). This article examines the usability of a low-cost bilingual touchscreen computerized survey system using Android tablets, piloted with a sample of 201 community residents in Tampa, Florida, from November 2013 to March 2014. Needs assessment questions were designed with the droidSURVEY software, and deployed using Android tablet computers. In addition, participants were asked questions about system usability. The mean system usability was 77.57 ± 17.66 (range, 0-100). The mean completion time for taking the 63 survey questions in the needs assessment was 23.11 ± 9.62 minutes. The survey completion rate was optimal (100%), with only 6.34% missingness per variable. We found no sociodemographic differences in usability scores. Our findings indicate that Android tablets could serve as useful tools in CBPR studies.

Physician satisfaction with a critical care clinical information system using a multimethod evaluation of usability.

PubMed

Hudson, Darren; Kushniruk, Andre; Borycki, Elizabeth; Zuege, Danny J

2018-04-01

Physician satisfaction with electronic medical records has often been poor. Usability has frequently been identified as a source for decreased satisfaction. While surveys can identify many issues, and are logistically easier to administer, they may miss issues identified using other methods This study sought to understand the level of physician satisfaction and usability issues associated with a critical care clinical information system (eCritical Alberta) implemented throughout the province of Alberta, Canada. All critical care attending physicians using the system were invited to participate in an online survey. Questions included components of the User Acceptance of Information Technology and Usability Questionnaire as well as free text feedback on system components. Physicians were also invited to participate in a think aloud test using simulated scenarios. The transcribed think aloud text and questionnaire were subjected to textual analysis. 82% of all eligible physicians completed the on-line survey (n = 61). Eight physicians were invited and seven completed the think aloud test. Overall satisfaction with the system was moderate. Usability was identified as a significant factor contributing to satisfaction. The major usability factors identified were system response time and layout. The think aloud component identified additional factors beyond those identified in the on-line survey. This study found a modestly high level of physician satisfaction with a province-wide clinical critical care information system. Usability continues to be a significant factor in physician satisfaction. Using multiple methods of evaluation can capture the benefits of a large sample size and deeper understanding of the issues. Copyright © 2018 Elsevier B.V. All rights reserved.

Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics.

PubMed

Georgsson, Mattias; Staggers, Nancy

2016-01-01

Mobile health (mHealth) systems are becoming more common for chronic disease management, but usability studies are still needed on patients' perspectives and mHealth interaction performance. This deficiency is addressed by our quantitative usability study of a mHealth diabetes system evaluating patients' task performance, satisfaction, and the relationship of these measures to user characteristics. We used metrics in the International Organization for Standardization (ISO) 9241-11 standard. After standardized training, 10 patients performed representative tasks and were assessed on individual task success, errors, efficiency (time on task), satisfaction (System Usability Scale [SUS]) and user characteristics. Tasks of exporting and correcting values proved the most difficult, had the most errors, the lowest task success rates, and consumed the longest times on task. The average SUS satisfaction score was 80.5, indicating good but not excellent system usability. Data trends showed males were more successful in task completion, and younger participants had higher performance scores. Educational level did not influence performance, but a more recent diabetes diagnosis did. Patients with more experience in information technology (IT) also had higher performance rates. Difficult task performance indicated areas for redesign. Our methods can assist others in identifying areas in need of improvement. Data about user background and IT skills also showed how user characteristics influence performance and can provide future considerations for targeted mHealth designs. Using the ISO 9241-11 usability standard, the SUS instrument for satisfaction and measuring user characteristics provided objective measures of patients' experienced usability. These could serve as an exemplar for standardized, quantitative methods for usability studies on mHealth systems. © The Author 2015. Published by Oxford University Press on behalf of the American Medical Informatics Association.

Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics

PubMed Central

Staggers, Nancy

2016-01-01

Objective Mobile health (mHealth) systems are becoming more common for chronic disease management, but usability studies are still needed on patients’ perspectives and mHealth interaction performance. This deficiency is addressed by our quantitative usability study of a mHealth diabetes system evaluating patients’ task performance, satisfaction, and the relationship of these measures to user characteristics. Materials and Methods We used metrics in the International Organization for Standardization (ISO) 9241-11 standard. After standardized training, 10 patients performed representative tasks and were assessed on individual task success, errors, efficiency (time on task), satisfaction (System Usability Scale [SUS]) and user characteristics. Results Tasks of exporting and correcting values proved the most difficult, had the most errors, the lowest task success rates, and consumed the longest times on task. The average SUS satisfaction score was 80.5, indicating good but not excellent system usability. Data trends showed males were more successful in task completion, and younger participants had higher performance scores. Educational level did not influence performance, but a more recent diabetes diagnosis did. Patients with more experience in information technology (IT) also had higher performance rates. Discussion Difficult task performance indicated areas for redesign. Our methods can assist others in identifying areas in need of improvement. Data about user background and IT skills also showed how user characteristics influence performance and can provide future considerations for targeted mHealth designs. Conclusion Using the ISO 9241-11 usability standard, the SUS instrument for satisfaction and measuring user characteristics provided objective measures of patients’ experienced usability. These could serve as an exemplar for standardized, quantitative methods for usability studies on mHealth systems. PMID:26377990

CUNY+ Web: Usability Study of the Web-Based GUI Version of the Bibliographic Database of the City University of New York (CUNY).

ERIC Educational Resources Information Center

Oulanov, Alexei; Pajarillo, Edmund J. Y.

2002-01-01

Describes the usability evaluation of the CUNY (City University of New York) information system in Web and Graphical User Interface (GUI) versions. Compares results to an earlier usability study of the basic information database available on CUNY's wide-area network and describes the applicability of the previous usability instrument to this…

Evaluation of Nursing Information Systems: Application of Usability Aspects in the Development of Systems.

PubMed

Moghaddasi, Hamid; Rabiei, Reza; Asadi, Farkhondeh; Ostvan, Negin

2017-04-01

As the largest group providing healthcare services, nurses require well-designed information systems in their practice. This study aims to evaluate the usability of nursing information systems (NIS). This cross-sectional survey was conducted in 2015. The settings of the study consisted of four hospitals affiliated with three medical universities in Tehran (Iran). The subjects of the study included nurses who had access to and used a NIS developed by four major software companies. The data were collected using a modified version of a usability questionnaire known as IsoMetrics, based on the International Standard ISO 9241, Part 11. The questionnaire is composed of 35 questions divided into seven general criteria. The validity of the questionnaire was determined by experts in the field, and the reliability was checked using Cronbach's alpha (α = 0.91). The questionnaire was then distributed to 184 nurses. The response rate was 64.6%. Among the seven ISO usability criteria, suitability for the task (3.10 ± 1.24) and suitability for learning (3.10 ± 1.27) had the highest mean value. The lowest mean value (2.37 ± 1.29) was related to the suitability for individualization. Addressing issues related to individualization and self-descriptiveness could improve the usability of nursing systems. Considering usability requirements in the design of a NIS will lead to the efficient and effective use of these systems.

Lessons learned from the usability assessment of home-based telemedicine systems.

PubMed

Agnisarman, Sruthy Orozhiyathumana; Chalil Madathil, Kapil; Smith, Kevin; Ashok, Aparna; Welch, Brandon; McElligott, James T

2017-01-01

At-home telemedicine visits are quickly becoming an acceptable alternative for in-person patient visits. However, little work has been done to understand the usability of these home-based telemedicine solutions. It is critical for user acceptance and real-world applicability to evaluate available telemedicine solutions within the context-specific needs of the users of this technology. To address this need, this study evaluated the usability of four home-based telemedicine software platforms: Doxy.me, Vidyo, VSee, and Polycom. Using a within-subjects experimental design, twenty participants were asked to complete a telemedicine session involving several tasks using the four platforms. Upon completion of these tasks for each platform, participants completed the IBM computer system usability questionnaire (CSUQ) and the NASA Task Load Index test. Upon completing the tasks on all four platforms, the participants completed a final post-test subjective questionnaire ranking the platforms based on their preference. Of the twenty participants, 19 completed the study. Statistically significant differences among the telemedicine software platforms were found for task completion time, total workload, mental demand, effort, frustration, preference ranking and computer system usability scores. Usability problems with installation and account creation led to high mental demand and task completion time, suggesting the participants preferred a system without such requirements. Majority of the usability issues were identified at the telemedicine initiation phase. The findings from this study can be used by software developers to develop user-friendly telemedicine systems. Copyright © 2016 Elsevier Ltd. All rights reserved.

Evaluating the usability of an interactive, bi-lingual, touchscreen-enabled breastfeeding educational programme: application of Nielson's heuristics.

PubMed

Joshi, Ashish; Perin, Douglas M Puricelli; Amadi, Chioma; Trout, Kate

2015-03-05

The study purpose was to conduct heuristic evaluation of an interactive, bilingual touchscreen-enabled breastfeeding educational programme for Hispanic women living in rural settings in Nebraska. Three raters conducted the evaluation during May 2013 using principles of Nielson's heuristics. A total of 271 screens were evaluated and included: interface (n = 5), programme sections (n = 223) and educational content (n = 43). A total of 97 heuristic violations were identified and were mostly related to interface (8 violations/5 screens) and programme components (89 violations/266 screens). The most common heuristic violations reported were recognition rather than recall (62%, n = 60), consistency and standards (14%, n = 14) and match between the system and real world (9%, n = 9). Majority of the heuristic violations had minor usability issues (73%, n = 71). The only grade 4 heuristic violation reported was due to the visibility of system status in the assessment modules. The results demonstrated that the system was more consistent with Nielsen's usability heuristics. With Nielsen's usability heuristics, it is possible to identify problems in a timely manner, and help facilitate the identification and prioritisation of problems needing urgent attention at an earlier stage before the final deployment of the system.

Usability testing of an mHealth device for swallowing therapy in head and neck cancer survivors.

PubMed

Constantinescu, Gabriela; Kuffel, Kristina; King, Ben; Hodgetts, William; Rieger, Jana

2018-04-01

The objective of this study was to conduct the first patient usability testing of a mobile health (mHealth) system for in-home swallowing therapy. Five participants with a history of head and neck cancer evaluated the mHealth system. After completing an in-application (app) tutorial with the clinician, participants were asked to independently complete five tasks: pair the device to the smartphone, place the device correctly, exercise, interpret progress displays, and close the system. Quantitative and qualitative methods were used to evaluate the effectiveness, efficiency, and satisfaction with the system. Critical changes to the app were found in three of the tasks, resulting in recommendations for the next iteration. These issues were related to ease of Bluetooth pairing, placement of device, and interpretation of statistics. Usability testing with patients identified issues that were essential to address prior to implementing the mHealth system in subsequent clinical trials. Of the usability methods used, video observation (synced screen capture with videoed gestures) revealed the most information.

Pattern mining of user interaction logs for a post-deployment usability evaluation of a radiology PACS client.

PubMed

Jorritsma, Wiard; Cnossen, Fokie; Dierckx, Rudi A; Oudkerk, Matthijs; van Ooijen, Peter M A

2016-01-01

To perform a post-deployment usability evaluation of a radiology Picture Archiving and Communication System (PACS) client based on pattern mining of user interaction log data, and to assess the usefulness of this approach compared to a field study. All user actions performed on the PACS client were logged for four months. A data mining technique called closed sequential pattern mining was used to automatically extract frequently occurring interaction patterns from the log data. These patterns were used to identify usability issues with the PACS. The results of this evaluation were compared to the results of a field study based usability evaluation of the same PACS client. The interaction patterns revealed four usability issues: (1) the display protocols do not function properly, (2) the line measurement tool stays active until another tool is selected, rather than being deactivated after one use, (3) the PACS's built-in 3D functionality does not allow users to effectively perform certain 3D-related tasks, (4) users underuse the PACS's customization possibilities. All usability issues identified based on the log data were also found in the field study, which identified 48 issues in total. Post-deployment usability evaluation based on pattern mining of user interaction log data provides useful insights into the way users interact with the radiology PACS client. However, it reveals few usability issues compared to a field study and should therefore not be used as the sole method of usability evaluation. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Integrating Low-Cost Rapid Usability Testing into Agile System Development of Healthcare IT: A Methodological Perspective.

PubMed

Kushniruk, Andre W; Borycki, Elizabeth M

2015-01-01

The development of more usable and effective healthcare information systems has become a critical issue. In the software industry methodologies such as agile and iterative development processes have emerged to lead to more effective and usable systems. These approaches highlight focusing on user needs and promoting iterative and flexible development practices. Evaluation and testing of iterative agile development cycles is considered an important part of the agile methodology and iterative processes for system design and re-design. However, the issue of how to effectively integrate usability testing methods into rapid and flexible agile design cycles has remained to be fully explored. In this paper we describe our application of an approach known as low-cost rapid usability testing as it has been applied within agile system development in healthcare. The advantages of the integrative approach are described, along with current methodological considerations.

INCOMMANDS TDP: Human Factors Evaluation of the Command Decision Support Capability Prototype

DTIC Science & Technology

2009-03-01

CDSC) design with guidelines and principles stipulated in the INCOMMANDS TDP Human Factors Design and Evaluation Guide. 2. Usability and Utility...INCOMMANDS CDSC. The Heuristic Evaluation helps to ensure that the design of system is compliant with HCI best practices. Whereas the Usability and...implemented to confirm compliance with the INCOMMANDS TDP Human Factors Design and Evaluation Guide

Using Heuristic Evaluation to Improve Sepsis Alert Usability.

PubMed

Pertiwi, Ariani Arista Putri; Fraczkowski, Dan; Stogis, Sheryl L; Lopez, Karen Dunn

2018-06-01

Sepsis, life-threatening organ dysfunction in response to infection, is an alarmingly common and aggressive illness in US hospitals, especially for intensive care patients. Preventing sepsis deaths rests on the clinicians' ability to promptly recognize and treat sepsis. To aid early recognition, many organizations have employed clinician-facing electronic sepsis alert systems. However, the effectiveness of the alert relies on heavily on the visual interface, textual information, and overall usability. This article reports a usability inspection of a sepsis alert system. The authors found violations in 12 of the 14 usability principles and promote use of this method in practice to systematically identify usability problems. Copyright © 2018 Elsevier Inc. All rights reserved.

Tree Testing of Hierarchical Menu Structures for Health Applications

PubMed Central

Le, Thai; Chaudhuri, Shomir; Chung, Jane; Thompson, Hilaire J; Demiris, George

2014-01-01

To address the need for greater evidence-based evaluation of Health Information Technology (HIT) systems we introduce a method of usability testing termed tree testing. In a tree test, participants are presented with an abstract hierarchical tree of the system taxonomy and asked to navigate through the tree in completing representative tasks. We apply tree testing to a commercially available health application, demonstrating a use case and providing a comparison with more traditional in-person usability testing methods. Online tree tests (N=54) and in-person usability tests (N=15) were conducted from August to September 2013. Tree testing provided a method to quantitatively evaluate the information structure of a system using various navigational metrics including completion time, task accuracy, and path length. The results of the analyses compared favorably to the results seen from the traditional usability test. Tree testing provides a flexible, evidence-based approach for researchers to evaluate the information structure of HITs. In addition, remote tree testing provides a quick, flexible, and high volume method of acquiring feedback in a structured format that allows for quantitative comparisons. With the diverse nature and often large quantities of health information available, addressing issues of terminology and concept classifications during the early development process of a health information system will improve navigation through the system and save future resources. Tree testing is a usability method that can be used to quickly and easily assess information hierarchy of health information systems. PMID:24582924

Usability evaluation of the interactive Personal Patient Profile-Prostate decision support system with African American men.

PubMed

Jaja, Cheedy; Pares-Avila, Jose; Wolpin, Seth; Berry, Donna

2010-04-01

The Personal Patient Profile-Prostate (P4) program is an interactive Web-based decision support system that provides men with localized prostate cancer customized education and coaching with which to make the best personal treatment decision. This study assessed functionality and usability of the P4 program and identified problems in user-computer interaction in a sample of African American men. Usability testing was conducted with 12 community-dwelling African American adult men. The health status of participants was not known or collected by the research team. Each participant worked with the P4 program and provided simultaneous feedback using the "think aloud" technique. Handwritten field notes were collated and assigned to 3 standard coded categories. Aspects of P4 program usability was made based on common issues in the assigned categories. Summary statistics were derived for types and frequency of usability issues noted in the coded data. Twelve participants reported a total of 122 usability comments, with a mean of 9 usability comments. The most common usability issue by participant was completeness of information content, which comprised 53 (43%) of the total issues. Comprehensibility of text and graphics was second, comprising 51 (42%) of the total issues. This study provided initial inventory of usability issues for community African American men that may potentially interfere with application of the P4 system in the community setting and overall system usability, confirming the need for usability testing of a culturally appropriate Internet-based decision support system before community application.

Usability Testing of Two Ambulatory EHR Navigators.

PubMed

Hultman, Gretchen; Marquard, Jenna; Arsoniadis, Elliot; Mink, Pamela; Rizvi, Rubina; Ramer, Tim; Khairat, Saif; Fickau, Keri; Melton, Genevieve B

2016-01-01

Despite widespread electronic health record (EHR) adoption, poor EHR system usability continues to be a significant barrier to effective system use for end users. One key to addressing usability problems is to employ user testing and user-centered design. To understand if redesigning an EHR-based navigation tool with clinician input improved user performance and satisfaction. A usability evaluation was conducted to compare two versions of a redesigned ambulatory navigator. Participants completed tasks for five patient cases using the navigators, while employing a think-aloud protocol. The tasks were based on Meaningful Use (MU) requirements. The version of navigator did not affect perceived workload, and time to complete tasks was longer in the redesigned navigator. A relatively small portion of navigator content was used to complete the MU-related tasks, though navigation patterns were highly variable across participants for both navigators. Preferences for EHR navigation structures appeared to be individualized. This study demonstrates the importance of EHR usability assessments to evaluate group and individual performance of different interfaces and preferences for each design.

77 FR 8217 - Evaluating the Usability of Electronic Health Record (EHR) Systems

Federal Register 2010, 2011, 2012, 2013, 2014

2012-02-14

... interface design guidelines for EHRs. Manufacturers interested in participating in this research will be... the usability of health information technology (HIT) systems. NIST research is designed to: (1... develop performance-oriented user interface design guidelines for EHRs, and a framework for assessing the...

Heuristic Evaluation on Mobile Interfaces: A New Checklist

PubMed Central

Yáñez Gómez, Rosa; Cascado Caballero, Daniel; Sevillano, José-Luis

2014-01-01

The rapid evolution and adoption of mobile devices raise new usability challenges, given their limitations (in screen size, battery life, etc.) as well as the specific requirements of this new interaction. Traditional evaluation techniques need to be adapted in order for these requirements to be met. Heuristic evaluation (HE), an Inspection Method based on evaluation conducted by experts over a real system or prototype, is based on checklists which are desktop-centred and do not adequately detect mobile-specific usability issues. In this paper, we propose a compilation of heuristic evaluation checklists taken from the existing bibliography but readapted to new mobile interfaces. Selecting and rearranging these heuristic guidelines offer a tool which works well not just for evaluation but also as a best-practices checklist. The result is a comprehensive checklist which is experimentally evaluated as a design tool. This experimental evaluation involved two software engineers without any specific knowledge about usability, a group of ten users who compared the usability of a first prototype designed without our heuristics, and a second one after applying the proposed checklist. The results of this experiment show the usefulness of the proposed checklist for avoiding usability gaps even with nontrained developers. PMID:25295300

Getting a Technology-based Diabetes Intervention Ready for Primetime: A Review of Usability Testing Studies

PubMed Central

Lyles, Courtney R.; Sarkar, Urmimala; Osborn, Chandra Y.

2014-01-01

Consumer health technologies can educate patients about diabetes and support their self-management, yet usability evidence is rarely published even though it determines patient engagement, optimal benefit of any intervention, and an understanding of generalizability. Therefore, we conducted a narrative review of peer-reviewed articles published from 2009–2013 that tested the usability of a web- or mobile-delivered system/application designed to educate and support patients with diabetes. Overall, the 23 papers included in our review used mixed (n=11), descriptive quantitative (n=9), and qualitative methods (n=3) to assess usability, such as documenting which features performed as intended and how patients rated their experiences. More sophisticated usability evaluations combined several complementary approaches to elucidate more aspects of functionality. Future work pertaining to the design and evaluation of technology-delivered diabetes education/support interventions should aim to standardize the usability testing processes, and publish usability findings to inform interpretation of why an intervention succeeded or failed, and for whom. PMID:25173689

How to Present Evidence-Based Usability Design Principles Dedicated to Medication-Related Alerting Systems to Designers and Evaluators? Results from a Workshop.

PubMed

Marcilly, Romaric; Monkman, Helen; Villumsen, Sidsel; Kaufman, David; Beuscart-Zephir, Marie-Catherine

2016-01-01

Medication alerting system use errors and lack of adoption are often attributed to usability issues. Previous work has used evidence from the literature to reveal usability principles specific to medication alerting systems and identify potential consequences of violating these principles. The current study sought to explore how best to convey these principles to designers and evaluators of these systems to facilitate their work. To this aim, a workshop with 19 participants was used to generate ideas and opinions on how to deliver these topic-specific design principles in a way that would be most helpful for them. Participants generated ideas for how (e.g., a collaborative, continuously updated forum) and what (e.g., illustrations, checklists, evidence sources and strength, consequences of violations) information is most useful to disseminate usability principles for medication alerting systems. Participants, especially designers, expressed desire to use these principles in practice and avoid previously documented mistakes and therefore make design and evaluation of these systems more effective and efficient. Those insights are discussed in terms of feasibility and logistical challenges to developing the proposed documentation). To move this work forward, a more collaborative approach of Human Factors specialists in medical informatics is necessary.

Using scenarios and personas to enhance the effectiveness of heuristic usability evaluations for older adults and their care team.

PubMed

Kneale, Laura; Mikles, Sean; Choi, Yong K; Thompson, Hilaire; Demiris, George

2017-09-01

Using heuristics to evaluate user experience is a common methodology for human-computer interaction studies. One challenge of this method is the inability to tailor results towards specific end-user needs. This manuscript reports on a method that uses validated scenarios and personas of older adults and care team members to enhance heuristics evaluations of the usability of commercially available personal health records for homebound older adults. Our work extends the Chisnell and Redish heuristic evaluation methodology by using a protocol that relies on multiple expert reviews of each system. It further standardizes the heuristic evaluation process through the incorporation of task-based scenarios. We were able to use the modified version of the Chisnell and Redish heuristic evaluation methodology to identify potential usability challenges of two commercially available personal health record systems. This allowed us to: (1) identify potential usability challenges for specific types of users, (2) describe improvements that would be valuable to all end-users of the system, and (3) better understand how the interactions of different users may vary within a single personal health record. The methodology described in this paper may help designers of consumer health information technology tools, such as personal health records, understand the needs of diverse end-user populations. Such methods may be particularly helpful when designing systems for populations that are difficult to recruit for end-user evaluations through traditional methods. Copyright © 2017 Elsevier Inc. All rights reserved.

An evaluation method of computer usability based on human-to-computer information transmission model.

PubMed

Ogawa, K

1992-01-01

This paper proposes a new evaluation and prediction method for computer usability. This method is based on our two previously proposed information transmission measures created from a human-to-computer information transmission model. The model has three information transmission levels: the device, software, and task content levels. Two measures, called the device independent information measure (DI) and the computer independent information measure (CI), defined on the software and task content levels respectively, are given as the amount of information transmitted. Two information transmission rates are defined as DI/T and CI/T, where T is the task completion time: the device independent information transmission rate (RDI), and the computer independent information transmission rate (RCI). The method utilizes the RDI and RCI rates to evaluate relatively the usability of software and device operations on different computer systems. Experiments using three different systems, in this case a graphical information input task, confirm that the method offers an efficient way of determining computer usability.

Cognitive engineering of film library transition from film medium to digital environment in a Texas teaching hospital.

PubMed

Koperwhats, Martha A; Chang, Wei-Chih; Xiao, Jianguo

2002-01-01

Digital imaging technology promises efficient, economical, and fast service for patient care, but the challenges are great in the transition from film to a filmless (digital) environment. This change has a significant impact on the film library's personnel (film librarians) who play a leading roles in storage, classification, and retrieval of images. The objectives of this project were to study film library errors and the usability of a physical computerized system that could not be changed, while developing an intervention to reduce errors and test the usability of the intervention. Cognitive and human factors analysis were used to evaluate human-computer interaction. A workflow analysis was performed to understand the film and digital imaging processes. User and task analyses were applied to account for all behaviors involved in interaction with the system. A heuristic evaluation was used to probe the usability issues in the picture archiving and communication systems (PACS) modules. Simplified paper-based instructions were designed to familiarize the film librarians with the digital system. A usability survey evaluated the effectiveness of the instruction. The user and task analyses indicated that different users faced challenges based on their computer literacy, education, roles, and frequency of use of diagnostic imaging. The workflow analysis showed that the approaches to using the digital library differ among the various departments. The heuristic evaluation of the PACS modules showed the human-computer interface to have usability issues that prevented easy operation. Simplified instructions were designed for operation of the modules. Usability surveys conducted before and after revision of the instructions showed that performance improved. Cognitive and human factor analysis can help film librarians and other users adapt to the filmless system. Use of cognitive science tools will aid in successful transition of the film library from a film environment to a digital environment.

Usability Evaluation of a Research Repository and Collaboration Web Site

ERIC Educational Resources Information Center

Zhang, Tao; Maron, Deborah J.; Charles, Christopher C.

2013-01-01

This article reports results from an empirical usability evaluation of Human-Animal Bond Research Initiative Central as part of the effort to develop an open access research repository and collaboration platform for human-animal bond researchers. By repurposing and altering key features of the original HUBzero system, Human-Animal Bond Research…

A usability study of a mobile health application for rural Ghanaian midwives.

PubMed

Vélez, Olivia; Okyere, Portia Boakye; Kanter, Andrew S; Bakken, Suzanne

2014-01-01

Midwives in rural Ghana work at the frontline of the health care system, where they have access to essential data about the patient population. However, current methods of data capture, primarily pen and paper, make the data neither accessible nor usable for monitoring patient care or program evaluation. Electronic health (eHealth) systems present a potential mechanism for enhancing the roles of midwives by providing tools for collecting, exchanging, and viewing patient data as well as offering midwives the possibility for receiving information and decision support. Introducing such technology in low-resource settings has been challenging because of low levels of user acceptance, software design that does not match the end-user environment, and/or unforeseen challenges such as irregular power availability. These challenges are often attributable to a lack of understanding by the software developers of the end users' needs and work environment. A mobile health (mHealth) application known as mClinic was designed to support midwife access to the Millennium Village-Global Network, an eHealth delivery platform that captures data for managing patient care as well as program evaluation and monitoring, decision making, and management. We conducted a descriptive usability study composed of 3 phases to evaluate an mClinic prototype: 1) hybrid lab-live software evaluation of mClinic to identify usability issues; 2) completion of a usability questionnaire; and 3) interviews that included low-fidelity prototyping of new functionality proposed by midwives. The heuristic evaluation identified usability problems related to 4 of 8 usability categories. Analysis of usability questionnaire data indicated that the midwives perceived mClinic as useful but were more neutral about the ease of use. Analysis of midwives' reactions to low-fidelity prototypes during the interview process supported the applicability of mClinic to midwives' work and identified the need for additional functionality. User acceptance is essential for the success of any mHealth implementation. Usability testing identified mClinic development flaws and needed software enhancements. © 2014 by the American College of Nurse-Midwives.

Developing and Validation a Usability Evaluation Tools for Distance Education Websites: Persian Version

ERIC Educational Resources Information Center

Hafezi, Soheila; Farahi, Ahmad; Mehri, Soheil Najafi; Mahmoodi, Hosein

2010-01-01

The web is playing a central role in distance education. The word "usability" is usually synonymous with functionality of the system for the user. Also, usability of a website is defined as something that can be used by a specific group of people to carry out specific objectives in an effective way, with efficiency and satisfaction.…

The E-health Literacy Demands of Australia's My Health Record: A Heuristic Evaluation of Usability.

PubMed

Walsh, Louisa; Hemsley, Bronwyn; Allan, Meredith; Adams, Natalie; Balandin, Susan; Georgiou, Andrew; Higgins, Isabel; McCarthy, Shaun; Hill, Sophie

2017-01-01

My Health Record is Australia's electronic personal health record system, which was introduced in July 2012. As of August 2017, approximately 21 percent of Australia's total population was registered to use My Health Record. Internationally, usability issues have been shown to negatively influence the uptake and use of electronic health record systems, and this scenario may particularly affect people who have low e-health literacy. It is likely that usability issues are negatively affecting the uptake and use of My Health Record in Australia. To identify potential e-health literacy-related usability issues within My Health Record through a heuristic evaluation method. Between September 14 and October 12, 2016, three of the authors conducted a heuristic evaluation of the two consumer-facing components of My Health Record-the information website and the electronic health record itself. These two components were evaluated against two sets of heuristics-the Health Literacy Online checklist and the Monkman Heuristics. The Health Literacy Online checklist and Monkman Heuristics are evidence-based checklists of web design elements with a focus on design for audiences with low health literacy. During this heuristic evaluation, the investigators individually navigated through the consumer-facing components of My Health Record, recording instances where the My Health Record did not conform to the checklist criteria. After the individual evaluations were completed, the investigators conferred and aggregated their results. From this process, a list of usability violations was constructed. When evaluated against the Health Literacy Online Checklist, the information website demonstrated violations in 12 of 35 criteria, and the electronic health record demonstrated violations in 16 of 35 criteria. When evaluated against the Monkman Heuristics, the information website demonstrated violations in 7 of 11 criteria, and the electronic health record demonstrated violations in 9 of 11 criteria. The identified violations included usability issues with the reading levels used within My Health Record, the graphic design elements, the layout of web pages, and a lack of images and audiovisual tools to support learning. Other important usability issues included a lack of translated resources, difficulty using accessibility tools, and complexity of the registration processes. My Health Record is an important piece of technology that has the potential to facilitate better communication between consumers and their health providers. However, this heuristic evaluation demonstrated that many usability-related elements of My Health Record cater poorly to users at risk of having low e-health literacy. Usability issues have been identified as an important barrier to use of personal health records internationally, and the findings of this heuristic evaluation demonstrate that usability issues may be substantial barriers to the uptake and use of My Health Record.

The E-health Literacy Demands of Australia's My Health Record: A Heuristic Evaluation of Usability

PubMed Central

Walsh, Louisa; Hemsley, Bronwyn; Allan, Meredith; Adams, Natalie; Balandin, Susan; Georgiou, Andrew; Higgins, Isabel; McCarthy, Shaun; Hill, Sophie

2017-01-01

Background My Health Record is Australia's electronic personal health record system, which was introduced in July 2012. As of August 2017, approximately 21 percent of Australia's total population was registered to use My Health Record. Internationally, usability issues have been shown to negatively influence the uptake and use of electronic health record systems, and this scenario may particularly affect people who have low e-health literacy. It is likely that usability issues are negatively affecting the uptake and use of My Health Record in Australia. Objective To identify potential e-health literacy–related usability issues within My Health Record through a heuristic evaluation method. Methods Between September 14 and October 12, 2016, three of the authors conducted a heuristic evaluation of the two consumer-facing components of My Health Record—the information website and the electronic health record itself. These two components were evaluated against two sets of heuristics—the Health Literacy Online checklist and the Monkman Heuristics. The Health Literacy Online checklist and Monkman Heuristics are evidence-based checklists of web design elements with a focus on design for audiences with low health literacy. During this heuristic evaluation, the investigators individually navigated through the consumer-facing components of My Health Record, recording instances where the My Health Record did not conform to the checklist criteria. After the individual evaluations were completed, the investigators conferred and aggregated their results. From this process, a list of usability violations was constructed. Results When evaluated against the Health Literacy Online Checklist, the information website demonstrated violations in 12 of 35 criteria, and the electronic health record demonstrated violations in 16 of 35 criteria. When evaluated against the Monkman Heuristics, the information website demonstrated violations in 7 of 11 criteria, and the electronic health record demonstrated violations in 9 of 11 criteria. The identified violations included usability issues with the reading levels used within My Health Record, the graphic design elements, the layout of web pages, and a lack of images and audiovisual tools to support learning. Other important usability issues included a lack of translated resources, difficulty using accessibility tools, and complexity of the registration processes. Conclusion My Health Record is an important piece of technology that has the potential to facilitate better communication between consumers and their health providers. However, this heuristic evaluation demonstrated that many usability-related elements of My Health Record cater poorly to users at risk of having low e-health literacy. Usability issues have been identified as an important barrier to use of personal health records internationally, and the findings of this heuristic evaluation demonstrate that usability issues may be substantial barriers to the uptake and use of My Health Record. PMID:29118683

Adapting Nielsen’s Design Heuristics to Dual Processing for Clinical Decision Support

PubMed Central

Taft, Teresa; Staes, Catherine; Slager, Stacey; Weir, Charlene

2016-01-01

The study objective was to improve the applicability of Nielson’s standard design heuristics for evaluating electronic health record (EHR) alerts and linked ordering support by integrating them with Dual Process theory. Through initial heuristic evaluation and a user study of 7 physicians, usability problems were identified. Through independent mapping of specific usability criteria to support for each of the Dual Cognitive processes (S1 and S2) and deliberation, agreement was reached on mapping criteria. Finally, usability errors from the heuristic and user study were mapped to S1 and S2. Adding a dual process perspective to specific heuristic analysis increases the applicability and relevance of computerized health information design evaluations. This mapping enables designers to measure that their systems are tailored to support attention allocation. System 1 will be supported by improving pattern recognition and saliency, and system 2 through efficiency and control of information access. PMID:28269915

Adapting Nielsen's Design Heuristics to Dual Processing for Clinical Decision Support.

PubMed

Taft, Teresa; Staes, Catherine; Slager, Stacey; Weir, Charlene

2016-01-01

The study objective was to improve the applicability of Nielson's standard design heuristics for evaluating electronic health record (EHR) alerts and linked ordering support by integrating them with Dual Process theory. Through initial heuristic evaluation and a user study of 7 physicians, usability problems were identified. Through independent mapping of specific usability criteria to support for each of the Dual Cognitive processes (S1 and S2) and deliberation, agreement was reached on mapping criteria. Finally, usability errors from the heuristic and user study were mapped to S1 and S2. Adding a dual process perspective to specific heuristic analysis increases the applicability and relevance of computerized health information design evaluations. This mapping enables designers to measure that their systems are tailored to support attention allocation. System 1 will be supported by improving pattern recognition and saliency, and system 2 through efficiency and control of information access.

Usability evaluation of the SMART application for youth with mTBI.

PubMed

Dexheimer, Judith W; Kurowski, Brad G; Anders, Shilo H; McClanahan, Nicole; Wade, Shari L; Babcock, Lynn

2017-01-01

There is a dearth of evidence-based treatments available to address the significant morbidity associated with mild traumatic brain injury (mTBI). To address this gap, we designed a novel user-friendly, web-based application. We describe the preliminary evaluation of feasibility and usability of the application to promote recovery following mTBI in youth, the Self-Monitoring Activity-Restriction and Relaxation Treatment (SMART). SMART incorporates real-time recommendations for individualized symptom management and activity restriction along with training in cognitive-behavioral coping strategies. We conducted a usability evaluation to assess and modify the SMART system prior to further study and deployment. Children ages 11-18 years presenting to the emergency department were recruited after symptoms resolved. Usability was assessed using a 60-min think-aloud protocol of teens and parents describing their interaction with the application. Upon completion of the tasks, each participant also completed the system usability scale (SUS). We performed tests with 4 parent/child dyads. The average age of the children was 13 years (standard deviation=1.8). The parents were an average of 41.5 years old (standard deviation=6.2). Research revealed that the participants were enthusiastic about the interactive portions of the tool particularly the video based sessions. Parents were concerned about the speed at which their child might move through the program and the children thought that the system required large amounts of reading. Based on user feedback, researchers modified SMART to include an audio file in every module and improved the system's aesthetic properties. The mean SUS score was 85, with high SUS scores (>68) indicating satisfactory usability. High initial usability and favorable user feedback provide a foundation for further iterative development and testing of the SMART application as a tool for managing recovery from concussion. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards.

PubMed

Moumane, Karima; Idri, Ali; Abran, Alain

2016-01-01

This paper presents an empirical study based on a set of measures to evaluate the usability of mobile applications running on different mobile operating systems, including Android, iOS and Symbian. The aim is to evaluate empirically a framework that we have developed on the use of the Software Quality Standard ISO 9126 in mobile environments, especially the usability characteristic. To do that, 32 users had participated in the experiment and we have used ISO 25062 and ISO 9241 standards for objective measures by working with two widely used mobile applications: Google Apps and Google Maps. The QUIS 7.0 questionnaire have been used to collect measures assessing the users' level of satisfaction when using these two mobile applications. By analyzing the results we highlighted a set of mobile usability issues that are related to the hardware as well as to the software and that need to be taken into account by designers and developers in order to improve the usability of mobile applications.

Usability evaluation of user interface of thesis title review system

NASA Astrophysics Data System (ADS)

Tri, Y.; Erna, A.; Gellysa, U.

2018-03-01

Presentation of programs with user interface that can be accessed online through the website of course greatly provide user benefits. User can easily access the program they need. There are usability values that serve as a benchmark for the success of a user accessible program, ie efficiency, effectiveness, and convenience. These usability values also determine the development of the program for the better use. Therefore, on the review title thesis program that will be implemented in STT Dumai was measured usability evaluation. It aims to see which sides are not yet perfect and need to be improved to improve the performance and utilization of the program. Usability evaluation was measured by using smartPLS software. Database used was the result of respondent questionnaires that include questions about the experience when they used program. The result of a review of thesis title program implemented in STT Dumai has an efficiency value of 22.615, the effectiveness of 20.612, and satisfaction of 33.177.

ODMSummary: A Tool for Automatic Structured Comparison of Multiple Medical Forms Based on Semantic Annotation with the Unified Medical Language System.

PubMed

Storck, Michael; Krumm, Rainer; Dugas, Martin

2016-01-01

Medical documentation is applied in various settings including patient care and clinical research. Since procedures of medical documentation are heterogeneous and developed further, secondary use of medical data is complicated. Development of medical forms, merging of data from different sources and meta-analyses of different data sets are currently a predominantly manual process and therefore difficult and cumbersome. Available applications to automate these processes are limited. In particular, tools to compare multiple documentation forms are missing. The objective of this work is to design, implement and evaluate the new system ODMSummary for comparison of multiple forms with a high number of semantically annotated data elements and a high level of usability. System requirements are the capability to summarize and compare a set of forms, enable to estimate the documentation effort, track changes in different versions of forms and find comparable items in different forms. Forms are provided in Operational Data Model format with semantic annotations from the Unified Medical Language System. 12 medical experts were invited to participate in a 3-phase evaluation of the tool regarding usability. ODMSummary (available at https://odmtoolbox.uni-muenster.de/summary/summary.html) provides a structured overview of multiple forms and their documentation fields. This comparison enables medical experts to assess multiple forms or whole datasets for secondary use. System usability was optimized based on expert feedback. The evaluation demonstrates that feedback from domain experts is needed to identify usability issues. In conclusion, this work shows that automatic comparison of multiple forms is feasible and the results are usable for medical experts.

A Human Reliability Based Usability Evaluation Method for Safety-Critical Software

DOE Office of Scientific and Technical Information (OSTI.GOV)

Phillippe Palanque; Regina Bernhaupt; Ronald Boring

2006-04-01

Recent years have seen an increasing use of sophisticated interaction techniques including in the field of safety critical interactive software [8]. The use of such techniques has been required in order to increase the bandwidth between the users and systems and thus to help them deal efficiently with increasingly complex systems. These techniques come from research and innovation done in the field of humancomputer interaction (HCI). A significant effort is currently being undertaken by the HCI community in order to apply and extend current usability evaluation techniques to these new kinds of interaction techniques. However, very little has been donemore » to improve the reliability of software offering these kinds of interaction techniques. Even testing basic graphical user interfaces remains a challenge that has rarely been addressed in the field of software engineering [9]. However, the non reliability of interactive software can jeopardize usability evaluation by showing unexpected or undesired behaviors. The aim of this SIG is to provide a forum for both researchers and practitioners interested in testing interactive software. Our goal is to define a roadmap of activities to cross fertilize usability and reliability testing of these kinds of systems to minimize duplicate efforts in both communities.« less

Preliminary Evaluation of a Personal Healthcare System Prototype for Cognitive eRehabilitation in a Living Assistance Domain

PubMed Central

Pastorino, Matteo; Fioravanti, Alessio; Arredondo, Maria Teresa; Cogollor, José M.; Rojo, Javier; Ferre, Manuel; Bienkiewicz, Marta; Hermsdörfer, Joachim; Fringi, Evangelia; Wing, Alan M.

2014-01-01

The integration of rehabilitation systems in an ambient assisted living environment can provide a powerful and versatile tool for long-term stroke rehabilitation goals. This paper introduces a novel concept of a personalized cognitive rehabilitation system in a naturalistic setting. The proposed platform was developed within the CogWatch project, with the intent of fostering independence in activities of daily living in patients with apraxia and action disorganization syndrome. Technical usability was evaluated in a series of pilot experiments, which illustrate how this approach may help to retrain patients in activities of daily living. The first system prototype has been tested with 36 participants divided into three groups, providing an exploratory evaluation of the usability of this solution and its acceptability. The technical solutions used within the CogWatch project are targeted to meet both the end users' needs from the interaction and usability point of views and the clinical requirements associated with the use of such systems. The challenges behind the development of ambient assisted living systems for cognitive rehabilitation are discussed. PMID:24922452

A continuous usability evaluation of an electronic medication administration record application.

PubMed

Vicente Oliveros, Noelia; Gramage Caro, Teresa; Pérez Menéndez-Conde, Covadonga; Álvarez-Diaz, Ana María; Martín-Aragón Álvarez, Sagrario; Bermejo Vicedo, Teresa; Delgado Silveira, Eva

2017-12-01

The complexity of an electronic medication administration record (eMAR) has been underestimated by most designers in the past. Usability issues, such as poorly designed user application flow in eMAR, are therefore of vital importance, since they can have a negative impact on nursing activities and result in poor outcomes. The purpose of this study was to evaluate the usability of an eMAR application during its development. A usability evaluation was conducted during the development of the eMAR application. Two usability methods were used: a heuristic evaluation complemented by usability testing. Each eMAR application version provided by the vendor was evaluated by 2 hospital pharmacists, who applied the heuristic method. They reviewed the eMAR tasks, detected usability problems and their heuristic violations, and rated the severity of the usability problems. Usability testing was used to assess the final application version by observing how 3 nurses interacted with the application. Thirty-four versions were assessed before the eMAR application was considered usable. During the heuristic evaluation, the usability problems decreased from 46 unique usability problems in version 1 (V1) to 9 in version 34 (V34). In V1, usability problems were categorized into 154 heuristic violations, which decreased to 27 in V34. The average severity rating also decreased from major usability problem (2.96) to no problem (0.23). During usability testing, the 3 nurses did not encounter new usability problems. A thorough heuristic evaluation is a good method for obtaining a usable eMAR application. This evaluation points key areas for improvement and decreases usability problems and their severity. © 2017 John Wiley & Sons, Ltd.

Evaluation of a Telerehabilitation System for Community-Based Rehabilitation

PubMed Central

Schutte, Jamie; Gales, Sara; Filippone, Ashlee; Saptono, Andi; Parmanto, Bambang; McCue, Michael

2012-01-01

The use of web-based portals, while increasing in popularity in the fields of medicine and research, are rarely reported on in community-based rehabilitation programs. A program within the Pennsylvania Office of Vocational Rehabilitation’s Hiram G. Andrews Center, the Cognitive Skills Enhancement Program (CSEP), sought to enhance organization of program and participant information and communication between part- and full-time employees, supervisors and consultants. A telerehabilitation system was developed consisting of (1) a web-based portal to support a variety of clinical activities, and (2) the Versatile Integrated System for Telerehabilitation (VISYTER) video-conferencing system to support the collaboration and delivery of rehabilitation services remotely. This descriptive evaluation examines the usability of the telerehabilitation system incorporating both the portal and VISYTER. Telerehabilitation system users include CSEP staff members from three geographical locations and employed by two institutions. The IBM After-Scenario Questionnaire (ASQ) and Post-Study System Usability Questionnaire (PSSUQ), the Telehealth Usability Questionnaire (TUQ), and two demographic surveys were administered to gather both objective and subjective information. Results showed generally high levels of usability. Users commented that the telerehabilitation system improved communication, increased access to information, improved speed of completing tasks, and had an appealing interface. Areas where users would like to see improvements, including ease of accessing/editing documents and searching for information, are discussed. PMID:25945193

Graphical Weather Information System Evaluation: Usability, Perceived Utility, and Preferences from General Aviation Pilots

NASA Technical Reports Server (NTRS)

Latorella, Kara A.; Chamberlain, James P.

2002-01-01

Weather is a significant factor in General Aviation (GA) accidents and fatality rates. Graphical Weather Information Systems (GWISs) for the flight deck are appropriate technologies for mitigating the difficulties GA pilots have with current aviation weather information sources. This paper describes usability evaluations of a prototype GWIS by 12 GA pilots after using the system in flights towards convective weather. We provide design guidance for GWISs and discuss further research required to support weather situation awareness and in-flight decision making for GA pilots.

Exploring user experience and technology acceptance for a fall prevention system: results from a randomized clinical trial and a living lab.

PubMed

Vaziri, Daryoush D; Aal, Konstantin; Ogonowski, Corinna; Von Rekowski, Thomas; Kroll, Michael; Marston, Hannah R; Poveda, Rakel; Gschwind, Yves J; Delbaere, Kim; Wieching, Rainer; Wulf, Volker

2016-01-01

Falls are common in older adults and can result in serious injuries. Due to demographic changes, falls and related healthcare costs are likely to increase over the next years. Participation and motivation of older adults in fall prevention measures remain a challenge. The iStoppFalls project developed an information and communication technology (ICT)-based system for older adults to use at home in order to reduce common fall risk factors such as impaired balance and muscle weakness. The system aims at increasing older adults' motivation to participate in ICT-based fall prevention measures. This article reports on usability, user-experience and user-acceptance aspects affecting the use of the iStoppFalls system by older adults. In the course of a 16-week international multicenter study, 153 community-dwelling older adults aged 65+ participated in the iStoppFalls randomized controlled trial, of which half used the system in their home to exercise and assess their risk of falling. During the study, 60 participants completed questionnaires regarding the usability, user experience and user acceptance of the iStoppFalls system. Usability was measured with the System Usability Scale (SUS). For user experience the Physical Activity Enjoyment Scale (PACES) was applied. User acceptance was assessed with the Dynamic Acceptance Model for the Re-evaluation of Technologies (DART). To collect more detailed data on usability, user experience and user acceptance, additional qualitative interviews and observations were conducted with participants. Participants evaluated the usability of the system with an overall score of 62 (Standard Deviation, SD 15.58) out of 100, which suggests good usability. Most users enjoyed the iStoppFalls games and assessments, as shown by the overall PACES score of 31 (SD 8.03). With a score of 0.87 (SD 0.26), user acceptance results showed that participants accepted the iStoppFalls system for use in their own home. Interview data suggested that certain factors such as motivation, complexity or graphical design were different for gender and age. The results suggest that the iStoppFalls system has good usability, user experience and user acceptance. It will be important to take these along with factors such as motivation, gender and age into consideration when designing and further developing ICT-based fall prevention systems.

Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers.

PubMed

Trafton, Jodie; Martins, Susana; Michel, Martha; Lewis, Eleanor; Wang, Dan; Combs, Ann; Scates, Naquell; Tu, Samson; Goldstein, Mary K

2010-04-01

To develop and evaluate a clinical decision support system (CDSS) named Assessment and Treatment in Healthcare: Evidenced-Based Automation (ATHENA)-Opioid Therapy, which encourages safe and effective use of opioid therapy for chronic, noncancer pain. CDSS development and iterative evaluation using the analysis, design, development, implementation, and evaluation process including simulation-based and in-clinic assessments of usability for providers followed by targeted system revisions. Volunteers provided detailed feedback to guide improvements in the graphical user interface, and content and design changes to increase clinical usefulness, understandability, clinical workflow fit, and ease of completing guideline recommended practices. Revisions based on feedback increased CDSS usability ratings over time. Practice concerns outside the scope of the CDSS were also identified. Usability testing optimized the CDSS to better address barriers such as lack of provider education, confusion in dosing calculations and titration schedules, access to relevant patient information, provider discontinuity, documentation, and access to validated assessment tools. It also highlighted barriers to good clinical practice that are difficult to address with CDSS technology in its current conceptualization. For example, clinicians indicated that constraints on time and competing priorities in primary care, discomfort in patient-provider communications, and lack of evidence to guide opioid prescribing decisions impeded their ability to provide effective, guideline-adherent pain management. Iterative testing was essential for designing a highly usable and acceptable CDSS; however, identified barriers may limit the impact of the ATHENA-Opioid Therapy system and other CDSS on clinical practices and outcomes unless CDSS are paired with parallel initiatives to address these issues.

Usability of a Web-Based School Experience System: Opinions of IT Teachers and Teacher Candidates

ERIC Educational Resources Information Center

Genç, Zülfü

2015-01-01

With advances in information and communication technologies, the classical nature of educational institutions has changed. One innovative effort within teacher training is the Web-Based School Experience System (WBSES) developed by the researcher. In this study, the usability of an existing WBSES is evaluated from both teachers' (n = 13) and…

Exploring the Usability of Mobile Apps Supporting Radiologists' Training in Diagnostic Decision Making.

PubMed

Kim, Min Soon; Aro, Michael R; Lage, Kraig J; Ingalls, Kevin L; Sindhwani, Vivek; Markey, Mia K

2016-03-01

The objective of this study was to conduct a usability evaluation of mobile apps for supporting education and training in radiologic diagnostic decision-making processes. Of 381 mobile apps available at two major stores (Google Play and iTunes), eight iOS apps were selected for laboratory-based usability tests. Six staff radiologists completed eight app-specific task sets, using a think-aloud strategy. The triangular methods approach included quantitative performance measures, System Usability Scale (SUS), and qualitative thematic analysis using heuristic usability principles of usability issues. Overall, radiologists achieved higher than 70% success, with favorable SUS scores, in completing the tasks for seven of the eight apps. However, task success rate and SUS score had a weak relation (r = 0.23), indicating that the perceived usability may not reflect the holistic usability of the app. Task analysis and self-report revealed 108 usability issues, which were condensed to 55 unique issues and categorized by nine usability themes and mapped to ten usability heuristics. Nonintuitive functionality (eg, nonintuitive or misleading labels) was the most frequent theme observed, leading to inefficient navigation. These usability findings were consistent with the 13 improvements the radiologists suggested. This study demonstrates the feasibility of usability evaluation of radiology mobile apps and suggests potential improvements in the development of radiology mobile apps. This study also suggests that proficiency with mobile devices may not be equivalent to being an expert user, proficient in using the apps. Copyright © 2016 American College of Radiology. Published by Elsevier Inc. All rights reserved.

Ten factors to consider when developing usability scenarios and tasks for health information technology.

PubMed

Russ, Alissa L; Saleem, Jason J

2018-02-01

The quality of usability testing is highly dependent upon the associated usability scenarios. To promote usability testing as part of electronic health record (EHR) certification, the Office of the National Coordinator (ONC) for Health Information Technology requires that vendors test specific capabilities of EHRs with clinical end-users and report their usability testing process - including the test scenarios used - along with the results. The ONC outlines basic expectations for usability testing, but there is little guidance in usability texts or scientific literature on how to develop usability scenarios for healthcare applications. The objective of this article is to outline key factors to consider when developing usability scenarios and tasks to evaluate computer-interface based health information technologies. To achieve this goal, we draw upon a decade of our experience conducting usability tests with a variety of healthcare applications and a wide range of end-users, to include healthcare professionals as well as patients. We discuss 10 key factors that influence scenario development: objectives of usability testing; roles of end-user(s); target performance goals; evaluation time constraints; clinical focus; fidelity; scenario-related bias and confounders; embedded probes; minimize risks to end-users; and healthcare related outcome measures. For each factor, we present an illustrative example. This article is intended to aid usability researchers and practitioners in their efforts to advance health information technologies. The article provides broad guidance on usability scenario development and can be applied to a wide range of clinical information systems and applications. Published by Elsevier Inc.

Using a medical simulation center as an electronic health record usability laboratory

PubMed Central

Landman, Adam B; Redden, Lisa; Neri, Pamela; Poole, Stephen; Horsky, Jan; Raja, Ali S; Pozner, Charles N; Schiff, Gordon; Poon, Eric G

2014-01-01

Usability testing is increasingly being recognized as a way to increase the usability and safety of health information technology (HIT). Medical simulation centers can serve as testing environments for HIT usability studies. We integrated the quality assurance version of our emergency department (ED) electronic health record (EHR) into our medical simulation center and piloted a clinical care scenario in which emergency medicine resident physicians evaluated a simulated ED patient and documented electronically using the ED EHR. Meticulous planning and close collaboration with expert simulation staff was important for designing test scenarios, pilot testing, and running the sessions. Similarly, working with information systems teams was important for integration of the EHR. Electronic tools are needed to facilitate entry of fictitious clinical results while the simulation scenario is unfolding. EHRs can be successfully integrated into existing simulation centers, which may provide realistic environments for usability testing, training, and evaluation of human–computer interactions. PMID:24249778

From expert-derived user needs to user-perceived ease of use and usefulness: a two-phase mixed-methods evaluation framework.

PubMed

Boland, Mary Regina; Rusanov, Alexander; So, Yat; Lopez-Jimenez, Carlos; Busacca, Linda; Steinman, Richard C; Bakken, Suzanne; Bigger, J Thomas; Weng, Chunhua

2014-12-01

Underspecified user needs and frequent lack of a gold standard reference are typical barriers to technology evaluation. To address this problem, this paper presents a two-phase evaluation framework involving usability experts (phase 1) and end-users (phase 2). In phase 1, a cross-system functionality alignment between expert-derived user needs and system functions was performed to inform the choice of "the best available" comparison system to enable a cognitive walkthrough in phase 1 and a comparative effectiveness evaluation in phase 2. During phase 2, five quantitative and qualitative evaluation methods are mixed to assess usability: time-motion analysis, software log, questionnaires - System Usability Scale and the Unified Theory of Acceptance of Use of Technology, think-aloud protocols, and unstructured interviews. Each method contributes data for a unique measure (e.g., time motion analysis contributes task-completion-time; software log contributes action transition frequency). The measures are triangulated to yield complementary insights regarding user-perceived ease-of-use, functionality integration, anxiety during use, and workflow impact. To illustrate its use, we applied this framework in a formative evaluation of a software called Integrated Model for Patient Care and Clinical Trials (IMPACT). We conclude that this mixed-methods evaluation framework enables an integrated assessment of user needs satisfaction and user-perceived usefulness and usability of a novel design. This evaluation framework effectively bridges the gap between co-evolving user needs and technology designs during iterative prototyping and is particularly useful when it is difficult for users to articulate their needs for technology support due to the lack of a baseline. Copyright © 2013 Elsevier Inc. All rights reserved.

Evaluation of the functional performance and technical quality of an Electronic Documentation System of the Nursing Process.

PubMed

de Oliveira, Neurilene Batista; Peres, Heloisa Helena Ciqueto

2015-01-01

To evaluate the functional performance and the technical quality of the Electronic Documentation System of the Nursing Process of the Teaching Hospital of the University of São Paulo. exploratory-descriptive study. The Quality Model of regulatory standard 25010 and the Evaluation Process defined under regulatory standard 25040, both of the International Organization for Standardization/International Electrotechnical Commission. The quality characteristics evaluated were: functional suitability, reliability, usability, performance efficiency, compatibility, security, maintainability and portability. The sample was made up of 37 evaluators. in the evaluation of the specialists in information technology, only the characteristic of usability obtained a rate of positive responses of less than 70%. For the nurse lecturers, all the quality characteristics obtained a rate of positive responses of over 70%. The staff nurses of the medical and surgical clinics with experience in using the system) and staff nurses from other units of the hospital and from other health institutions (without experience in using the system) obtained rates of positive responses of more than 70% referent to the functional suitability, usability, and security. However, performance efficiency, reliability and compatibility all obtained rates below the parameter established. the software achieved rates of positive responses of over 70% for the majority of the quality characteristics evaluated.

Usability of American Nurses Association State Web Sites: A Follow-up Evaluation.

PubMed

Koch, Gina; Wakefield, Bonnie J; Alexander, Gregory L; Wilson, Melissa A; Becker, Colleen

2016-05-01

The American Nurses Association supports professional nurses through Web sites administered by state nursing associations, providing important information for current and potential members. Optimal usability of these Web sites is critical for nurses to obtain the information they seek. Heuristic evaluations are general criteria used to evaluate the usability of technology such as Web sites. A study published in 2014, using heuristic criteria from Nielsen's 10 principles and Health on The Web, evaluated 27 state nursing Web sites to identify usability concerns that could prevent nurses from obtaining accurate information regarding state nursing practice. The purpose of this study is to conduct a second heuristic evaluation to assess for changes in a subset of 12 Web sites. The analysis comparing the evaluation from 2012 to 2014 found that mean scores increased and variance decreased; however, no statistically significant difference was found between the two studies. Scores increased in 2014 for "help users to diagnose, and recover from errors," "match between the system and real world," and "consistency and standards." Scores decreased due to absence of mission statements and identification of intended audience. Ideally, Web site designers will use the feedback from this study and make changes that improve their usability to provide information to nurses.

The design and evaluation of an activity monitoring user interface for people with stroke.

PubMed

Hart, Phil; Bierwirth, Rebekah; Fulk, George; Sazonov, Edward

2014-01-01

Usability is an important topic in the field of telerehabilitation research. Older users with disabilities in particular, present age-related and disability-related challenges that should be accommodated for in the design of a user interface for a telerehabilitation system. This paper describes the design, implementation, and assessment of a telerehabilitation system user interface that tries to maximize usability for an elderly user who has experienced a stroke. An Internet-connected Nintendo(®) Wii™ gaming system is selected as a hardware platform, and a server and website are implemented to process and display the feedback information. The usability of the interface is assessed with a trial consisting of 18 subjects: 10 healthy Doctor of Physical Therapy students and 8 people with a stroke. Results show similar levels of usability and high satisfaction with the gaming system interface from both groups of subjects.

National questionnaire study on clinical ICT systems proofs: physicians suffer from poor usability.

PubMed

Viitanen, Johanna; Hyppönen, Hannele; Lääveri, Tinja; Vänskä, Jukka; Reponen, Jarmo; Winblad, Ilkka

2011-10-01

In the health informatics field, usability studies typically focus on evaluating a single information system and involve a rather small group of end-users. However, little is known about the usability of clinical information and communication technology (ICT) environment in which healthcare professionals work daily. This paper aims at contributing to usability research and user-oriented development of healthcare technologies with three objectives: inform researchers and practitioners about the current state of usability of clinical ICT systems, increase the understanding of usability aspects specific for clinical context, and encourage a more holistic approach on studying usability issues in health informatics field. A national web questionnaire study was conducted in Finland in spring 2010 with 3929 physicians actively working in patient care. For the purposes of the study, we described three dimensions of clinical ICT system usability that reflect the physicians' viewpoint on system usage: (1) compatibility between clinical ICT systems and physicians' tasks, (2) ICT support for information exchange, communication and collaboration in clinical work, and (3) interoperability and reliability. The dimensions derive from the definitions of usability and clinical context of use analysis, and reflect the ability of ICT systems to have a positive impact on patient care by supporting physicians in achieving their goals with a pleasant user experience. The research data incorporated 32 statements with a five-point Likert-scale on physicians' experiences on usability of their currently used ICT systems and a summative question about school grade given to electronic health record (EHR) systems. Physicians' estimates of their EHR systems were very critical. With the rating scale from 4 or fail to 10 or excellent, the average of the grades varied from 6.1 to 8.4 dependent on the kind of facility the physician is working. Questionnaire results indicated several usability problems and deficiencies which considerably hindered the efficiency of clinical ICT use and physician's routine work. Systems lacked the appropriate features to support typical clinical tasks, such as decision making, prevention of medical errors, and review of a patient's treatment chart. The systems also required physicians to perform fixed sequences of steps and tasks, and poorly supported the documentation and retrieval of patient data. The findings on ICT support for collaboration showed mainly negative results, aside from collaboration between co-located physicians. In addition, the study results pointed out physicians suffering from system failures and a lack of integration between the systems. The described study and related results are unique in several ways. A national usability study with nearly 4000 respondents had not been conducted in other countries in which healthcare technologies are widely adopted. The questionnaire study provided a generalized picture about the usability problems, however, it should be noted that there were significant differences between legacy systems in use. Previously, researchers had not approached contextual aspects of usability the context of clinical work, where numerous systems are in use. The described usability dimensions and the presented study results can be considered as the first step towards conceptualizing ICT usability in the unique setting of clinical work. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Usability: a critical success factor for managing change in the clinical info-structure.

PubMed

Kay, S

2005-06-01

There can be no doubt that the clinical info-structure is being significantly enriched with the deployment of new systems throughout the health sector. From a technological perspective, the initial emphasis has been mainly on functionality and only latterly on the usability of these clinical information systems. However, the large scale and rapid pace of the changes being wrought in the health sector will have a major impact on clinicians and patients, not least in how they interact with the technology. Therefore, it is not only hardware and software but people-ware, too, that needs to be actively managed; not simply a one-off functional specification but an ongoing, complex relationship. Usability is the human factor that encompasses the ethical, educational, and evaluative aspects of design. There is also a strong case for regarding usability of clinical information systems as a key critical success factor for the management of change within the health-care domain. In particular, the relationship between usability, and education and training is examined.

Designing Telemedicine Systems for Geriatric Patients: A Review of the Usability Studies.

PubMed

Narasimha, Shraddhaa; Madathil, Kapil Chalil; Agnisarman, Sruthy; Rogers, Hunter; Welch, Brandon; Ashok, Aparna; Nair, Aswathi; McElligott, James

2017-06-01

One area where telemedicine may prove to be highly effective is in providing medical care to the geriatric population, an age group predicted to account for 20% of the population in the near future. However, even though telemedicine has certain advantages, the usability of these systems with this population merits investigation. This article reviews the literature published from 2000 to 2016 with the goal of analyzing the characteristics of usability-related studies conducted using geriatric participants and the subsequent usability challenges identified. Articles were found using Web of Knowledge and PubMed citation indexing portals using the keywords (1) Telemedicine* AND Geriatrics* (2) Telemedicine* AND Usability* (3) Telemedicine* AND Usability* AND Older Adults*. A total of 297 articles were obtained from the initial search. After further detailed screening, 16 articles were selected for review based on the inclusion criteria. Of these, 60% of the studies focused on the overall usability of telemedicine systems; 6.25% focused on the usability of a telepresence robot; 12.5% compared a face-to-face medical consultation with the use of telemedicine systems, and 25% focused on the study of other aspects of telemedicine in addition to its usability. Findings reported in the studies included high patient satisfaction with telemedicine in 31.25%, whereas another 31.25% indicated a high acceptance of this method of medical consultation. Care coordination in 6.25% of the studies; confidence in telemedicine in 6.25%; trust, privacy, and reliability in 6.25%; and increased convenience when compared to personal visits in 18.75% were also reported. This review suggests limited research providing scientifically valid and reproducible usability evaluation at various stages of telemedicine system development. Telemedicine system designers need to consider the age-related issues in cognition, perception, and behavior of geriatric patients while designing telemedicine applications. Future directions for research were developed based on the limitations as well as other results found in this systematic review.

A methodology for evaluating the usability of audiovisual consumer electronic products.

PubMed

Kwahk, Jiyoung; Han, Sung H

2002-09-01

Usability evaluation is now considered an essential procedure in consumer product development. Many studies have been conducted to develop various techniques and methods of usability evaluation hoping to help the evaluators choose appropriate methods. However, planning and conducting usability evaluation requires considerations of a number of factors surrounding the evaluation process including the product, user, activity, and environmental characteristics. In this perspective, this study suggested a new methodology of usability evaluation through a simple, structured framework. The framework was outlined by three major components: the interface features of a product as design variables, the evaluation context consisting of user, product, activity, and environment as context variables, and the usability measures as dependent variables. Based on this framework, this study established methods to specify the product interface features, to define evaluation context, and to measure usability. The effectiveness of this methodology was demonstrated through case studies in which the usability of audiovisual products was evaluated by using the methods developed in this study. This study is expected to help the usability practitioners in consumer electronics industry in various ways. Most directly, it supports the evaluators' plan and conduct usability evaluation sessions in a systematic and structured manner. In addition, it can be applied to other categories of consumer products (such as appliances, automobiles, communication devices, etc.) with minor modifications as necessary.

ODMSummary: A Tool for Automatic Structured Comparison of Multiple Medical Forms Based on Semantic Annotation with the Unified Medical Language System

PubMed Central

Krumm, Rainer; Dugas, Martin

2016-01-01

Introduction Medical documentation is applied in various settings including patient care and clinical research. Since procedures of medical documentation are heterogeneous and developed further, secondary use of medical data is complicated. Development of medical forms, merging of data from different sources and meta-analyses of different data sets are currently a predominantly manual process and therefore difficult and cumbersome. Available applications to automate these processes are limited. In particular, tools to compare multiple documentation forms are missing. The objective of this work is to design, implement and evaluate the new system ODMSummary for comparison of multiple forms with a high number of semantically annotated data elements and a high level of usability. Methods System requirements are the capability to summarize and compare a set of forms, enable to estimate the documentation effort, track changes in different versions of forms and find comparable items in different forms. Forms are provided in Operational Data Model format with semantic annotations from the Unified Medical Language System. 12 medical experts were invited to participate in a 3-phase evaluation of the tool regarding usability. Results ODMSummary (available at https://odmtoolbox.uni-muenster.de/summary/summary.html) provides a structured overview of multiple forms and their documentation fields. This comparison enables medical experts to assess multiple forms or whole datasets for secondary use. System usability was optimized based on expert feedback. Discussion The evaluation demonstrates that feedback from domain experts is needed to identify usability issues. In conclusion, this work shows that automatic comparison of multiple forms is feasible and the results are usable for medical experts. PMID:27736972

User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people

PubMed Central

Vermeulen, Joan; Neyens, Jacques CL; Spreeuwenberg, Marieke D; van Rossum, Erik; Sipers, Walther; Habets, Herbert; Hewson, David J; de Witte, Luc P

2013-01-01

Purpose To involve elderly people during the development of a mobile interface of a monitoring system that provides feedback to them regarding changes in physical functioning and to test the system in a pilot study. Methods and participants The iterative user-centered development process consisted of the following phases: (1) selection of user representatives; (2) analysis of users and their context; (3) identification of user requirements; (4) development of the interface; and (5) evaluation of the interface in the lab. Subsequently, the monitoring and feedback system was tested in a pilot study by five patients who were recruited via a geriatric outpatient clinic. Participants used a bathroom scale to monitor weight and balance, and a mobile phone to monitor physical activity on a daily basis for six weeks. Personalized feedback was provided via the interface of the mobile phone. Usability was evaluated on a scale from 1 to 7 using a modified version of the Post-Study System Usability Questionnaire (PSSUQ); higher scores indicated better usability. Interviews were conducted to gain insight into the experiences of the participants with the system. Results The developed interface uses colors, emoticons, and written and/or spoken text messages to provide daily feedback regarding (changes in) weight, balance, and physical activity. The participants rated the usability of the monitoring and feedback system with a mean score of 5.2 (standard deviation 0.90) on the modified PSSUQ. The interviews revealed that most participants liked using the system and appreciated that it signaled changes in their physical functioning. However, usability was negatively influenced by a few technical errors. Conclusion Involvement of elderly users during the development process resulted in an interface with good usability. However, the technical functioning of the monitoring system needs to be optimized before it can be used to support elderly people in their self-management. PMID:24039407

Usability of a real-time tracked augmented reality display system in musculoskeletal injections

NASA Astrophysics Data System (ADS)

Baum, Zachary; Ungi, Tamas; Lasso, Andras; Fichtinger, Gabor

2017-03-01

PURPOSE: Image-guided needle interventions are seldom performed with augmented reality guidance in clinical practice due to many workspace and usability restrictions. We propose a real-time optically tracked image overlay system to make image-guided musculoskeletal injections more efficient and assess its usability in a bed-side clinical environment. METHODS: An image overlay system consisting of an optically tracked viewbox, tablet computer, and semitransparent mirror allows users to navigate scanned patient volumetric images in real-time using software built on the open-source 3D Slicer application platform. A series of experiments were conducted to evaluate the latency and screen refresh rate of the system using different image resolutions. To assess the usability of the system and software, five medical professionals were asked to navigate patient images while using the overlay and completed a questionnaire to assess the system. RESULTS: In assessing the latency of the system with scanned images of varying size, screen refresh rates were approximately 5 FPS. The study showed that participants found using the image overlay system easy, and found the table-mounted system was significantly more usable and effective than the handheld system. CONCLUSION: It was determined that the system performs comparably with scanned images of varying size when assessing the latency of the system. During our usability study, participants preferred the table-mounted system over the handheld. The participants also felt that the system itself was simple to use and understand. With these results, the image overlay system shows promise for use in a clinical environment.

The value of Retrospective and Concurrent Think Aloud in formative usability testing of a physician data query tool.

PubMed

Peute, Linda W P; de Keizer, Nicolette F; Jaspers, Monique W M

2015-06-01

To compare the performance of the Concurrent (CTA) and Retrospective (RTA) Think Aloud method and to assess their value in a formative usability evaluation of an Intensive Care Registry-physician data query tool designed to support ICU quality improvement processes. Sixteen representative intensive care physicians participated in the usability evaluation study. Subjects were allocated to either the CTA or RTA method by a matched randomized design. Each subject performed six usability-testing tasks of varying complexity in the query tool in a real-working context. Methods were compared with regard to number and type of problems detected. Verbal protocols of CTA and RTA were analyzed in depth to assess differences in verbal output. Standardized measures were applied to assess thoroughness in usability problem detection weighted per problem severity level and method overall effectiveness in detecting usability problems with regard to the time subjects spent per method. The usability evaluation of the data query tool revealed a total of 43 unique usability problems that the intensive care physicians encountered. CTA detected unique usability problems with regard to graphics/symbols, navigation issues, error messages, and the organization of information on the query tool's screens. RTA detected unique issues concerning system match with subjects' language and applied terminology. The in-depth verbal protocol analysis of CTA provided information on intensive care physicians' query design strategies. Overall, CTA performed significantly better than RTA in detecting usability problems. CTA usability problem detection effectiveness was 0.80 vs. 0.62 (p<0.05) respectively, with an average difference of 42% less time spent per subject compared to RTA. In addition, CTA was more thorough in detecting usability problems of a moderate (0.85 vs. 0.7) and severe nature (0.71 vs. 0.57). In this study, the CTA is more effective in usability-problem detection and provided clarification of intensive care physician query design strategies to inform redesign of the query tool. However, CTA does not outperform RTA. The RTA additionally elucidated unique usability problems and new user requirements. Based on the results of this study, we recommend the use of CTA in formative usability evaluation studies of health information technology. However, we recommend further research on the application of RTA in usability studies with regard to user expertise and experience when focusing on user profile customized (re)design. Copyright © 2015 Elsevier Inc. All rights reserved.

Usability: Human Research Program - Space Human Factors and Habitability

NASA Technical Reports Server (NTRS)

Sandor, Aniko; Holden, Kritina L.

2009-01-01

The Usability project addresses the need for research in the area of metrics and methodologies used in hardware and software usability testing in order to define quantifiable and verifiable usability requirements. A usability test is a human-in-the-loop evaluation where a participant works through a realistic set of representative tasks using the hardware/software under investigation. The purpose of this research is to define metrics and methodologies for measuring and verifying usability in the aerospace domain in accordance with FY09 focus on errors, consistency, and mobility/maneuverability. Usability metrics must be predictive of success with the interfaces, must be easy to obtain and/or calculate, and must meet the intent of current Human Systems Integration Requirements (HSIR). Methodologies must work within the constraints of the aerospace domain, be cost and time efficient, and be able to be applied without extensive specialized training.

Integrated Model for E-Learning Acceptance

NASA Astrophysics Data System (ADS)

Ramadiani; Rodziah, A.; Hasan, S. M.; Rusli, A.; Noraini, C.

2016-01-01

E-learning is not going to work if the system is not used in accordance with user needs. User Interface is very important to encourage using the application. Many theories had discuss about user interface usability evaluation and technology acceptance separately, actually why we do not make it correlation between interface usability evaluation and user acceptance to enhance e-learning process. Therefore, the evaluation model for e-learning interface acceptance is considered important to investigate. The aim of this study is to propose the integrated e-learning user interface acceptance evaluation model. This model was combined some theories of e-learning interface measurement such as, user learning style, usability evaluation, and the user benefit. We formulated in constructive questionnaires which were shared at 125 English Language School (ELS) students. This research statistics used Structural Equation Model using LISREL v8.80 and MANOVA analysis.

Post-deployment usability evaluation of a radiology workstation.

PubMed

Jorritsma, Wiard; Cnossen, Fokie; Dierckx, Rudi A; Oudkerk, Matthijs; Van Ooijen, Peter M A

2016-01-01

To determine the number, nature and severity of usability issues radiologists encounter while using a commercially available radiology workstation in clinical practice, and to assess how well the results of a pre-deployment usability evaluation of this workstation generalize to clinical practice. The usability evaluation consisted of semi-structured interviews and observations of twelve users using the workstation during their daily work. Usability issues and positive usability findings were documented. Each issue was given a severity rating and its root cause was determined. Results were compared to the results of a pre-deployment usability evaluation of the same workstation. Ninety-two usability issues were identified, ranging from issues that cause minor frustration or delay, to issues that cause significant delays, prevent users from completing tasks, or even pose a potential threat to patient safety. The results of the pre-deployment usability evaluation had limited generalizability to clinical practice. This study showed that radiologists encountered a large number and a wide variety of usability issues when using a commercially available radiology workstation in clinical practice. This underlines the need for effective usability engineering in radiology. Given the limitations of pre-deployment usability evaluation in radiology, which were confirmed by our finding that the results of a pre-deployment usability evaluation of this workstation had limited generalizability to clinical practice, it is vital that radiology workstation vendors devote significant resources to usability engineering efforts before deployment of their workstation, and to continue these efforts after the workstation is deployed in a hospital. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Using a fuzzy comprehensive evaluation method to determine product usability: A test case

PubMed Central

Zhou, Ronggang; Chan, Alan H. S.

2016-01-01

BACKGROUND: In order to take into account the inherent uncertainties during product usability evaluation, Zhou and Chan [1] proposed a comprehensive method of usability evaluation for products by combining the analytic hierarchy process (AHP) and fuzzy evaluation methods for synthesizing performance data and subjective response data. This method was designed to provide an integrated framework combining the inevitable vague judgments from the multiple stages of the product evaluation process. OBJECTIVE AND METHODS: In order to illustrate the effectiveness of the model, this study used a summative usability test case to assess the application and strength of the general fuzzy usability framework. To test the proposed fuzzy usability evaluation framework [1], a standard summative usability test was conducted to benchmark the overall usability of a specific network management software. Based on the test data, the fuzzy method was applied to incorporate both the usability scores and uncertainties involved in the multiple components of the evaluation. Then, with Monte Carlo simulation procedures, confidence intervals were used to compare the reliabilities among the fuzzy approach and two typical conventional methods combining metrics based on percentages. RESULTS AND CONCLUSIONS: This case study showed that the fuzzy evaluation technique can be applied successfully for combining summative usability testing data to achieve an overall usability quality for the network software evaluated. Greater differences of confidence interval widths between the method of averaging equally percentage and weighted evaluation method, including the method of weighted percentage averages, verified the strength of the fuzzy method. PMID:28035942

Using a fuzzy comprehensive evaluation method to determine product usability: A test case.

PubMed

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to take into account the inherent uncertainties during product usability evaluation, Zhou and Chan [1] proposed a comprehensive method of usability evaluation for products by combining the analytic hierarchy process (AHP) and fuzzy evaluation methods for synthesizing performance data and subjective response data. This method was designed to provide an integrated framework combining the inevitable vague judgments from the multiple stages of the product evaluation process. In order to illustrate the effectiveness of the model, this study used a summative usability test case to assess the application and strength of the general fuzzy usability framework. To test the proposed fuzzy usability evaluation framework [1], a standard summative usability test was conducted to benchmark the overall usability of a specific network management software. Based on the test data, the fuzzy method was applied to incorporate both the usability scores and uncertainties involved in the multiple components of the evaluation. Then, with Monte Carlo simulation procedures, confidence intervals were used to compare the reliabilities among the fuzzy approach and two typical conventional methods combining metrics based on percentages. This case study showed that the fuzzy evaluation technique can be applied successfully for combining summative usability testing data to achieve an overall usability quality for the network software evaluated. Greater differences of confidence interval widths between the method of averaging equally percentage and weighted evaluation method, including the method of weighted percentage averages, verified the strength of the fuzzy method.

An efficient approach to improve the usability of e-learning resources: the role of heuristic evaluation.

PubMed

Davids, Mogamat Razeen; Chikte, Usuf M E; Halperin, Mitchell L

2013-09-01

Optimizing the usability of e-learning materials is necessary to maximize their potential educational impact, but this is often neglected when time and other resources are limited, leading to the release of materials that cannot deliver the desired learning outcomes. As clinician-teachers in a resource-constrained environment, we investigated whether heuristic evaluation of our multimedia e-learning resource by a panel of experts would be an effective and efficient alternative to testing with end users. We engaged six inspectors, whose expertise included usability, e-learning, instructional design, medical informatics, and the content area of nephrology. They applied a set of commonly used heuristics to identify usability problems, assigning severity scores to each problem. The identification of serious problems was compared with problems previously found by user testing. The panel completed their evaluations within 1 wk and identified a total of 22 distinct usability problems, 11 of which were considered serious. The problems violated the heuristics of visibility of system status, user control and freedom, match with the real world, intuitive visual layout, consistency and conformity to standards, aesthetic and minimalist design, error prevention and tolerance, and help and documentation. Compared with user testing, heuristic evaluation found most, but not all, of the serious problems. Combining heuristic evaluation and user testing, with each involving a small number of participants, may be an effective and efficient way of improving the usability of e-learning materials. Heuristic evaluation should ideally be used first to identify the most obvious problems and, once these are fixed, should be followed by testing with typical end users.

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL.

PubMed

Cheung, Kei Long; Hiligsmann, Mickaël; Präger, Maximilian; Jones, Teresa; Józwiak-Hagymásy, Judit; Muñoz, Celia; Lester-George, Adam; Pokhrel, Subhash; López-Nicolás, Ángel; Trapero-Bertran, Marta; Evers, Silvia M A A; de Vries, Hein

2018-01-01

Economic decision-support tools can provide valuable information for tobacco control stakeholders, but their usability may impact the adoption of such tools. This study aims to illustrate a mixed-method usability evaluation of an economic decision-support tool for tobacco control, using the EQUIPT ROI tool prototype as a case study. A cross-sectional mixed methods design was used, including a heuristic evaluation, a thinking aloud approach, and a questionnaire testing and exploring the usability of the Return of Investment tool. A total of sixty-six users evaluated the tool (thinking aloud) and completed the questionnaire. For the heuristic evaluation, four experts evaluated the interface. In total twenty-one percent of the respondents perceived good usability. A total of 118 usability problems were identified, from which twenty-six problems were categorized as most severe, indicating high priority to fix them before implementation. Combining user-based and expert-based evaluation methods is recommended as these were shown to identify unique usability problems. The evaluation provides input to optimize usability of a decision-support tool, and may serve as a vantage point for other developers to conduct usability evaluations to refine similar tools before wide-scale implementation. Such studies could reduce implementation gaps by optimizing usability, enhancing in turn the research impact of such interventions.

Low-Cost Rapid Usability Testing: Its Application in Both Product Development and System Implementation.

PubMed

Kushniruk, Andre; Borycki, Elizabeth

2017-01-01

In recent years there has been considerable discussion around the need for certification and regulation of healthcare information technology (IT). In particular, the usability of the products being developed needs to be evaluated. This has included the application of standards designed to ensure the process of system development is user-centered and takes usability into consideration while a product is being developed. In addition to this, in healthcare, organizations in the United States and Europe have also addressed the need and requirement for product certification. However, despite these efforts there are continued reports of unusable and unsafe implementations. In this paper we discuss the need to not only include (and require) usability testing in the one-time development process of health IT products (such as EHRs), but we also argue for the need to additionally develop specific usability standards and requirements for usability testing during the implementation of vendor products (i.e. post product development) in healthcare settings. It is further argued that health IT products that may have been certified regarding their development process will still require application of usability testing in the process of implementing them in real hospital settings in order to ensure usability and safety. This is needed in order to ensure that the final result of both product development and implementation processes take into account and apply the latest usability principles and methods.

Development of the Telehealth Usability Questionnaire (TUQ).

PubMed

Parmanto, Bambang; Lewis, Allen Nelson; Graham, Kristin M; Bertolet, Marnie H

2016-01-01

Current telehealth usability questionnaires are designed primarily for older technologies, where telehealth interaction is conducted over dedicated videoconferencing applications. However, telehealth services are increasingly conducted over computer-based systems that rely on commercial software and a user supplied computer interface. Therefore, a usability questionnaire that addresses the changes in telehealth service delivery and technology is needed. The Telehealth Usability Questionnaire (TUQ) was developed to evaluate the usability of telehealth implementation and services. This paper addresses: (1) the need for a new measure of telehealth usability, (2) the development of the TUQ, (3) intended uses for the TUQ, and (4) the reliability of the TUQ. Analyses indicate that the TUQ is a solid, robust, and versatile measure that can be used to measure the quality of the computer-based user interface and the quality of the telehealth interaction and services.

From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies. Contribution of the IMIA Human Factors Engineering for Healthcare Informatics Working Group.

PubMed

Kushniruk, A; Nohr, C; Jensen, S; Borycki, E M

2013-01-01

The objective of this paper is to explore human factors approaches to understanding the use of health information technology (HIT) by extending usability engineering approaches to include analysis of the impact of clinical context through use of clinical simulations. Methods discussed are considered on a continuum from traditional laboratory-based usability testing to clinical simulations. Clinical simulations can be conducted in a simulation laboratory and they can also be conducted in real-world settings. The clinical simulation approach attempts to bring the dimension of clinical context into stronger focus. This involves testing of systems with representative users doing representative tasks, in representative settings/environments. Application of methods where realistic clinical scenarios are used to drive the study of users interacting with systems under realistic conditions and settings can lead to identification of problems and issues with systems that may not be detected using traditional usability engineering methods. In conducting such studies, careful consideration is needed in creating ecologically valid test scenarios. The evidence obtained from such evaluation can be used to improve both the usability and safety of HIT. In addition, recent work has shown that clinical simulations, in particular those conducted in-situ, can lead to considerable benefits when compared to the costs of running such studies. In order to bring context of use into the testing of HIT, clinical simulation, involving observing representative users carrying out tasks in representative settings, holds considerable promise.

Example of a Human Factors Engineering approach to a medication administration work system: potential impact on patient safety.

PubMed

Beuscart-Zéphir, Marie-Catherine; Pelayo, Sylvia; Bernonville, Stéphanie

2010-04-01

The objectives of this paper are: In this approach, the implementation of such a complex IT solution is considered a major redesign of the work system. The paper describes the Human Factor (HF) tasks embedded in the project lifecycle: (1) analysis and modelling of the current work system and usability assessment of the medication CPOE solution; (2) HF recommendations for work re-design and usability recommendations for IT system re-engineering both aiming at a safer and more efficient work situation. Standard ethnographic methods were used to support the analysis of the current work system and work situations, coupled with cognitive task analysis methods and documents review. Usability inspection (heuristic evaluation) and both in-lab (simulated tasks) and on-site (real tasks) usability tests were performed for the evaluation of the CPOE candidate. Adapted software engineering models were used in combination with usual textual descriptions, tasks models and mock-ups to support the recommendations for work and product re-design. The analysis of the work situations identified different work organisations and procedures across the hospital's departments. The most important differences concerned the doctor-nurse communications and cooperation modes and the procedures for preparing and administering the medications. The assessment of the medication CPOE functions uncovered a number of usability problems including severe ones leading to impossible to detect or to catch errors. Models of the actual and possible distribution of tasks and roles were used to support decision making in the work design process. The results of the usability assessment were translated into requirements to support the necessary re-engineering of the IT application. The HFE approach to medication CPOE efficiently identifies and distinguishes currently unsafe or uncomfortable work situations that could obviously benefit from an IT solution from other work situations incorporating efficient work procedures that might be impaired by the implementation of the CPOE. In this context, a careful redesign of the work situation and of the entire work system is necessary to actually benefit from the installation of the product in terms of patient safety and human performances. In parallel, a usability assessment of the product to be implemented is mandatory to identify potentially dangerous usability flaws and to fix them before the installation. (c) 2009 Elsevier Ireland Ltd. All rights reserved.

Usability evaluation techniques in mobile commerce applications: A systematic review

NASA Astrophysics Data System (ADS)

Hussain, Azham; Mkpojiogu, Emmanuel O. C.

2016-08-01

Obviously, there are a number of literatures concerning the usability of mobile commerce (m-commerce) applications and related areas, but they do not adequately provide knowledge about usability techniques used in most of the empirical usability evaluation for m-commerce application. Therefore, this paper is aimed at producing the usability techniques frequently used in the aspect of usability evaluation for m-commerce applications. To achieve the stated objective, systematic literature review was employed. Sixty seven papers were downloaded in usability evaluation for m-commerce and related areas; twenty one most relevant studies were selected for review in order to extract the appropriate information. The results from the review shows that heuristic evaluation, formal test and think aloud methods are the most commonly used methods in m-commerce application in comparison to cognitive walkthrough and the informal test methods. Moreover, most of the studies applied control experiment (33.3% of the total studies); other studies that applied case study for usability evaluation are 14.28%. The results from this paper provide additional knowledge to the usability practitioners and research community for the current state and use of usability techniques in m-commerce application.

Usability Testing of a National Substance Use Screening Tool Embedded in Electronic Health Records.

PubMed

Press, Anne; DeStio, Catherine; McCullagh, Lauren; Kapoor, Sandeep; Morley, Jeanne; Conigliaro, Joseph

2016-07-08

Screening, brief intervention, and referral to treatment (SBIRT) is currently being implemented into health systems nationally via paper and electronic methods. The purpose of this study was to evaluate the integration of an electronic SBIRT tool into an existing paper-based SBIRT clinical workflow in a patient-centered medical home. Usability testing was conducted in an academic ambulatory clinic. Two rounds of usability testing were done with medical office assistants (MOAs) using a paper and electronic version of the SBIRT tool, with two and four participants, respectively. Qualitative and quantitative data was analyzed to determine the impact of both tools on clinical workflow. A second round of usability testing was done with the revised electronic version and compared with the first version. Personal workflow barriers cited in the first round of testing were that the electronic health record (EHR) tool was disruptive to patient's visits. In Round 2 of testing, MOAs reported favoring the electronic version due to improved layout and the inclusion of an alert system embedded in the EHR. For example, using the system usability scale (SUS), MOAs reported a grade "1" for the statement, "I would like to use this system frequently" during the first round of testing but a "5" during the second round of analysis. The importance of testing usability of various mediums of tools used in health care screening is highlighted by the findings of this study. In the first round of testing, the electronic tool was reported as less user friendly, being difficult to navigate, and time consuming. Many issues faced in the first generation of the tool were improved in the second generation after usability was evaluated. This study demonstrates how usability testing of an electronic SBRIT tool can help to identify challenges that can impact clinical workflow. However, a limitation of this study was the small sample size of MOAs that participated. The results may have been biased to Northwell Health workers' perceptions of the SBIRT tool and their specific clinical workflow.

Measurement properties and usability of non-contact scanners for measuring transtibial residual limb volume.

PubMed

Kofman, Rianne; Beekman, Anna M; Emmelot, Cornelis H; Geertzen, Jan H B; Dijkstra, Pieter U

2018-06-01

Non-contact scanners may have potential for measurement of residual limb volume. Different non-contact scanners have been introduced during the last decades. Reliability and usability (practicality and user friendliness) should be assessed before introducing these systems in clinical practice. The aim of this study was to analyze the measurement properties and usability of four non-contact scanners (TT Design, Omega Scanner, BioSculptor Bioscanner, and Rodin4D Scanner). Quasi experimental. Nine (geometric and residual limb) models were measured on two occasions, each consisting of two sessions, thus in total 4 sessions. In each session, four observers used the four systems for volume measurement. Mean for each model, repeatability coefficients for each system, variance components, and their two-way interactions of measurement conditions were calculated. User satisfaction was evaluated with the Post-Study System Usability Questionnaire. Systematic differences between the systems were found in volume measurements. Most of the variances were explained by the model (97%), while error variance was 3%. Measurement system and the interaction between system and model explained 44% of the error variance. Repeatability coefficient of the systems ranged from 0.101 (Omega Scanner) to 0.131 L (Rodin4D). Differences in Post-Study System Usability Questionnaire scores between the systems were small and not significant. The systems were reliable in determining residual limb volume. Measurement systems and the interaction between system and residual limb model explained most of the error variances. The differences in repeatability coefficient and usability between the four CAD/CAM systems were small. Clinical relevance If accurate measurements of residual limb volume are required (in case of research), modern non-contact scanners should be taken in consideration nowadays.

A Usability Study of Users’ Perceptions Toward a Multimedia Computer-Assisted Learning Tool for Neuroanatomy

PubMed Central

Gould, Douglas J.; Terrell, Mark A.; Fleming, Jo

2015-01-01

This usability study evaluated users’ perceptions of a multimedia prototype for a new e-learning tool: Anatomy of the Central Nervous System: A Multimedia Course. Usability testing is a collection of formative evaluation methods that inform the developmental design of e-learning tools to maximize user acceptance, satisfaction, and adoption. Sixty-two study participants piloted the prototype and completed a usability questionnaire designed to measure two usability properties: program need and program applicability. Statistical analyses were used to test the hypothesis that the multimedia prototype was well designed and highly usable, it was perceived as: 1) highly needed across a spectrum of educational contexts, 2) highly applicable in supporting the pedagogical processes of teaching and learning neuroanatomy, and 3) was highly usable by all types of users. Three independent variables represented user differences: level of expertise (faculty vs. student), age, and gender. Analysis of the results supports the research hypotheses that the prototype was designed well for different types of users in various educational contexts and for supporting the pedagogy of neuroanatomy. In addition, the results suggest that the multimedia program will be most useful as a neuroanatomy review tool for health-professions students preparing for licensing or board exams. This study demonstrates the importance of integrating quality properties of usability with principles of human learning during the instructional design process for multimedia products. PMID:19177405

Ten Demands of Improved Usability in eHealth and Some Progress - Co-Creation by Health and Social Care Professionals.

PubMed

Scandurra, Isabella; Liljequist, David

2016-01-01

Current healthcare organizations often do not accomplish the intended effects of their eHealth systems due to inadequate usability. Commissioned by the Swedish Ministry of Health and Social Affairs, the usability of current eHealth systems in Swedish health and social care has been analysed from the perspective of their professionals. The objective of the study was to report on current problems, potential solutions as well as to relate these to research in relevant areas. Using a participatory approach, nine workshops were held where health informatics researchers guided staff from different care organizations, representatives of the national associations of health and social care professionals and the national eHealth system vendor organization. This paper presents ten demands that Swedish health and social care professionals find imperative to prioritize. The study emphasizes that development of eHealth systems must be integrated into the care practice improvement process and iteratively evaluated regarding usability.

Usability evaluation of in-housed developed ERP system

NASA Astrophysics Data System (ADS)

Faisal, Chaudhry Muhammad Nadeem; Shakeel Faridi, Muhammad; Javed, Zahid

2011-10-01

Enterprise Resource Planning systems are the combination of different business IS (Information System) applications that are designed according to the organization requirements. Generally ERP systems are suffering from complex user interface issues. Recent research shows that there is a need for improvement concerning, the user interface from their perspectives. In order to design the software applications that are easy to use, memorize and apply to new problems, we must know the users philosophy and something about learning, reminiscence and problems solving. The Usability engineering is the only way to study the deeds of users while interacting with ERP (Enterprise Resource & Planning). This paper will focus on the users' experiences view of financial module in ERP system. The HCI research method, explicitly survey questionnaire method was adopted to gather users understanding in order to evaluate the selected modules for in-housed ERP system. The study involved group of users from two industries, the results can not be generalized as a whole. The study was first time successfully applied Usability evaluation on in-housed ERP in local industry (Masood Textile Mills, Interloop Ltd) in Pakistan. The results may hopefully opened-up an area of research and methodology that could provide considerable further benefits to Industry in developments of Industrial information systems.

Usability of a patient education and motivation tool using heuristic evaluation.

PubMed

Joshi, Ashish; Arora, Mohit; Dai, Liwei; Price, Kathleen; Vizer, Lisa; Sears, Andrew

2009-11-06

Computer-mediated educational applications can provide a self-paced, interactive environment to deliver educational content to individuals about their health condition. These programs have been used to deliver health-related information about a variety of topics, including breast cancer screening, asthma management, and injury prevention. We have designed the Patient Education and Motivation Tool (PEMT), an interactive computer-based educational program based on behavioral, cognitive, and humanistic learning theories. The tool is designed to educate users and has three key components: screening, learning, and evaluation. The objective of this tutorial is to illustrate a heuristic evaluation using a computer-based patient education program (PEMT) as a case study. The aims were to improve the usability of PEMT through heuristic evaluation of the interface; to report the results of these usability evaluations; to make changes based on the findings of the usability experts; and to describe the benefits and limitations of applying usability evaluations to PEMT. PEMT was evaluated by three usability experts using Nielsen's usability heuristics while reviewing the interface to produce a list of heuristic violations with severity ratings. The violations were sorted by heuristic and ordered from most to least severe within each heuristic. A total of 127 violations were identified with a median severity of 3 (range 0 to 4 with 0 = no problem to 4 = catastrophic problem). Results showed 13 violations for visibility (median severity = 2), 38 violations for match between system and real world (median severity = 2), 6 violations for user control and freedom (median severity = 3), 34 violations for consistency and standards (median severity = 2), 11 violations for error severity (median severity = 3), 1 violation for recognition and control (median severity = 3), 7 violations for flexibility and efficiency (median severity = 2), 9 violations for aesthetic and minimalist design (median severity = 2), 4 violations for help users recognize, diagnose, and recover from errors (median severity = 3), and 4 violations for help and documentation (median severity = 4). We describe the heuristic evaluation method employed to assess the usability of PEMT, a method which uncovers heuristic violations in the interface design in a quick and efficient manner. Bringing together usability experts and health professionals to evaluate a computer-mediated patient education program can help to identify problems in a timely manner. This makes this method particularly well suited to the iterative design process when developing other computer-mediated health education programs. Heuristic evaluations provided a means to assess the user interface of PEMT.

Usability of a Patient Education and Motivation Tool Using Heuristic Evaluation

PubMed Central

Arora, Mohit; Dai, Liwei; Price, Kathleen; Vizer, Lisa; Sears, Andrew

2009-01-01

Background Computer-mediated educational applications can provide a self-paced, interactive environment to deliver educational content to individuals about their health condition. These programs have been used to deliver health-related information about a variety of topics, including breast cancer screening, asthma management, and injury prevention. We have designed the Patient Education and Motivation Tool (PEMT), an interactive computer-based educational program based on behavioral, cognitive, and humanistic learning theories. The tool is designed to educate users and has three key components: screening, learning, and evaluation. Objective The objective of this tutorial is to illustrate a heuristic evaluation using a computer-based patient education program (PEMT) as a case study. The aims were to improve the usability of PEMT through heuristic evaluation of the interface; to report the results of these usability evaluations; to make changes based on the findings of the usability experts; and to describe the benefits and limitations of applying usability evaluations to PEMT. Methods PEMT was evaluated by three usability experts using Nielsen’s usability heuristics while reviewing the interface to produce a list of heuristic violations with severity ratings. The violations were sorted by heuristic and ordered from most to least severe within each heuristic. Results A total of 127 violations were identified with a median severity of 3 (range 0 to 4 with 0 = no problem to 4 = catastrophic problem). Results showed 13 violations for visibility (median severity = 2), 38 violations for match between system and real world (median severity = 2), 6 violations for user control and freedom (median severity = 3), 34 violations for consistency and standards (median severity = 2), 11 violations for error severity (median severity = 3), 1 violation for recognition and control (median severity = 3), 7 violations for flexibility and efficiency (median severity = 2), 9 violations for aesthetic and minimalist design (median severity = 2), 4 violations for help users recognize, diagnose, and recover from errors (median severity = 3), and 4 violations for help and documentation (median severity = 4). Conclusion We describe the heuristic evaluation method employed to assess the usability of PEMT, a method which uncovers heuristic violations in the interface design in a quick and efficient manner. Bringing together usability experts and health professionals to evaluate a computer-mediated patient education program can help to identify problems in a timely manner. This makes this method particularly well suited to the iterative design process when developing other computer-mediated health education programs. Heuristic evaluations provided a means to assess the user interface of PEMT. PMID:19897458

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework.

PubMed

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process.

Empirical studies on usability of mHealth apps: a systematic literature review.

PubMed

Zapata, Belén Cruz; Fernández-Alemán, José Luis; Idri, Ali; Toval, Ambrosio

2015-02-01

The release of smartphones and tablets, which offer more advanced communication and computing capabilities, has led to the strong emergence of mHealth on the market. mHealth systems are being used to improve patients' lives and their health, in addition to facilitating communication between doctors and patients. Researchers are now proposing mHealth applications for many health conditions such as dementia, autism, dysarthria, Parkinson's disease, and so on. Usability becomes a key factor in the adoption of these applications, which are often used by people who have problems when using mobile devices and who have a limited experience of technology. The aim of this paper is to investigate the empirical usability evaluation processes described in a total of 22 selected studies related to mHealth applications by means of a Systematic Literature Review. Our results show that the empirical evaluation methods employed as regards usability could be improved by the adoption of automated mechanisms. The evaluation processes should also be revised to combine more than one method. This paper will help researchers and developers to create more usable applications. Our study demonstrates the importance of adapting health applications to users' need.

Usability and accessibility in consumer health informatics current trends and future challenges.

PubMed

Goldberg, Larry; Lide, Bettijoyce; Lowry, Svetlana; Massett, Holly A; O'Connell, Trisha; Preece, Jennifer; Quesenbery, Whitney; Shneiderman, Ben

2011-05-01

It is a truism that, for innovative eHealth systems to have true value and impact, they must first and foremost be usable and accessible by clinicians, consumers, and other stakeholders. In this paper, current trends and future challenges in the usability and accessibility of consumer health informatics will be described. Consumer expectations of their healthcare providers and healthcare records in this new era of consumer-directed care will be explored, and innovative visualizations, assistive technologies, and other ways that healthcare information is currently being provided and/or shared will be described. Challenges for ensuring the usability of current and future systems will also be discussed. An innovative model for conducting systematic, timely, user-centered research on consumer-facing websites at the National Cancer Institute (NCI) and the ongoing efforts at the National Institute of Standards and Technology (NIST) to promote health information technology (HIT) usability standards and evaluation criteria will also be presented. Copyright © 2011 American Journal of Preventive Medicine. All rights reserved.

Evaluating Nurses' Satisfaction With Two Nursing Information Systems.

PubMed

Khajouei, Reza; Abbasi, Reza

2017-06-01

Evaluating user satisfaction is one of the methods to ensure the usability of information systems. Considering the importance of nursing information systems in patient health, the objective of this study is to evaluate nurses' satisfaction with two widely used nursing information systems (Peyvand Dadeh and Tirazhe) in Iran. This descriptive-analytical study was done on 230 nurses in all teaching hospitals of Kerman University of Medical Sciences in 2015. Data were collected using an augmented version of a questionnaire developed by IBM. Data were analyzed by SPSS.16 using descriptive and analytical statistical methods including t test, analysis of variance, and Pearson correlation coefficient. The mean of overall satisfaction with the two systems was 61 ± 2.2 and 74 ± 2.4, respectively. The mean of satisfaction with different systems dimensions, that is, ease of use, information quality, and interface quality, was, respectively, 24 ± 1.9, 26 ± 9.7, and 12 ± 4.7 for Tirazhe and 29 ± 1.1, 39 ± 1.04 and 13 ± 5.3 for Peyvand Dadeh system. Nurses' satisfaction with both systems was at a medium level. The majority of nurses were relatively satisfied with the information quality and user interface quality of these systems. The results suggest that designing nursing information systems in accordance with their users' need improves usability. Hence, policy and decision makers of healthcare institutions should invest on usability when purchasing such systems.

A personal health information toolkit for health intervention research.

PubMed

Kizakevich, Paul N; Eckhoff, Randall; Weger, Stacey; Weeks, Adam; Brown, Janice; Bryant, Stephanie; Bakalov, Vesselina; Zhang, Yuying; Lyden, Jennifer; Spira, James

2014-01-01

With the emergence of mobile health (mHealth) apps, there is a growing demand for better tools for developing and evaluating mobile health interventions. Recently we developed the Personal Health Intervention Toolkit (PHIT), a software framework which eases app implementation and facilitates scientific evaluation. PHIT integrates self-report and physiological sensor instruments, evidence-based advisor logic, and self-help interventions such as meditation, health education, and cognitive behavior change. PHIT can be used to facilitate research, interventions for chronic diseases, risky behaviors, sleep, medication adherence, environmental monitoring, momentary data collection health screening, and clinical decision support. In a series of usability evaluations, participants reported an overall usability score of 4.5 on a 1-5 Likert scale and an 85 score on the System Usability Scale, indicating a high percentile rank of 95%.

Effect of improving the usability of an e-learning resource: a randomized trial.

PubMed

Davids, Mogamat Razeen; Chikte, Usuf M E; Halperin, Mitchell L

2014-06-01

Optimizing the usability of e-learning materials is necessary to reduce extraneous cognitive load and maximize their potential educational impact. However, this is often neglected, especially when time and other resources are limited. We conducted a randomized trial to investigate whether a usability evaluation of our multimedia e-learning resource, followed by fixing of all problems identified, would translate into improvements in usability parameters and learning by medical residents. Two iterations of our e-learning resource [version 1 (V1) and version 2 (V2)] were compared. V1 was the first fully functional version and V2 was the revised version after all identified usability problems were addressed. Residents in internal medicine and anesthesiology were randomly assigned to one of the versions. Usability was evaluated by having participants complete a user satisfaction questionnaire and by recording and analyzing their interactions with the application. The effect on learning was assessed by questions designed to test the retention and transfer of knowledge. Participants reported high levels of satisfaction with both versions, with good ratings on the System Usability Scale and adjective rating scale. In contrast, analysis of video recordings revealed significant differences in the occurrence of serious usability problems between the two versions, in particular in the interactive HandsOn case with its treatment simulation, where there was a median of five serious problem instances (range: 0-50) recorded per participant for V1 and zero instances (range: 0-1) for V2 (P < 0.001). There were no differences in tests of retention or transfer of knowledge between the two versions. In conclusion, usability evaluation followed by a redesign of our e-learning resource resulted in significant improvements in usability. This is likely to translate into improved motivation and willingness to engage with the learning material. In this population of relatively high-knowledge participants, learning scores were similar across the two versions. Copyright © 2014 The American Physiological Society.

Effect of improving the usability of an e-learning resource: a randomized trial

PubMed Central

Chikte, Usuf M. E.; Halperin, Mitchell L.

2014-01-01

Optimizing the usability of e-learning materials is necessary to reduce extraneous cognitive load and maximize their potential educational impact. However, this is often neglected, especially when time and other resources are limited. We conducted a randomized trial to investigate whether a usability evaluation of our multimedia e-learning resource, followed by fixing of all problems identified, would translate into improvements in usability parameters and learning by medical residents. Two iterations of our e-learning resource [version 1 (V1) and version 2 (V2)] were compared. V1 was the first fully functional version and V2 was the revised version after all identified usability problems were addressed. Residents in internal medicine and anesthesiology were randomly assigned to one of the versions. Usability was evaluated by having participants complete a user satisfaction questionnaire and by recording and analyzing their interactions with the application. The effect on learning was assessed by questions designed to test the retention and transfer of knowledge. Participants reported high levels of satisfaction with both versions, with good ratings on the System Usability Scale and adjective rating scale. In contrast, analysis of video recordings revealed significant differences in the occurrence of serious usability problems between the two versions, in particular in the interactive HandsOn case with its treatment simulation, where there was a median of five serious problem instances (range: 0–50) recorded per participant for V1 and zero instances (range: 0–1) for V2 (P < 0.001). There were no differences in tests of retention or transfer of knowledge between the two versions. In conclusion, usability evaluation followed by a redesign of our e-learning resource resulted in significant improvements in usability. This is likely to translate into improved motivation and willingness to engage with the learning material. In this population of relatively high-knowledge participants, learning scores were similar across the two versions. PMID:24913451

Proposed Framework for the Evaluation of Standalone Corpora Processing Systems: An Application to Arabic Corpora

PubMed Central

Al-Thubaity, Abdulmohsen; Alqifari, Reem

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework. PMID:25610910

Proposed framework for the evaluation of standalone corpora processing systems: an application to Arabic corpora.

PubMed

Al-Thubaity, Abdulmohsen; Al-Khalifa, Hend; Alqifari, Reem; Almazrua, Manal

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework.

Iterative User Interface Design for Automated Sequential Organ Failure Assessment Score Calculator in Sepsis Detection

PubMed Central

Herasevich, Vitaly

2017-01-01

Background The new sepsis definition has increased the need for frequent sequential organ failure assessment (SOFA) score recalculation and the clerical burden of information retrieval makes this score ideal for automated calculation. Objective The aim of this study was to (1) estimate the clerical workload of manual SOFA score calculation through a time-motion analysis and (2) describe a user-centered design process for an electronic medical record (EMR) integrated, automated SOFA score calculator with subsequent usability evaluation study. Methods First, we performed a time-motion analysis by recording time-to-task-completion for the manual calculation of 35 baseline and 35 current SOFA scores by 14 internal medicine residents over a 2-month period. Next, we used an agile development process to create a user interface for a previously developed automated SOFA score calculator. The final user interface usability was evaluated by clinician end users with the Computer Systems Usability Questionnaire. Results The overall mean (standard deviation, SD) time-to-complete manual SOFA score calculation time was 61.6 s (33). Among the 24% (12/50) usability survey respondents, our user-centered user interface design process resulted in >75% favorability of survey items in the domains of system usability, information quality, and interface quality. Conclusions Early stakeholder engagement in our agile design process resulted in a user interface for an automated SOFA score calculator that reduced clinician workload and met clinicians’ needs at the point of care. Emerging interoperable platforms may facilitate dissemination of similarly useful clinical score calculators and decision support algorithms as “apps.” A user-centered design process and usability evaluation should be considered during creation of these tools. PMID:28526675

Iterative User Interface Design for Automated Sequential Organ Failure Assessment Score Calculator in Sepsis Detection.

PubMed

Aakre, Christopher Ansel; Kitson, Jaben E; Li, Man; Herasevich, Vitaly

2017-05-18

The new sepsis definition has increased the need for frequent sequential organ failure assessment (SOFA) score recalculation and the clerical burden of information retrieval makes this score ideal for automated calculation. The aim of this study was to (1) estimate the clerical workload of manual SOFA score calculation through a time-motion analysis and (2) describe a user-centered design process for an electronic medical record (EMR) integrated, automated SOFA score calculator with subsequent usability evaluation study. First, we performed a time-motion analysis by recording time-to-task-completion for the manual calculation of 35 baseline and 35 current SOFA scores by 14 internal medicine residents over a 2-month period. Next, we used an agile development process to create a user interface for a previously developed automated SOFA score calculator. The final user interface usability was evaluated by clinician end users with the Computer Systems Usability Questionnaire. The overall mean (standard deviation, SD) time-to-complete manual SOFA score calculation time was 61.6 s (33). Among the 24% (12/50) usability survey respondents, our user-centered user interface design process resulted in >75% favorability of survey items in the domains of system usability, information quality, and interface quality. Early stakeholder engagement in our agile design process resulted in a user interface for an automated SOFA score calculator that reduced clinician workload and met clinicians' needs at the point of care. Emerging interoperable platforms may facilitate dissemination of similarly useful clinical score calculators and decision support algorithms as "apps." A user-centered design process and usability evaluation should be considered during creation of these tools. ©Christopher Ansel Aakre, Jaben E Kitson, Man Li, Vitaly Herasevich. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 18.05.2017.

Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces.

PubMed Central

Kushniruk, A. W.; Patel, V. L.; Cimino, J. J.

1997-01-01

This paper describes an approach to the evaluation of health care information technologies based on usability engineering and a methodological framework from the study of medical cognition. The approach involves collection of a rich set of data including video recording of health care workers as they interact with systems, such as computerized patient records and decision support tools. The methodology can be applied in the laboratory setting, typically involving subjects "thinking aloud" as they interact with a system. A similar approach to data collection and analysis can also be extended to study of computer systems in the "live" environment of hospital clinics. Our approach is also influenced from work in the area of cognitive task analysis, which aims to characterize the decision making and reasoning of subjects of varied levels of expertise as they interact with information technology in carrying out representative tasks. The stages involved in conducting cognitively-based usability analyses are detailed and the application of such analysis in the iterative process of system and interface development is discussed. PMID:9357620

Older Adults' Acceptance of Activity Trackers

PubMed Central

Preusse, Kimberly C.; Mitzner, Tracy L.; Fausset, Cara Bailey; Rogers, Wendy A.

2016-01-01

Objective To assess the usability and acceptance of activity tracking technologies by older adults. Method First in our multi-method approach, we conducted heuristic evaluations of two activity trackers that revealed potential usability barriers to acceptance. Next, questionnaires and interviews were administered to 16 older adults (Mage=70, SDage=3.09, rangeage= 65-75) before and after a 28-day field study to understand facilitators and additional barriers to acceptance. These measurements were supplemented with diary and usage data and assessed if and why users overcame usability issues. Results The heuristic evaluation revealed usability barriers in System Status Visibility; Error Prevention; and Consistency and Standards. The field study revealed additional barriers (e.g., accuracy, format), and acceptance-facilitators (e.g., goal-tracking, usefulness, encouragement). Discussion The acceptance of wellness management technologies, such as activity trackers, may be increased by addressing acceptance-barriers during deployment (e.g., providing tutorials on features that were challenging, communicating usefulness). PMID:26753803

Comparing two anesthesia information management system user interfaces: a usability evaluation.

PubMed

Wanderer, Jonathan P; Rao, Anoop V; Rothwell, Sarah H; Ehrenfeld, Jesse M

2012-11-01

Anesthesia information management systems (AIMS) have been developed by multiple vendors and are deployed in thousands of operating rooms around the world, yet not much is known about measuring and improving AIMS usability. We developed a methodology for evaluating AIMS usability in a low-fidelity simulated clinical environment and used it to compare an existing user interface with a revised version. We hypothesized that the revised user interface would be more useable. In a low-fidelity simulated clinical environment, twenty anesthesia providers documented essential anesthetic information for the start of the case using both an existing and a revised user interface. Participants had not used the revised user interface previously and completed a brief training exercise prior to the study task. All participants completed a workload assessment and a satisfaction survey. All sessions were recorded. Multiple usability metrics were measured. The primary outcome was documentation accuracy. Secondary outcomes were perceived workload, number of documentation steps, number of user interactions, and documentation time. The interfaces were compared and design problems were identified by analyzing recorded sessions and survey results. Use of the revised user interface was shown to improve documentation accuracy from 85.1% to 92.4%, a difference of 7.3% (95% confidence interval [CI] for the difference 1.8 to 12.7). The revised user interface decreased the number of user interactions by 6.5 for intravenous documentation (95% CI 2.9 to 10.1) and by 16.1 for airway documentation (95% CI 11.1 to 21.1). The revised user interface required 3.8 fewer documentation steps (95% CI 2.3 to 5.4). Airway documentation time was reduced by 30.5 seconds with the revised workflow (95% CI 8.5 to 52.4). There were no significant time differences noted in intravenous documentation or in total task time. No difference in perceived workload was found between the user interfaces. Two user interface design problems were identified in the revised user interface. The usability of anesthesia information management systems can be evaluated using a low-fidelity simulated clinical environment. User testing of the revised user interface showed improvement in some usability metrics and highlighted areas for further revision. Vendors of AIMS and those who use them should consider adopting methods to evaluate and improve AIMS usability.

Usability Evaluation of An Electronic Medication Administration Record (eMAR) Application

PubMed Central

Guo, J.; Iribarren, S.; Kapsandoy, S.; Perri, S.; Staggers, N.

2011-01-01

Background Electronic medication administration records (eMARs) have been widely used in recent years. However, formal usability evaluations are not yet available for these vendor applications, especially from the perspective of nurses, the largest group of eMAR users. Objective To conduct a formal usability evaluation of an implemented eMAR. Methods Four evaluators examined a commercial vendor eMAR using heuristic evaluation techniques. The evaluators defined seven tasks typical of eMAR use and independently evaluated the application. Consensus techniques were used to obtain 100% agreement of identified usability problems and severity ratings. Findings were reviewed with 5 clinical staff nurses and the Director of Clinical Informatics who verified findings with a small group of clinical nurses. Results Evaluators found 60 usability problems categorized into 233 heuristic violations. Match, Error, and Visibility heuristics were the most frequently violated. Administer Medication and Order and Modify Medications tasks had the highest number of heuristic violations and usability problems rated as major or catastrophic. Conclusion The high number of usability problems could impact the effectiveness, efficiency and satisfaction of nurses’ medication administration activities and may include concerns about patient safety. Usability is a joint responsibility between sites and vendors. We offer a call to action for usability evaluations at all sites and eMAR application redesign as necessary to improve the user experience and promote patient safety. PMID:23616871

Mediating the Cognitive Walkthrough with Patient Groups to achieve Personalized Health in Chronic Disease Self-Management System Evaluation.

PubMed

Georgsson, Mattias; Kushniruk, Andre

2016-01-01

The cognitive walkthrough (CW) is a task-based, expert inspection usability evaluation method involving benefits such as cost effectiveness and efficiency. A drawback of the method is that it doesn't involve the user perspective from real users but instead is based on experts' predictions about the usability of the system and how users interact. In this paper, we propose a way of involving the user in an expert evaluation method by modifying the CW with patient groups as mediators. This along with other modifications include a dual domain session facilitator, specific patient groups and three different phases: 1) a preparation phase where suitable tasks are developed by a panel of experts and patients, validated through the content validity index 2) a patient user evaluation phase including an individual and collaborative process part 3) an analysis and coding phase where all data is digitalized and synthesized making use of Qualitative Data Analysis Software (QDAS) to determine usability deficiencies. We predict that this way of evaluating will utilize the benefits of the expert methods, also providing a way of including the patient user of these self-management systems. Results from this prospective study should provide evidence of the usefulness of this method modification.

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework

PubMed Central

Zhou, Ronggang; Chan, Alan H. S.

2016-01-01

BACKGROUND: In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. OBJECTIVE: This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. METHODS: With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. RESULTS AND CONCLUSIONS: Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process. PMID:28035943

Usability Evaluation of a Private Social Network on Mental Health for Relatives.

PubMed

Toribio-Guzmán, José Miguel; García-Holgado, Alicia; Soto Pérez, Felipe; García-Peñalvo, Francisco J; Franco Martín, Manuel

2017-09-01

Usability is one of the most prominent criteria that must be fulfilled by a software product. This study aims to evaluate the usability of SocialNet, a private social network for monitoring the daily progress of patients by their relatives, using a mixed usability approach: heuristic evaluation conducted by experts and user testing. A double heuristic evaluation with one expert evaluator identified the issues related to consistency, design, and privacy. User testing was conducted on 20 users and one evaluator using observation techniques and questionnaires. The main usability problems were found to be related to the structure of SocialNet, and the users presented some difficulties in locating the buttons or links. The results show a high level of usability and satisfaction with the product. This evaluation provides data on the usability of SocialNet based on the difficulties experienced by the users and the expert. The results help in redesigning the tool to resolve the identified problems as part of an iterative process.

Usability Evaluation Methods for Gesture-Based Games: A Systematic Review.

PubMed

Simor, Fernando Winckler; Brum, Manoela Rogofski; Schmidt, Jaison Dairon Ebertz; Rieder, Rafael; De Marchi, Ana Carolina Bertoletti

2016-10-04

Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user's age and limitations. Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology and a test protocol may offer the user more comfort, welfare, and confidence.

Usability Evaluation Methods for Gesture-Based Games: A Systematic Review

PubMed Central

Simor, Fernando Winckler; Brum, Manoela Rogofski; Schmidt, Jaison Dairon Ebertz; De Marchi, Ana Carolina Bertoletti

2016-01-01

Background Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. Objective This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. Methods The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. Results In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user’s age and limitations. Conclusions Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology and a test protocol may offer the user more comfort, welfare, and confidence. PMID:27702737

Hybrid P300-based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task.

PubMed

Riccio, Angela; Holz, Elisa Mira; Aricò, Pietro; Leotta, Francesco; Aloise, Fabio; Desideri, Lorenzo; Rimondini, Matteo; Kübler, Andrea; Mattia, Donatella; Cincotti, Febo

2015-03-01

To evaluate the impact of a hybrid control on usability of a P300-based brain-computer interface (BCI) system that was designed to control an assistive technology software and was integrated with an electromyographic channel for error correction. Proof-of-principle study with a convenience sample. Neurologic rehabilitation hospital. Participants (N=11) in this pilot study included healthy (n=8) and severely motor impaired (n=3) persons. The 3 people with severe motor disability were identified as potential candidates to benefit from the proposed hybrid BCI system for communication and environmental interaction. To eventually investigate the improvement in usability, we compared 2 modalities of BCI system control: a P300-based and a hybrid P300 electromyographic-based mode of control. System usability was evaluated according to the following outcome measures within 3 domains: (1) effectiveness (overall system accuracy and P300-based BCI accuracy); (2) efficiency (throughput time and users' workload); and (3) satisfaction (users' satisfaction). We also considered the information transfer rate and time for selection. Findings obtained in healthy participants were in favor of a higher usability of the hybrid control as compared with the nonhybrid. A similar trend was indicated by the observational results gathered from each of the 3 potential end-users. The proposed hybrid BCI control modality could provide end-users with severe motor disability with an option to exploit some residual muscular activity, which could not be fully reliable for properly controlling an assistive technology device. The findings reported in this pilot study encourage the implementation of a clinical trial involving a large cohort of end-users. Copyright © 2015 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study.

PubMed

Schnall, Rebecca; Cho, Hwayoung; Liu, Jianfang

2018-01-05

Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. ©Rebecca Schnall, Hwayoung Cho, Jianfang Liu. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 05.01.2018.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study

PubMed Central

Cho, Hwayoung; Liu, Jianfang

2018-01-01

Background Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. Objective The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. Methods A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. Results The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. Conclusions The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. PMID:29305343

Human Factors for Nursing: From In-Situ Testing to Mobile Usability Engineering.

PubMed

Kushniruk, Andre W; Borycki, Elizabeth M; Solvoll, Terje; Hullin, Carola

2016-01-01

The tutorial goal is to familiarize participants with human aspects of health informatics and human-centered approaches to the design, evaluation and deployment of both usable and safe healthcare information systems. The focus will be on demonstrating and teaching practical and low-cost methods for evaluating mobile applications in nursing. Basic background to testing methods will be provided, followed by live demonstration of the methods. Then the audience will break into small groups to explore the application of the methods to applications of interest (there will be a number of possible applications that will be available for applications in areas such as electronic health records and decision support, however, if the groups have applications of specific interest to them that will be possible). The challenges of conducting usability testing, and in particular mobile usability testing will be discussed along with practical solutions. The target audience includes practicing nurses and nurse researchers, nursing informatics specialists, nursing students, nursing managers and health informatics professionals interested in improving the usability and safety of healthcare applications.

A human reliability based usability evaluation method for safety-critical software

DOE Office of Scientific and Technical Information (OSTI.GOV)

Boring, R. L.; Tran, T. Q.; Gertman, D. I.

2006-07-01

Boring and Gertman (2005) introduced a novel method that augments heuristic usability evaluation methods with that of the human reliability analysis method of SPAR-H. By assigning probabilistic modifiers to individual heuristics, it is possible to arrive at the usability error probability (UEP). Although this UEP is not a literal probability of error, it nonetheless provides a quantitative basis to heuristic evaluation. This method allows one to seamlessly prioritize and identify usability issues (i.e., a higher UEP requires more immediate fixes). However, the original version of this method required the usability evaluator to assign priority weights to the final UEP, thusmore » allowing the priority of a usability issue to differ among usability evaluators. The purpose of this paper is to explore an alternative approach to standardize the priority weighting of the UEP in an effort to improve the method's reliability. (authors)« less

Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record.

PubMed

Hodgson, Tobias; Magrabi, Farah; Coiera, Enrico

2018-05-01

To conduct a usability study exploring the value of using speech recognition (SR) for clinical documentation tasks within an electronic health record (EHR) system. Thirty-five emergency department clinicians completed a system usability scale (SUS) questionnaire. The study was undertaken after participants undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR. SUS scores were analyzed and the results with KBM were compared to SR results. Significant difference in SUS scores between EHR system use with and without SR were observed (KBM 67, SR 61; P = 0.045; CI, 0.1 to 12.0). Nineteen of 35 participants scored higher for EHR with KBM, 11 higher for EHR with SR and 5 gave the same score for both. Factor analysis showed no significant difference in scores for the sub-element of usability (EHR with KBM 65, EHR with SR 62; P = 0.255; CI, -2.6 to 9.5). Scores for the sub-element of learnability were significantly different (KBM 72, SR 55; P < 0.001; CI, 9.8 to 23.5). A significant correlation was found between the perceived usability of the two system configurations (EHR with KBM or SR) and the efficiency of documentation (time to document) (P = 0.002; CI, 10.5 to -0.1) but not with safety (number of errors) (P = 0.90; CI, -2.3 to 2.6). SR was associated with significantly reduced overall usability scores, even though it is often positioned as ease of use technology. SR was perceived to impose larger costs in terms of learnability via training and support requirements for EHR based documentation when compared to using KBM. Lower usability scores were significantly associated with longer documentation times. The usability of EHR systems with any input modality is an area that requires continued development. The addition of an SR component to an EHR system may cause a significant reduction in terms of perceived usability by clinicians. Copyright © 2018 Elsevier B.V. All rights reserved.

Usability Evaluation of an Unstructured Clinical Document Query Tool for Researchers.

PubMed

Hultman, Gretchen; McEwan, Reed; Pakhomov, Serguei; Lindemann, Elizabeth; Skube, Steven; Melton, Genevieve B

2018-01-01

Natural Language Processing - Patient Information Extraction for Researchers (NLP-PIER) was developed for clinical researchers for self-service Natural Language Processing (NLP) queries with clinical notes. This study was to conduct a user-centered analysis with clinical researchers to gain insight into NLP-PIER's usability and to gain an understanding of the needs of clinical researchers when using an application for searching clinical notes. Clinical researcher participants (n=11) completed tasks using the system's two existing search interfaces and completed a set of surveys and an exit interview. Quantitative data including time on task, task completion rate, and survey responses were collected. Interviews were analyzed qualitatively. Survey scores, time on task and task completion proportions varied widely. Qualitative analysis indicated that participants found the system to be useful and usable in specific projects. This study identified several usability challenges and our findings will guide the improvement of NLP-PIER 's interfaces.

Knowledge management impact of information technology Web 2.0/3.0. The case study of agent software technology usability in knowledge management system

NASA Astrophysics Data System (ADS)

Sołtysik-Piorunkiewicz, Anna

2015-02-01

How we can measure the impact of internet technology Web 2.0/3.0 for knowledge management? How we can use the Web 2.0/3.0 technologies for generating, evaluating, sharing, organizing knowledge in knowledge-based organization? How we can evaluate it from user-centered perspective? Article aims to provide a method for evaluate the usability of web technologies to support knowledge management in knowledge-based organizations of the various stages of the cycle knowledge management, taking into account: generating knowledge, evaluating knowledge, sharing knowledge, etc. for the modern Internet technologies based on the example of agent technologies. The method focuses on five areas of evaluation: GUI, functional structure, the way of content publication, organizational aspect, technological aspect. The method is based on the proposed indicators relating respectively to assess specific areas of evaluation, taking into account the individual characteristics of the scoring. Each of the features identified in the evaluation is judged first point wise, then this score is subject to verification and clarification by means of appropriate indicators of a given feature. The article proposes appropriate indicators to measure the impact of Web 2.0/3.0 technologies for knowledge management and verification them in an example of agent technology usability in knowledge management system.

Advancing Usability Evaluation through Human Reliability Analysis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ronald L. Boring; David I. Gertman

2005-07-01

This paper introduces a novel augmentation to the current heuristic usability evaluation methodology. The SPAR-H human reliability analysis method was developed for categorizing human performance in nuclear power plants. Despite the specialized use of SPAR-H for safety critical scenarios, the method also holds promise for use in commercial off-the-shelf software usability evaluations. The SPAR-H method shares task analysis underpinnings with human-computer interaction, and it can be easily adapted to incorporate usability heuristics as performance shaping factors. By assigning probabilistic modifiers to heuristics, it is possible to arrive at the usability error probability (UEP). This UEP is not a literal probabilitymore » of error but nonetheless provides a quantitative basis to heuristic evaluation. When combined with a consequence matrix for usability errors, this method affords ready prioritization of usability issues.« less

Evaluating the Usability of Pinchigator, a system for Navigating Virtual Worlds using Pinch Gloves

NASA Technical Reports Server (NTRS)

Hamilton, George S.; Brookman, Stephen; Dumas, Joseph D. II; Tilghman, Neal

2003-01-01

Appropriate design of two dimensional user interfaces (2D U/I) utilizing the well known WIMP (Window, Icon, Menu, Pointing device) environment for computer software is well studied and guidance can be found in several standards. Three-dimensional U/I design is not nearly so mature as 2D U/I, and standards bodies have not reached consensus on what makes a usable interface. This is especially true when the tools for interacting with the virtual environment may include stereo viewing, real time trackers and pinch gloves instead of just a mouse & keyboard. Over the last several years the authors have created a 3D U/I system dubbed Pinchigator for navigating virtual worlds based on the dVise dV/Mockup visualization software, Fakespace Pinch Gloves and Pohlemus trackers. The current work is to test the usability of the system on several virtual worlds, suggest improvements to increase Pinchigator s usability, and then to generalize about what was learned and how those lessons might be applied to improve other 3D U/I systems.

Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency.

PubMed

Fritz, Fleur; Balhorn, Sebastian; Riek, Markus; Breil, Bernhard; Dugas, Martin

2012-05-01

The objective of this evaluation study is to assess a web-based application, currently available on iPad, to document questionnaires regarding patient reported outcomes such as quality of life. Based on the single source approach, the results of these questionnaires are available in the electronic health record to be used for treatment and research purposes. The assessment focuses on the usability and efficiency of the system. The system usability scale questionnaire with seven additional items was used to rate the usability by the patients. It was formally validated by a Cronbach Alpha test. In addition, semi-structured interviews were conducted with patients and medical staff. Time and cost measures, based on official tables of costs, were taken through workflow observations. This study was conducted in the department of dermatology at the University Hospital of Münster, Germany from April to June 2011. Using the web-based application questionnaire, results about patient reported outcomes like quality of life are immediately available in the electronic health record and can be used for treatment or research purposes. 118 patients and four staff members participated in the study. The usability score reached 80 from 100 points and patients as well as medical staff stated in the interviews that the usability of the web-based system was high, and they preferred it to the previously used paper-based questionnaires. In the setting of our pilot department the mobile devices amortized their costs after 6.7 months. In general, depending on the professional group who are going to post process the paper-based forms, the earliest break-even point to use mobile questionnaires is at 1737 paper sheets per year. The mobile patient questionnaires, integrated into the electronic health record, were well accepted in our pilot setting with high usability scores from patients and medical staff alike. The system has also proved to be cost-efficient compared to the paper-based workflow, given that a certain number of questionnaires is used per year. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Multi-method laboratory user evaluation of an actionable clinical performance information system: Implications for usability and patient safety.

PubMed

Brown, Benjamin; Balatsoukas, Panos; Williams, Richard; Sperrin, Matthew; Buchan, Iain

2018-01-01

Electronic audit and feedback (e-A&F) systems are used worldwide for care quality improvement. They measure health professionals' performance against clinical guidelines, and some systems suggest improvement actions. However, little is known about optimal interface designs for e-A&F, in particular how to present suggested actions for improvement. We developed a novel theory-informed system for primary care (the Performance Improvement plaN GeneratoR; PINGR) that covers the four principal interface components: clinical performance summaries; patient lists; detailed patient-level information; and suggested actions. As far as we are aware, this is the first report of an e-A&F system with all four interface components. (1) Use a combination of quantitative and qualitative methods to evaluate the usability of PINGR with target end-users; (2) refine existing design recommendations for e-A&F systems; (3) determine the implications of these recommendations for patient safety. We recruited seven primary care physicians to perform seven tasks with PINGR, during which we measured on-screen behaviour and eye movements. Participants subsequently completed usability questionnaires, and were interviewed in-depth. Data were integrated to: gain a more complete understanding of usability issues; enhance and explain each other's findings; and triangulate results to increase validity. Participants committed a median of 10 errors (range 8-21) when using PINGR's interface, and completed a median of five out of seven tasks (range 4-7). Errors violated six usability heuristics: clear response options; perceptual grouping and data relationships; representational formats; unambiguous description; visually distinct screens for confusable items; and workflow integration. Eye movement analysis revealed the integration of components largely supported effective user workflow, although the modular design of clinical performance summaries unnecessarily increased cognitive load. Interviews and questionnaires revealed PINGR is user-friendly, and that improved information prioritisation could further promote useful user action. Comparing our results with the wider usability literature we refine a previously published set of interface design recommendations for e-A&F. The implications for patient safety are significant regarding: user engagement; actionability; and information prioritisation. Our results also support adopting multi-method approaches in usability studies to maximise issue discovery and the credibility of findings. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.

Increasing EHR system usability through standards: Conformance criteria in the HL7 EHR-system functional model.

PubMed

Meehan, Rebecca A; Mon, Donald T; Kelly, Kandace M; Rocca, Mitra; Dickinson, Gary; Ritter, John; Johnson, Constance M

2016-10-01

Though substantial work has been done on the usability of health information technology, improvements in electronic health record system (EHR) usability have been slow, creating frustration, distrust of EHRs and the use of potentially unsafe work-arounds. Usability standards could be part of the solution for improving EHR usability. EHR system functional requirements and standards have been used successfully in the past to specify system behavior, the criteria of which have been gradually implemented in EHR systems through certification programs and other national health IT strategies. Similarly, functional requirements and standards for usability can help address the multitude of sequelae associated with poor usability. This paper describes the evidence-based functional requirements for usability contained in the Health Level Seven (HL7) EHR System Functional Model, and the benefits of open and voluntary EHR system usability standards. Copyright © 2016 Elsevier Inc. All rights reserved.

Towards a Usability and Error "Safety Net": A Multi-Phased Multi-Method Approach to Ensuring System Usability and Safety.

PubMed

Kushniruk, Andre; Senathirajah, Yalini; Borycki, Elizabeth

2017-01-01

The usability and safety of health information systems have become major issues in the design and implementation of useful healthcare IT. In this paper we describe a multi-phased multi-method approach to integrating usability engineering methods into system testing to ensure both usability and safety of healthcare IT upon widespread deployment. The approach involves usability testing followed by clinical simulation (conducted in-situ) and "near-live" recording of user interactions with systems. At key stages in this process, usability problems are identified and rectified forming a usability and technology-induced error "safety net" that catches different types of usability and safety problems prior to releasing systems widely in healthcare settings.

Seeing the System through the End Users' Eyes: Shadow Expert Technique for Evaluating the Consistency of a Learning Management System

NASA Astrophysics Data System (ADS)

Holzinger, Andreas; Stickel, Christian; Fassold, Markus; Ebner, Martin

Interface consistency is an important basic concept in web design and has an effect on performance and satisfaction of end users. Consistency also has significant effects on the learning performance of both expert and novice end users. Consequently, the evaluation of consistency within a e-learning system and the ensuing eradication of irritating discrepancies in the user interface redesign is a big issue. In this paper, we report of our experiences with the Shadow Expert Technique (SET) during the evaluation of the consistency of the user interface of a large university learning management system. The main objective of this new usability evaluation method is to understand the interaction processes of end users with a specific system interface. Two teams of usability experts worked independently from each other in order to maximize the objectivity of the results. The outcome of this SET method is a list of recommended changes to improve the user interaction processes, hence to facilitate high consistency.

Integrating Visual Mnemonics and Input Feedback With Passphrases to Improve the Usability and Security of Digital Authentication.

PubMed

Juang, Kevin; Greenstein, Joel

2018-04-01

We developed a new authentication system based on passphrases instead of passwords. Our new system incorporates a user-generated mnemonic picture displayed during login, definition tooltips, error correction to reduce typographical errors, a decoy-based input masking technique, and random passphrase generation using either a specialized wordlist or a sentence template. Passphrases exhibit a greater level of security than traditional passwords, but their wider adoption has been hindered by human factors issues. Our assertion is that the added features of our system work particularly well with passphrases and help address these shortcomings. We conducted a study to evaluate our new system with a customized 1,450-word list and our new system with a 6-word sentence structure against the control conditions of a user-created passphrase of at least 24 characters and a system-generated passphrase using a 10,326-word list. Fifty participants completed two sessions so that we could measure the usability and security of the authentication schemes. With the new system conditions, memorability was improved, and security was equivalent to or better than the control conditions. Usability and overall ratings also favored the new system conditions over the control conditions. Our research presents a new authentication system using innovative techniques that improve on the usability and security of existing password and passphrase authentication systems. In computer security, drastic changes should never happen overnight, but we recommend that our contributions be incorporated into current authentication systems to help facilitate a transition from passwords to usable passphrases.

Usability Evaluation of Public Web Mapping Sites

NASA Astrophysics Data System (ADS)

Wang, C.

2014-04-01

Web mapping sites are interactive maps that are accessed via Webpages. With the rapid development of Internet and Geographic Information System (GIS) field, public web mapping sites are not foreign to people. Nowadays, people use these web mapping sites for various reasons, in that increasing maps and related map services of web mapping sites are freely available for end users. Thus, increased users of web mapping sites led to more usability studies. Usability Engineering (UE), for instance, is an approach for analyzing and improving the usability of websites through examining and evaluating an interface. In this research, UE method was employed to explore usability problems of four public web mapping sites, analyze the problems quantitatively and provide guidelines for future design based on the test results. Firstly, the development progress for usability studies were described, and simultaneously several usability evaluation methods such as Usability Engineering (UE), User-Centered Design (UCD) and Human-Computer Interaction (HCI) were generally introduced. Then the method and procedure of experiments for the usability test were presented in detail. In this usability evaluation experiment, four public web mapping sites (Google Maps, Bing maps, Mapquest, Yahoo Maps) were chosen as the testing websites. And 42 people, who having different GIS skills (test users or experts), gender (male or female), age and nationality, participated in this test to complete the several test tasks in different teams. The test comprised three parts: a pretest background information questionnaire, several test tasks for quantitative statistics and progress analysis, and a posttest questionnaire. The pretest and posttest questionnaires focused on gaining the verbal explanation of their actions qualitatively. And the design for test tasks targeted at gathering quantitative data for the errors and problems of the websites. Then, the results mainly from the test part were analyzed. The success rate from different public web mapping sites was calculated and compared, and displayed by the means of diagram. And the answers from questionnaires were also classified and organized in this part. Moreover, based on the analysis, this paper expands the discussion about the layout, map visualization, map tools, search logic and etc. Finally, this paper closed with some valuable guidelines and suggestions for the design of public web mapping sites. Also, limitations for this research stated in the end.

[Exploring nurse, usage effectiveness of mobile nursing station].

PubMed

Chang, Fang-Mei; Lee, Ting-Ting

2013-04-01

A mobile nursing station is an innovative cart that integrates a wireless network, information technology devices, and online charts. In addition to improving clinical work and workflow efficiencies, data is integrated among different information systems and hardware devices to promote patient safety. This study investigated the effectiveness of mobile nursing cart use. We compared different distributions of nursing activity working samples to evaluate the nursing information systems in terms of interface usability and usage outcomes. There were two parts of this study. Part one used work sampling to collect nursing activity data necessary to compare a unit that used a mobile nursing cart (mobile group, n = 18) with another that did not (traditional group, n = 14). Part two applied a nursing information system interface usability questionnaire to survey the mobile unit with nurses who had used a mobile nursing station (including those who had worked in this unit as floating nurses) (n = 30) in order to explore interface usability and effectiveness. We found that using the mobile nursing station information system increased time spent on direct patient care and decreased time spent on indirect patient care and documentation. Results further indicated that participants rated interface usability as high and evaluated usage effectiveness positively. Comments made in the open-ended question section raised several points of concern, including problems / inadequacies related to hardware devices, Internet speed, and printing. This study indicates that using mobile nursing station can improve nursing activity distributions and that nurses hold generally positive attitudes toward mobile nursing station interface usability and usage effectiveness. The authors thus encourage the continued implementation of mobile nursing stations and related studies to further enhance clinical nursing care.

Usability and Culture as Two of the Value Criteria for Evaluating the Artifact

NASA Astrophysics Data System (ADS)

Kurosu, Masaaki

In this paper, the conceptual framework of the Artifact Development Analysis (ADA) and its relationship to the usability engineering are outlined. The ADA analyses the significance of all artifacts including hardware, software, humanware and system. Its viewpoint extends both in temporal and spatial dimensions. In short, it deals with the diversity of the artifact and casts the questions "why it is so" and "why it is not so". In this respect, the ADA is related to the usability engineering as one of the value attitudes. The usability engineering puts emphasis on effectiveness and efficiency. The usability is not always the value criterion of highest importance and some people sometimes put more emphasis on other criteria such as the aesthetic aspect, the cost, etc. Based on the findings of ADA, we should focus on the extent where the usability can provide the core satisfaction and we should also summarize the guideline on how the artifact should be designed.

The McMaster Optimal Aging Portal: Usability Evaluation of a Unique Evidence-Based Health Information Website.

PubMed

Barbara, Angela M; Dobbins, Maureen; Haynes, R Brian; Iorio, Alfonso; Lavis, John N; Raina, Parminder; Levinson, Anthony J

2016-05-11

Increasingly, older adults and their informal caregivers are using the Internet to search for health-related information. There is a proliferation of health information online, but the quality of this information varies, often based on exaggerated or dramatic findings, and not easily comprehended by consumers. The McMaster Optimal Aging Portal (Portal) was developed to provide Internet users with high-quality evidence about aging and address some of these current limitations of health information posted online. The Portal includes content for health professionals coming from three best-in-class resources (MacPLUS, Health Evidence, and Health Systems Evidence) and four types of content specifically prepared for the general public (Evidence Summaries, Web Resource Ratings, Blog Posts, and Twitter messages). Our objectives were to share the findings of the usability evaluation of the Portal with particular focus on the content features for the general public and to inform designers of health information websites and online resources for older adults about key usability themes. Data analysis included task performance during usability testing and qualitative content analyses of both the usability sessions and interviews to identify core themes. A total of 37 participants took part in 33 usability testing sessions and 21 focused interviews. Qualitative analysis revealed common themes regarding the Portal's strengths and challenges to usability. The strengths of the website were related to credibility, applicability, browsing function, design, and accessibility. The usability challenges included reluctance to register, process of registering, searching, terminology, and technical features. The study reinforced the importance of including end users during the development of this unique, dynamic, evidence-based health information website. The feedback was applied to iteratively improve website usability. Our findings can be applied by designers of health-related websites.

The McMaster Optimal Aging Portal: Usability Evaluation of a Unique Evidence-Based Health Information Website

PubMed Central

Dobbins, Maureen; Haynes, R. Brian; Iorio, Alfonso; Lavis, John N; Raina, Parminder

2016-01-01

Background Increasingly, older adults and their informal caregivers are using the Internet to search for health-related information. There is a proliferation of health information online, but the quality of this information varies, often based on exaggerated or dramatic findings, and not easily comprehended by consumers. The McMaster Optimal Aging Portal (Portal) was developed to provide Internet users with high-quality evidence about aging and address some of these current limitations of health information posted online. The Portal includes content for health professionals coming from three best-in-class resources (MacPLUS, Health Evidence, and Health Systems Evidence) and four types of content specifically prepared for the general public (Evidence Summaries, Web Resource Ratings, Blog Posts, and Twitter messages). Objective Our objectives were to share the findings of the usability evaluation of the Portal with particular focus on the content features for the general public and to inform designers of health information websites and online resources for older adults about key usability themes. Methods Data analysis included task performance during usability testing and qualitative content analyses of both the usability sessions and interviews to identify core themes. Results A total of 37 participants took part in 33 usability testing sessions and 21 focused interviews. Qualitative analysis revealed common themes regarding the Portal’s strengths and challenges to usability. The strengths of the website were related to credibility, applicability, browsing function, design, and accessibility. The usability challenges included reluctance to register, process of registering, searching, terminology, and technical features. Conclusions The study reinforced the importance of including end users during the development of this unique, dynamic, evidence-based health information website. The feedback was applied to iteratively improve website usability. Our findings can be applied by designers of health-related websites. PMID:27170443

A Model for Usability Evaluation for the Development and Implementation of Consumer eHealth Interventions.

PubMed

Parry, David; Carter, Philip; Koziol-McLain, Jane; Feather, Jacqueline

2015-01-01

Consumer eHealth products are often used by people in their own homes or other settings without dedicated clinical supervision, and often with minimal training and limited support--much as eCommerce and eGovernment applications are currently deployed. Internet based self-care systems have been advocated for over a decade as a way to reduce costs and allow more convenient care, and--because of the expectation that they will be used to reduced health cost--, by increasing self-care and avoiding hospitalization. However, the history of consumer eHealth interventions is mixed, with many unsuccessful implementations. Many consumer eHealth products will form part of a broader complex intervention, with many possible benefits and effects on both individuals and society. This poster describes a model of consumer eHealth assessment based on multiple methods of usability evaluation at different stages in the design and fielding of eHealth systems. We argue that different methods of usability evaluation are able to give valuable insights into the likely effects of an intervention in a way that is congruent with software development processes.

Usability Evaluation of Multimedia Courseware (MEL-SindD)

NASA Astrophysics Data System (ADS)

Yussof, Rahmah Lob; Badioze Zaman, Halimah

Constructive evaluations on any software are needed to ensure the effectiveness and usability of the software. This assesment on the multimedia courseware is part of the researcher's study towards the development and usability of the early reading software for students with Down Syndrome (MEL-SindD). This paper will discuss the usability assesment of this courseware, the methods used for the evaluation as well as suitable approaches that can be deployed to evaluate the courseware effectiveness to disabled children.

Design and Usability Evaluation of Social Mobile Diabetes Management System in the Gulf Region.

PubMed

Alanzi, Turki; Istepanian, Robert; Philip, Nada

2016-09-26

The prevalence of diabetes in the Gulf States is one of the highest globally. It is estimated that 20% of the population in the region has been diagnosed with diabetes and according to the International Diabetes Federation (IDF), five of the IDF's "top 10" countries for diabetes prevalence in 2011 and projected for 2030 are in this region. In recent years, there have been an increasing number of clinical studies advocating the use of mobile phone technology for diabetes self-management with improved clinical outcomes. However, there are few studies to date addressing the application of mobile diabetes management in the Gulf region, particularly in the Kingdom of Saudi Arabia (KSA), where there is exponential increase in mobile phone usage and access to social networking. The objective of this paper is to present the design and development of a new mobile health system for social behavioral change and management tailored for Saudi patients with diabetes called Saudi Arabia Networking for Aiding Diabetes (SANAD). A usability study for the SANAD system is presented to validate the acceptability of using mobile technologies among patients with diabetes in the KSA and the Gulf region. The SANAD system was developed using mobile phone technology with diabetes management and social networking modules. For the usability study the Questionnaire for User Interaction Satisfaction was used to evaluate the usability aspect of the SANAD system. A total of 33 users with type 2 diabetes participated in the study. The key modules of the SANAD system consist of (1) a mobile diabetes management module; (2) a social networking module; and (3) a cognitive behavioral therapy module for behavioral change issues. The preliminary results of the usability study indicated general acceptance of the patients in using the system with higher usability rating in patients with type 2 diabetes. We found that the acceptability of the system was high among Saudi patients with diabetes, and ongoing work in this research area is underway to conduct a clinical pilot study in the KSA for patients with type 2 diabetes. The wide deployment of such a system is timely and required in the Gulf region due to the wide use of mobile phones and social networking mediums.

Design and Usability Evaluation of Social Mobile Diabetes Management System in the Gulf Region

PubMed Central

2016-01-01

Background The prevalence of diabetes in the Gulf States is one of the highest globally. It is estimated that 20% of the population in the region has been diagnosed with diabetes and according to the International Diabetes Federation (IDF), five of the IDF’s “top 10” countries for diabetes prevalence in 2011 and projected for 2030 are in this region. In recent years, there have been an increasing number of clinical studies advocating the use of mobile phone technology for diabetes self-management with improved clinical outcomes. However, there are few studies to date addressing the application of mobile diabetes management in the Gulf region, particularly in the Kingdom of Saudi Arabia (KSA), where there is exponential increase in mobile phone usage and access to social networking. Objective The objective of this paper is to present the design and development of a new mobile health system for social behavioral change and management tailored for Saudi patients with diabetes called Saudi Arabia Networking for Aiding Diabetes (SANAD). A usability study for the SANAD system is presented to validate the acceptability of using mobile technologies among patients with diabetes in the KSA and the Gulf region. Methods The SANAD system was developed using mobile phone technology with diabetes management and social networking modules. For the usability study the Questionnaire for User Interaction Satisfaction was used to evaluate the usability aspect of the SANAD system. A total of 33 users with type 2 diabetes participated in the study. Results The key modules of the SANAD system consist of (1) a mobile diabetes management module; (2) a social networking module; and (3) a cognitive behavioral therapy module for behavioral change issues. The preliminary results of the usability study indicated general acceptance of the patients in using the system with higher usability rating in patients with type 2 diabetes. Conclusions We found that the acceptability of the system was high among Saudi patients with diabetes, and ongoing work in this research area is underway to conduct a clinical pilot study in the KSA for patients with type 2 diabetes. The wide deployment of such a system is timely and required in the Gulf region due to the wide use of mobile phones and social networking mediums. PMID:27670696

Usability flaws of medication-related alerting functions: A systematic qualitative review.

PubMed

Marcilly, Romaric; Ammenwerth, Elske; Vasseur, Francis; Roehrer, Erin; Beuscart-Zéphir, Marie-Catherine

2015-06-01

Medication-related alerting functions may include usability flaws that limit their optimal use. A first step on the way to preventing usability flaws is to understand the characteristics of these usability flaws. This systematic qualitative review aims to analyze the type of usability flaws found in medication-related alerting functions. Papers were searched via PubMed, Scopus and Ergonomics Abstracts databases, along with references lists. Paper selection, data extraction and data analysis was performed by two to three Human Factors experts. Meaningful semantic units representing instances of usability flaws were the main data extracted. They were analyzed through qualitative methods: categorization following general usability heuristics and through an inductive process for the flaws specific to medication-related alerting functions. From the 6380 papers initially identified, 26 met all eligibility criteria. The analysis of the papers identified a total of 168 instances of usability flaws that could be classified into 13 categories of usability flaws representing either violations of general usability principles (i.e. they could be found in any system, e.g. guidance and workload issues) or infractions specific to medication-related alerting functions. The latter refer to issues of low signal-to-noise ratio, incomplete content of alerts, transparency, presentation mode and timing, missing alert features, tasks and control distribution. The list of 168 instances of usability flaws of medication-related alerting functions provides a source of knowledge for checking the usability of medication-related alerting functions during their design and evaluation process and ultimately constructs evidence-based usability design principles for these functions. Copyright © 2015 Elsevier Inc. All rights reserved.

Navigation in the electronic health record: A review of the safety and usability literature.

PubMed

Roman, Lisette C; Ancker, Jessica S; Johnson, Stephen B; Senathirajah, Yalini

2017-03-01

Inefficient navigation in electronic health records has been shown to increase users' cognitive load, which may increase potential for errors, reduce efficiency, and increase fatigue. However, navigation has received insufficient recognition and attention in the electronic health record (EHR) literature as an independent construct and contributor to overall usability. Our aims in this literature review were to (1) assess the prevalence of navigation-related topics within the EHR usability and safety research literature, (2) categorize types of navigation actions within the EHR, (3) capture relationships between these navigation actions and usability principles, and (4) collect terms and concepts related to EHR navigation. Our goal was to improve access to navigation-related research in usability. We applied scoping literature review search methods with the assistance of a reference librarian to identify articles published since 1996 that reported evaluation of the usability or safety of an EHR user interface via user test, analytic methods, or inspection methods. The 4336 references collected from MEDLINE, EMBASE, Engineering Village, and expert referrals were de-duplicated and screened for relevance, and navigation-related concepts were abstracted from the 21 articles eligible for review using a standard abstraction form. Of the 21 eligible articles, 20 (95%) mentioned navigation in results and discussion of usability evaluations. Navigation between pages of the EHR was the more frequently documented type of navigation (86%) compared to navigation within a single page (14%). Navigation actions (e.g., scrolling through a medication list) were frequently linked to specific usability heuristic violations, among which flexibility and efficiency of use, recognition rather than recall, and error prevention were most common. Discussion of navigation was prevalent in results across all types of evaluation methods among the articles reviewed. Navigating between multiple screens was frequently identified as a usability barrier. The lack of standard terminology created some challenges to identifying and comparing articles. We observed that usability researchers are frequently capturing navigation-related issues even in articles that did not explicitly state navigation as a focus. Capturing and synthesizing the literature on navigation is challenging because of the lack of uniform vocabulary. Navigation is a potential target for normative recommendations for improved interaction design for safer systems. Future research in this domain, including development of normative recommendations for usability design and evaluation, will be facilitated by development of a standard terminology for describing EHR navigation. Copyright © 2017 Elsevier Inc. All rights reserved.

Usability evaluation of pharmacogenomics clinical decision support aids and clinical knowledge resources in a computerized provider order entry system: a mixed methods approach.

PubMed

Devine, Emily Beth; Lee, Chia-Ju; Overby, Casey L; Abernethy, Neil; McCune, Jeannine; Smith, Joe W; Tarczy-Hornoch, Peter

2014-07-01

Pharmacogenomics (PGx) is positioned to have a widespread impact on the practice of medicine, yet physician acceptance is low. The presentation of context-specific PGx information, in the form of clinical decision support (CDS) alerts embedded in a computerized provider order entry (CPOE) system, can aid uptake. Usability evaluations can inform optimal design, which, in turn, can spur adoption. The study objectives were to: (1) evaluate an early prototype, commercial CPOE system with PGx-CDS alerts in a simulated environment, (2) identify potential improvements to the system user interface, and (3) understand the contexts under which PGx knowledge embedded in an electronic health record is useful to prescribers. Using a mixed methods approach, we presented seven cardiologists and three oncologists with five hypothetical clinical case scenarios. Each scenario featured a drug for which a gene encoding drug metabolizing enzyme required consideration of dosage adjustment. We used Morae(®) to capture comments and on-screen movements as participants prescribed each drug. In addition to PGx-CDS alerts, 'Infobutton(®)' and 'Evidence' icons provided participants with clinical knowledge resources to aid decision-making. Nine themes emerged. Five suggested minor improvements to the CPOE user interface; two suggested presenting PGx information through PGx-CDS alerts using an 'Infobutton' or 'Evidence' icon. The remaining themes were strong recommendations to provide succinct, relevant guidelines and dosing recommendations of phenotypic information from credible and trustworthy sources; any more information was overwhelming. Participants' median rating of PGx-CDS system usability was 2 on a Likert scale ranging from 1 (strongly agree) to 7 (strongly disagree). Usability evaluation results suggest that participants considered PGx information important for improving prescribing decisions; and that they would incorporate PGx-CDS when information is presented in relevant and useful ways. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Experiences with establishing and implementing learning management system and computer-based test system in medical college.

PubMed

Park, Joo Hyun; Son, Ji Young; Kim, Sun

2012-09-01

The purpose of this study was to establish an e-learning system to support learning in medical education and identify solutions for improving the system. A learning management system (LMS) and computer-based test (CBT) system were established to support e-learning for medical students. A survey of 219 first- and second-grade medical students was administered. The questionnaire included 9 forced choice questions about the usability of system and 2 open-ended questions about necessary improvements to the system. The LMS consisted of a class management, class evaluation, and class attendance system. CBT consisted of a test management, item bank, and authoring tool system. The results of the survey showed a high level of satisfaction in all system usability items except for stability. Further, the advantages of the e-learning system were ensuring information accessibility, providing constant feedback, and designing an intuitive interface. Necessary improvements to the system were stability, user control, readability, and diverse device usage. Based on the findings, suggestions for developing an e-learning system to improve usability by medical students and support learning effectively are recommended.

Evaluating the parent-adolescent communication toolkit: Usability and preliminary content effectiveness of an online intervention.

PubMed

Toombs, Elaine; Unruh, Anita; McGrath, Patrick

2018-01-01

This study aimed to assess the Parent-Adolescent Communication Toolkit, an online intervention designed to help improve parent communication with their adolescents. Participant preferences for two module delivery systems (sequential and unrestricted module access) were identified. Usability assessment of the PACT intervention was completed using pre-test and posttest comparisons. Usability data, including participant completion and satisfaction ratings were examined. Parents ( N  =   18) of adolescents were randomized to a sequential or unrestricted chapter access group. Parent participants completed pre-test measures, the PACT intervention and posttest measures. Participants provided feedback for the intervention to improve modules and provided usability ratings. Adolescent pre- and posttest ratings were evaluated. Usability ratings were high and parent feedback was positive. The sequential module access groups rated the intervention content higher and completed more content than the unrestricted chapter access group, indicating support for the sequential access design. Parent mean posttest communication scores were significantly higher ( p  <   .05) than pre-test scores. No significant differences were detected for adolescent participants. Findings suggest that the Parent-Adolescent Communication Toolkit has potential to improve parent-adolescent communication but further effectiveness assessment is required.

Understanding Usefulness in Human-Computer Interaction to Enhance User Experience Evaluation

ERIC Educational Resources Information Center

MacDonald, Craig Matthew

2012-01-01

The concept of usefulness has implicitly played a pivotal role in evaluation research, but the meaning of usefulness has changed over time from system reliability to user performance and learnability/ease of use for non-experts. Despite massive technical and social changes, usability remains the "gold standard" for system evaluation.…

Usability Flaws in Medication Alerting Systems: Impact on Usage and Work System.

PubMed

Marcilly, R; Ammenwerth, E; Roehrer, E; Pelayo, S; Vasseur, F; Beuscart-Zéphir, M-C

2015-08-13

Previous research has shown that medication alerting systems face usability issues. There has been no previous attempt to systematically explore the consequences of usability flaws in such systems on users (i.e. usage problems) and work systems (i.e. negative outcomes). This paper aims at exploring and synthesizing the consequences of usability flaws in terms of usage problems and negative outcomes on the work system. A secondary analysis of 26 papers included in a prior systematic review of the usability flaws in medication alerting was performed. Usage problems and negative outcomes were extracted and sorted. Links between usability flaws, usage problems, and negative outcomes were also analyzed. Poor usability generates a large variety of consequences. It impacts the user from a cognitive, behavioral, emotional, and attitudinal perspective. Ultimately, usability flaws have negative consequences on the workflow, the effectiveness of the technology, the medication management process, and, more importantly, patient safety. Only few complete pathways leading from usability flaws to negative outcomes were identified. Usability flaws in medication alerting systems impede users, and ultimately their work system, and negatively impact patient safety. Therefore, the usability dimension may act as a hidden explanatory variable that could explain, at least partly, the (absence of) intended outcomes of new technology.

Usability Flaws in Medication Alerting Systems: Impact on Usage and Work System

PubMed Central

Ammenwerth, E.; Roehrer, E.; Pelayo, S.; Vasseur, F.; Beuscart-Zéphir, M.-C.

2015-01-01

Summary Objectives Previous research has shown that medication alerting systems face usability issues. There has been no previous attempt to systematically explore the consequences of usability flaws in such systems on users (i.e. usage problems) and work systems (i.e. negative outcomes). This paper aims at exploring and synthesizing the consequences of usability flaws in terms of usage problems and negative outcomes on the work system. Methods A secondary analysis of 26 papers included in a prior systematic review of the usability flaws in medication alerting was performed. Usage problems and negative outcomes were extracted and sorted. Links between usability flaws, usage problems, and negative outcomes were also analyzed. Results Poor usability generates a large variety of consequences. It impacts the user from a cognitive, behavioral, emotional, and attitudinal perspective. Ultimately, usability flaws have negative consequences on the workflow, the effectiveness of the technology, the medication management process, and, more importantly, patient safety. Only few complete pathways leading from usability flaws to negative outcomes were identified. Conclusion Usability flaws in medication alerting systems impede users, and ultimately their work system, and negatively impact patient safety. Therefore, the usability dimension may act as a hidden explanatory variable that could explain, at least partly, the (absence of) intended outcomes of new technology. PMID:26123906

Training Software Developers and Designers to Conduct Usability Evaluations

ERIC Educational Resources Information Center

Skov, Mikael Brasholt; Stage, Jan

2012-01-01

Many efforts to improve the interplay between usability evaluation and software development rely either on better methods for conducting usability evaluations or on better formats for presenting evaluation results in ways that are useful for software designers and developers. Both of these approaches depend on a complete division of work between…

Evaluation of a Dry EEG System for Application of Passive Brain-Computer Interfaces in Autonomous Driving

PubMed Central

Zander, Thorsten O.; Andreessen, Lena M.; Berg, Angela; Bleuel, Maurice; Pawlitzki, Juliane; Zawallich, Lars; Krol, Laurens R.; Gramann, Klaus

2017-01-01

We tested the applicability and signal quality of a 16 channel dry electroencephalography (EEG) system in a laboratory environment and in a car under controlled, realistic conditions. The aim of our investigation was an estimation how well a passive Brain-Computer Interface (pBCI) can work in an autonomous driving scenario. The evaluation considered speed and accuracy of self-applicability by an untrained person, quality of recorded EEG data, shifts of electrode positions on the head after driving-related movements, usability, and complexity of the system as such and wearing comfort over time. An experiment was conducted inside and outside of a stationary vehicle with running engine, air-conditioning, and muted radio. Signal quality was sufficient for standard EEG analysis in the time and frequency domain as well as for the use in pBCIs. While the influence of vehicle-induced interferences to data quality was insignificant, driving-related movements led to strong shifts in electrode positions. In general, the EEG system used allowed for a fast self-applicability of cap and electrodes. The assessed usability of the system was still acceptable while the wearing comfort decreased strongly over time due to friction and pressure to the head. From these results we conclude that the evaluated system should provide the essential requirements for an application in an autonomous driving context. Nevertheless, further refinement is suggested to reduce shifts of the system due to body movements and increase the headset's usability and wearing comfort. PMID:28293184

Evaluation of a Dry EEG System for Application of Passive Brain-Computer Interfaces in Autonomous Driving.

PubMed

Zander, Thorsten O; Andreessen, Lena M; Berg, Angela; Bleuel, Maurice; Pawlitzki, Juliane; Zawallich, Lars; Krol, Laurens R; Gramann, Klaus

2017-01-01

We tested the applicability and signal quality of a 16 channel dry electroencephalography (EEG) system in a laboratory environment and in a car under controlled, realistic conditions. The aim of our investigation was an estimation how well a passive Brain-Computer Interface (pBCI) can work in an autonomous driving scenario. The evaluation considered speed and accuracy of self-applicability by an untrained person, quality of recorded EEG data, shifts of electrode positions on the head after driving-related movements, usability, and complexity of the system as such and wearing comfort over time. An experiment was conducted inside and outside of a stationary vehicle with running engine, air-conditioning, and muted radio. Signal quality was sufficient for standard EEG analysis in the time and frequency domain as well as for the use in pBCIs. While the influence of vehicle-induced interferences to data quality was insignificant, driving-related movements led to strong shifts in electrode positions. In general, the EEG system used allowed for a fast self-applicability of cap and electrodes. The assessed usability of the system was still acceptable while the wearing comfort decreased strongly over time due to friction and pressure to the head. From these results we conclude that the evaluated system should provide the essential requirements for an application in an autonomous driving context. Nevertheless, further refinement is suggested to reduce shifts of the system due to body movements and increase the headset's usability and wearing comfort.

Reusable Software Usability Specifications for mHealth Applications.

PubMed

Cruz Zapata, Belén; Fernández-Alemán, José Luis; Toval, Ambrosio; Idri, Ali

2018-01-25

One of the key factors for the adoption of mobile technologies, and in particular of mobile health applications, is usability. A usable application will be easier to use and understand by users, and will improve user's interaction with it. This paper proposes a software requirements catalog for usable mobile health applications, which can be used for the development of new applications, or the evaluation of existing ones. The catalog is based on the main identified sources in literature on usability and mobile health applications. Our catalog was organized according to the ISO/IEC/IEEE 29148:2011 standard and follows the SIREN methodology to create reusable catalogs. The applicability of the catalog was verified by the creation of an audit method, which was used to perform the evaluation of a real app, S Health, application created by Samsung Electronics Co. The usability requirements catalog, along with the audit method, identified several usability flaws on the evaluated app, which scored 83%. Some flaws were detected in the app related to the navigation pattern. Some more issues related to the startup experience, empty screens or writing style were also found. The way a user navigates through an application improves or deteriorates user's experience with the application. We proposed a reusable usability catalog and an audit method. This proposal was used to evaluate a mobile health application. An audit report was created with the usability issues identified on the evaluated application.

Does user-centred design affect the efficiency, usability and safety of CPOE order sets?

PubMed

Chan, Julie; Shojania, Kaveh G; Easty, Anthony C; Etchells, Edward E

2011-05-01

Application of user-centred design principles to Computerized provider order entry (CPOE) systems may improve task efficiency, usability or safety, but there is limited evaluative research of its impact on CPOE systems. We evaluated the task efficiency, usability, and safety of three order set formats: our hospital's planned CPOE order sets (CPOE Test), computer order sets based on user-centred design principles (User Centred Design), and existing pre-printed paper order sets (Paper). 27 staff physicians, residents and medical students. Sunnybrook Health Sciences Centre, an academic hospital in Toronto, Canada. Methods Participants completed four simulated order set tasks with three order set formats (two CPOE Test tasks, one User Centred Design, and one Paper). Order of presentation of order set formats and tasks was randomized. Users received individual training for the CPOE Test format only. Completion time (efficiency), requests for assistance (usability), and errors in the submitted orders (safety). 27 study participants completed 108 order sets. Mean task times were: User Centred Design format 273 s, Paper format 293 s (p=0.73 compared to UCD format), and CPOE Test format 637 s (p<0.0001 compared to UCD format). Users requested assistance in 31% of the CPOE Test format tasks, whereas no assistance was needed for the other formats (p<0.01). There were no significant differences in number of errors between formats. The User Centred Design format was more efficient and usable than the CPOE Test format even though training was provided for the latter. We conclude that application of user-centred design principles can enhance task efficiency and usability, increasing the likelihood of successful implementation.

Does user-centred design affect the efficiency, usability and safety of CPOE order sets?

PubMed Central

Chan, Julie; Shojania, Kaveh G; Easty, Anthony C

2011-01-01

Background Application of user-centred design principles to Computerized provider order entry (CPOE) systems may improve task efficiency, usability or safety, but there is limited evaluative research of its impact on CPOE systems. Objective We evaluated the task efficiency, usability, and safety of three order set formats: our hospital's planned CPOE order sets (CPOE Test), computer order sets based on user-centred design principles (User Centred Design), and existing pre-printed paper order sets (Paper). Participants 27staff physicians, residents and medical students. Setting Sunnybrook Health Sciences Centre, an academic hospital in Toronto, Canada. Methods Participants completed four simulated order set tasks with three order set formats (two CPOE Test tasks, one User Centred Design, and one Paper). Order of presentation of order set formats and tasks was randomized. Users received individual training for the CPOE Test format only. Main Measures Completion time (efficiency), requests for assistance (usability), and errors in the submitted orders (safety). Results 27 study participants completed 108 order sets. Mean task times were: User Centred Design format 273 s, Paper format 293 s (p=0.73 compared to UCD format), and CPOE Test format 637 s (p<0.0001 compared to UCD format). Users requested assistance in 31% of the CPOE Test format tasks, whereas no assistance was needed for the other formats (p<0.01). There were no significant differences in number of errors between formats. Conclusions The User Centred Design format was more efficient and usable than the CPOE Test format even though training was provided for the latter. We conclude that application of user-centred design principles can enhance task efficiency and usability, increasing the likelihood of successful implementation. PMID:21486886

Evaluating Web Usability

ERIC Educational Resources Information Center

Snider, Jean; Martin, Florence

2012-01-01

Web usability focuses on design elements and processes that make web pages easy to use. A website for college students was evaluated for underutilization. One-on-one testing, focus groups, web analytics, peer university review and marketing focus group and demographic data were utilized to conduct usability evaluation. The results indicated that…

Usability Assessment of the Missouri Cancer Registry's Published Interactive Mapping Reports: Round One.

PubMed

Ben Ramadan, Awatef Ahmed; Jackson-Thompson, Jeannette; Schmaltz, Chester Lee

2017-08-04

 Many users of spatial data have difficulty interpreting information in health-related spatial reports. The Missouri Cancer Registry and Research Center (MCR-ARC) has produced interactive reports for several years. These reports have never been tested for usability.  The aims of this study were to: (1) conduct a multi-approach usability testing study to understand ease of use (user friendliness) and user satisfaction; and (2) evaluate the usability of MCR-ARC's published InstantAtlas reports.   An institutional review board (IRB) approved mixed methodology usability testing study using a convenience sample of health professionals. A recruiting email was sent to faculty in the Master of Public Health program and to faculty and staff in the Department of Health Management and Informatics at the University of Missouri-Columbia. The study included 7 participants. The test included a pretest questionnaire, a multi-task usability test, and the System Usability Scale (SUS). Also, the researchers collected participants' comments about the tested maps immediately after every trial. Software was used to record the computer screen during the trial and the participants' spoken comments. Several performance and usability metrics were measured to evaluate the usability of MCR-ARC's published mapping reports. Of the 10 assigned tasks, 6 reached a 100% completion success rate, and this outcome was relative to the complexity of the tasks. The simple tasks were handled more efficiently than the complicated tasks. The SUS score ranged between 20-100 points, with an average of 62.7 points and a median of 50.5 points. The tested maps' effectiveness outcomes were better than the efficiency and satisfaction outcomes. There was a statistically significant relationship between the subjects' performance on the study test and the users' previous experience with geographic information system (GIS) tools (P=.03). There were no statistically significant relationships between users' performance and satisfaction and their education level, work type, or previous experience in health care (P>.05). There were strong positive correlations between the three measured usability elements. The tested maps should undergo an extensive refining and updating to overcome all the discovered usability issues and meet the perspectives and needs of the tested maps' potential users. The study results might convey the perspectives of academic health professionals toward GIS health data. We need to conduct a second-round usability study with public health practitioners and cancer professionals who use GIS tools on a routine basis. Usability testing should be conducted before and after releasing MCR-ARC's maps in the future. ©Awatef Ahmed Ben Ramadan, Jeannette Jackson-Thompson, Chester Lee Schmaltz. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 04.08.2017.

Usability Assessment of the Missouri Cancer Registry’s Published Interactive Mapping Reports: Round One

PubMed Central

Jackson-Thompson, Jeannette; Schmaltz, Chester Lee

2017-01-01

Background  Many users of spatial data have difficulty interpreting information in health-related spatial reports. The Missouri Cancer Registry and Research Center (MCR-ARC) has produced interactive reports for several years. These reports have never been tested for usability. Objective  The aims of this study were to: (1) conduct a multi-approach usability testing study to understand ease of use (user friendliness) and user satisfaction; and (2) evaluate the usability of MCR-ARC’s published InstantAtlas reports. Methods   An institutional review board (IRB) approved mixed methodology usability testing study using a convenience sample of health professionals. A recruiting email was sent to faculty in the Master of Public Health program and to faculty and staff in the Department of Health Management and Informatics at the University of Missouri-Columbia. The study included 7 participants. The test included a pretest questionnaire, a multi-task usability test, and the System Usability Scale (SUS). Also, the researchers collected participants’ comments about the tested maps immediately after every trial. Software was used to record the computer screen during the trial and the participants’ spoken comments. Several performance and usability metrics were measured to evaluate the usability of MCR-ARC’s published mapping reports. Results Of the 10 assigned tasks, 6 reached a 100% completion success rate, and this outcome was relative to the complexity of the tasks. The simple tasks were handled more efficiently than the complicated tasks. The SUS score ranged between 20-100 points, with an average of 62.7 points and a median of 50.5 points. The tested maps’ effectiveness outcomes were better than the efficiency and satisfaction outcomes. There was a statistically significant relationship between the subjects’ performance on the study test and the users’ previous experience with geographic information system (GIS) tools (P=.03). There were no statistically significant relationships between users’ performance and satisfaction and their education level, work type, or previous experience in health care (P>.05). There were strong positive correlations between the three measured usability elements. Conclusions The tested maps should undergo an extensive refining and updating to overcome all the discovered usability issues and meet the perspectives and needs of the tested maps’ potential users. The study results might convey the perspectives of academic health professionals toward GIS health data. We need to conduct a second-round usability study with public health practitioners and cancer professionals who use GIS tools on a routine basis. Usability testing should be conducted before and after releasing MCR-ARC’s maps in the future. PMID:28778842

Checking an integrated model of web accessibility and usability evaluation for disabled people.

PubMed

Federici, Stefano; Micangeli, Andrea; Ruspantini, Irene; Borgianni, Stefano; Corradi, Fabrizio; Pasqualotto, Emanuele; Olivetti Belardinelli, Marta

2005-07-08

A combined objective-oriented and subjective-oriented method for evaluating accessibility and usability of web pages for students with disability was tested. The objective-oriented approach is devoted to verifying the conformity of interfaces to standard rules stated by national and international organizations responsible for web technology standardization, such as W3C. Conversely, the subjective-oriented approach allows assessing how the final users interact with the artificial system, accessing levels of user satisfaction based on personal factors and environmental barriers. Five kinds of measurements were applied as objective-oriented and subjective-oriented tests. Objective-oriented evaluations were performed on the Help Desk web page for students with disability, included in the website of a large Italian state university. Subjective-oriented tests were administered to 19 students labeled as disabled on the basis of their own declaration at the University enrolment: 13 students were tested by means of the SUMI test and six students by means of the 'Cooperative evaluation'. Objective-oriented and subjective-oriented methods highlighted different and sometimes conflicting results. Both methods have pointed out much more consistency regarding levels of accessibility than of usability. Since usability is largely affected by individual differences in user's own (dis)abilities, subjective-oriented measures underscored the fact that blind students encountered much more web surfing difficulties.

Human factors evaluation of TSO-C129A GPS receivers

DOT National Transportation Integrated Search

1998-10-22

This report documents an evaluation of the usability of TSO-C129a-certified Global Positioning System (GPS) receivers. Bench and flight tests were conducted on six GPS receivers. The evaluations covered 23 flight tasks. Both subjective and objective ...

An Online Social Constructivist Course: Toward a Framework for Usability Evaluations

ERIC Educational Resources Information Center

Phillips, Alana S.; Sheffield, Anneliese; Moore, Michelle; Robinson, Heather A.

2016-01-01

There is a need for a holistic usability evaluation framework that accommodates social constructivist online courses. Social knowledge construction may not be adequately evaluated using current frameworks. This qualitative research study examined the usability needs of a social constructivist online course. Data from an online course were analyzed…

A framework for reporting on human factor/usability studies of health information technologies.

PubMed

Peute, Linda W; Driest, Keiko F; Marcilly, Romaric; Bras Da Costa, Sabrina; Beuscart-Zephir, Marie-Catherine; Jaspers, Monique W M

2013-01-01

Increasingly, studies are being published on the potential negative effect of introducing poor designed Health Information Technology (HIT) into clinical settings, relating to technology-induced errors and adverse events. Academic research on HIT design and evaluation is an extremely important source of information in providing new insights into factors contributing to successful system (re)design efforts, system user-friendliness and usability issues and safety critical aspects of HIT design. However, these studies have been inconsistent and incomprehensive in their reporting, complicating the appraisal of outcomes, generalizability of study findings, meta-analysis and harmonization of the available evidence. To improve identification of type of use errors and safety related issues regarding design and implementation of HIT, consensus on issues to be reported on in scientific publications is a necessary step forward. This study presents the first approach to a framework providing a set of principles to follow for comprehensive and unambiguous reporting of HIT design and usability evaluation studies with the objective to reduce variation, improve on the publication reporting quality and proper indexation of these studies. This framework may be helpful in expanding the knowledge base not only concerning the application of Human Factors (HF)/Usability studies of HIT but also improve the knowledge base of how to (re)design and implement effective, efficient and safe HIT.

Addressing critical issues in the development of an Oncology Information System.

PubMed

Urda, D; Ribelles, N; Subirats, J L; Franco, L; Alba, E; Jerez, J M

2013-05-01

This paper presents the experience on the design and implementation of a user-centered Oncology Information System developed for the Medical Oncology Department at the "Hospital Universitario Virgen de la Victoria", in Málaga, Spain. The project focused on the aspects considered in the literature as critical factors for a successful deployment and usage of a health information system. System usability, adequate technology, integration of clinical routines, real-time statistical analysis of data, information confidentiality and standard protocol-based external interconnection were the key aspects considered. The developed system is based on a web application with a modular and layered architecture accounting for usability, ease of maintenance and further system development. Evaluation of system usability was carried at three and fifteen months after system deployment to analyze the advantages/disadvantages experienced by the end-users. A thorough prior analysis of clinical activities and workflows, the use of the adequate technology, and the availability of data analysis tools will almost guarantee success in the deployment of an Oncology Information System. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

A methodological approach for designing a usable ontology-based GUI in healthcare.

PubMed

Lasierra, N; Kushniruk, A; Alesanco, A; Borycki, E; García, J

2013-01-01

This paper presents a methodological approach to the design and evaluation of an interface for an ontology-based system used for designing care plans for monitoring patients at home. In order to define the care plans, physicians need a tool for creating instances of the ontology and configuring some rules. Our purpose is to develop an interface to allow clinicians to interact with the ontology. Although ontology-driven applications do not necessarily present the ontology in the user interface, it is our hypothesis that showing selected parts of the ontology in a "usable" way could enhance clinician's understanding and make easier the definition of the care plans. Based on prototyping and iterative testing, this methodology combines visualization techniques and usability methods. Preliminary results obtained after a formative evaluation indicate the effectiveness of suggested combination.

Determining the Number of Participants Needed for the Usability Evaluation of E-Learning Resources: A Monte Carlo Simulation

ERIC Educational Resources Information Center

Davids, Mogamat Razeen; Harvey, Justin; Halperin, Mitchell L.; Chikte, Usuf M. E.

2015-01-01

The usability of computer interfaces has a major influence on learning. Optimising the usability of e-learning resources is therefore essential. However, this may be neglected because of time and monetary constraints. User testing is a common approach to usability evaluation and involves studying typical end-users interacting with the application…

Usability in Mobile Electronic Data Collection Tools: Form Developers' Views.

PubMed

Mugisha, Alice; Babic, Ankica; Wakholi, Peter; Nankabirwa, Victoria; Tylleskar, Thorkild

2017-01-01

Mobile Electronic Data Collection Tools (MEDCTs) are created by form developers to collect data. Usability being one of the top quality attributes is of great concern to developers of any interactive applications. However, little is known about the form developers' understanding of usability, how they measure usability and their limitations in designing for usability. We conducted an empirical study where we aimed at getting the developers' views on usability by interviewing 8 form developers. These are creators of forms used for data collection. We found that developers knew about usability, but it was not their main focus during form development. Challenges included constraining deadlines, software limitations and the insufficient communication with the field users to establish the usability needs. Furthermore, the methods used to evaluate the usability of created forms varied amongst developers and these included in-house evaluations and feedback from piloting sessions with end users.

Effectiveness and usability of Scanning Wizard software: a tool for enhancing switch scanning.

PubMed

Koester, Heidi Horstmann; Simpson, Richard C

2017-11-24

Scanning Wizard software helps scanning users improve the setup of their switch and scanning system. This study evaluated Scanning Wizard's effectiveness and usability. Ten people who use switch scanning and ten practitioners used Scanning Wizard in the initial session. Usability was high, based on survey responses averaging over 4.5 out of 5, and qualitative feedback was very positive. Five switch users were able to complete the multi-week protocol, using settings on their own scanning system that were recommended from the Scanning Wizard session. Using these revised settings, text entry rates improved by an average of 71%, ranging from 29% to 172% improvement. Results suggest that Scanning Wizard is a useful tool for improving the configuration of scanning systems for people who use switch scanning to communicate. Implications for Rehabilitation Some individuals with severe physical impairments use switch scanning for spoken and written communication. Scanning Wizard software helps scanning users improve the setup of their switch and scanning system. This study demonstrated high usability of Scanning Wizard (with 10 switch userpractitioner teams) and increased text entry rate by an average of 71% (for five switch users). Results suggest that Scanning Wizard is a useful tool for improving the configuration of scanning systems for people who use switch scanning to communicate.

Evaluation of Home Health Care Devices: Remote Usability Assessment.

PubMed

Kortum, Philip; Peres, S Camille

2015-06-05

An increasing amount of health care is now performed in a home setting, away from the hospital. While there is growing anecdotal evidence about the difficulty patients and caregivers have using increasingly complex health care devices in the home, there has been little systematic scientific study to quantify the global nature of home health care device usability in the field. Research has tended to focus on a handful of devices, making it difficult to gain a broad view of the usability of home-care devices in general. The objective of this paper is to describe a remote usability assessment method using the System Usability Scale (SUS), and to report on the usability of a broad range of health care devices using this metric. A total of 271 participants selected and rated up to 10 home health care devices of their choice using the SUS, which scores usability from 0 (unusable) to 100 (highly usable). Participants rated a total of 455 devices in their own home without an experimenter present. Usability scores ranged from 98 (oxygen masks) to 59 (home hormone test kits). An analysis conducted on devices that had at least 10 ratings showed that the effect of device on SUS scores was significant (P<.001), and that the usability of these devices was on the low end when compared with other commonly used items in the home, such as microwave ovens and telephones. A large database of usability scores for home health care devices collected using this remote methodology would be beneficial for physicians, patients, and their caregivers.

Implementation and evaluation of LMS mobile application: scele mobile based on user-centered design

NASA Astrophysics Data System (ADS)

Banimahendra, R. D.; Santoso, H. B.

2018-03-01

The development of mobile technology is now increasing rapidly, demanding all activities including learning should be done on mobile devices. It shows that the implementation of mobile application as a learning medium needs to be done. This study describes the process of developing and evaluating the Moodle-based mobile Learning Management System (LMS) application called Student Centered e-Learning Environment (SCeLE). This study discusses the process of defining features, implementing features into the application, and evaluating the application. We define the features using user research and literature study, then we implement the application with user-centered design basis, at the last phase we evaluated the application using usability testing and system usability score (SUS). The purpose of this study is to determine the extent to which this application can help the users doing their tasks and provide recommendation for the next research and development.

Measuring AT Usability with the Modified System Usability Scale (SUS).

PubMed

Friesen, Emma L

2017-01-01

The modified System Usability Scale (SUS) is a widely used generic measure of product usability. This study concerns the usability of mobile shower commodes using correlations between the SUS and AT device-specific measures. Results suggest the modified SUS, and corresponding adjective-anchored rating scale, are appropriate for measuring MSC usability, and have potential for use with other AT devices.

Smartphone Apps for Cardiopulmonary Resuscitation Training and Real Incident Support: A Mixed-Methods Evaluation Study

PubMed Central

Felzen, Marc; Rossaint, Rolf; Tabuenca, Bernardo; Specht, Marcus; Skorning, Max

2014-01-01

Background No systematic evaluation of smartphone/mobile apps for resuscitation training and real incident support is available to date. To provide medical, usability, and additional quality criteria for the development of apps, we conducted a mixed-methods sequential evaluation combining the perspective of medical experts and end-users. Objective The study aims to assess the quality of current mobile apps for cardiopulmonary resuscitation (CPR) training and real incident support from expert as well as end-user perspective. Methods Two independent medical experts evaluated the medical content of CPR apps from the Google Play store and the Apple App store. The evaluation was based on pre-defined minimum medical content requirements according to current Basic Life Support (BLS) guidelines. In a second phase, non-medical end-users tested usability and appeal of the apps that had at least met the minimum requirements. Usability was assessed with the System Usability Scale (SUS); appeal was measured with the self-developed ReactionDeck toolkit. Results Out of 61 apps, 46 were included in the experts’ evaluation. A consolidated list of 13 apps resulted for the following layperson evaluation. The interrater reliability was substantial (kappa=.61). Layperson end-users (n=14) had a high interrater reliability (intraclass correlation 1 [ICC1]=.83, P<.001, 95% CI 0.75-0.882 and ICC2=.79, P<.001, 95% CI 0.695-0.869). Their evaluation resulted in a list of 5 recommendable apps. Conclusions Although several apps for resuscitation training and real incident support are available, very few are designed according to current BLS guidelines and offer an acceptable level of usability and hedonic quality for laypersons. The results of this study are intended to optimize the development of CPR mobile apps. The app ranking supports the informed selection of mobile apps for training situations and CPR campaigns as well as for real incident support. PMID:24647361

Smartphone apps for cardiopulmonary resuscitation training and real incident support: a mixed-methods evaluation study.

PubMed

Kalz, Marco; Lenssen, Niklas; Felzen, Marc; Rossaint, Rolf; Tabuenca, Bernardo; Specht, Marcus; Skorning, Max

2014-03-19

No systematic evaluation of smartphone/mobile apps for resuscitation training and real incident support is available to date. To provide medical, usability, and additional quality criteria for the development of apps, we conducted a mixed-methods sequential evaluation combining the perspective of medical experts and end-users. The study aims to assess the quality of current mobile apps for cardiopulmonary resuscitation (CPR) training and real incident support from expert as well as end-user perspective. Two independent medical experts evaluated the medical content of CPR apps from the Google Play store and the Apple App store. The evaluation was based on pre-defined minimum medical content requirements according to current Basic Life Support (BLS) guidelines. In a second phase, non-medical end-users tested usability and appeal of the apps that had at least met the minimum requirements. Usability was assessed with the System Usability Scale (SUS); appeal was measured with the self-developed ReactionDeck toolkit. Out of 61 apps, 46 were included in the experts' evaluation. A consolidated list of 13 apps resulted for the following layperson evaluation. The interrater reliability was substantial (kappa=.61). Layperson end-users (n=14) had a high interrater reliability (intraclass correlation 1 [ICC1]=.83, P<.001, 95% CI 0.75-0.882 and ICC2=.79, P<.001, 95% CI 0.695-0.869). Their evaluation resulted in a list of 5 recommendable apps. Although several apps for resuscitation training and real incident support are available, very few are designed according to current BLS guidelines and offer an acceptable level of usability and hedonic quality for laypersons. The results of this study are intended to optimize the development of CPR mobile apps. The app ranking supports the informed selection of mobile apps for training situations and CPR campaigns as well as for real incident support.

Feasibility and usability of a home monitoring concept based on mobile phones and near field communication (NFC) technology.

PubMed

Morak, Jürgen; Kollmann, Alexander; Schreier, Günter

2007-01-01

Utilization of mobile information and communication technologies in home monitoring applications is becoming more and more common. The mobile phone, acting as a patient terminal for patients suffering from chronic diseases, provides an active link to the caregiver to transmit health status information and receive feedback. In such a concept the usability is still limited by the necessity of entering the values via the mobile phone's small keypad. The near field communication technology (NFC), a touch-based wireless interface that became available recently, may improve the usability level of such applications significantly. The focus of this paper is to describe the development of a prototype application based on this technology embedded in a home monitoring system. The feasibility and usability of this approach are evaluated and compared with concepts used in previous approaches. The high quantifier with respect to overall usability indicates that NFC may be the technology of choice for some tasks in home monitoring applications.

Early-Stage Software Design for Usability

ERIC Educational Resources Information Center

Golden, Elspeth

2010-01-01

In spite of the goodwill and best efforts of software engineers and usability professionals, systems continue to be built and released with glaring usability flaws that are costly and difficult to fix after the system has been built. Although user interface (UI) designers, be they usability or design experts, communicate usability requirements to…

Record completeness and data concordance in an anesthesia information management system using context-sensitive mandatory data-entry fields.

PubMed

Avidan, Alexander; Weissman, Charles

2012-03-01

Use of an anesthesia information management system (AIMS) does not insure record completeness and data accuracy. Mandatory data-entry fields can be used to assure data completeness. However, they are not suited for data that is mandatory depending on the clinical situation (context sensitive). For example, information on equal breath sounds should be mandatory with tracheal intubation, but not with mask ventilation. It was hypothesized that employing context-sensitive mandatory data-entry fields can insure high data-completeness and accuracy while maintaining usability. A commercial off-the-shelf AIMS was enhanced using its built-in VBScript programming tool to build event-driven forms with context-sensitive mandatory data-entry fields. One year after introduction of the system, all anesthesia records were reviewed for data completeness. Data concordance, used as a proxy for accuracy, was evaluated using verifiable age-related data. Additionally, an anonymous satisfaction survey on general acceptance and usability of the AIMS was performed. During the initial 12 months of AIMS use, 12,241 (99.6%) of 12,290 anesthesia records had complete data. Concordances of entered data (weight, size of tracheal tubes, laryngoscopy blades and intravenous catheters) with patients' ages were 98.7-99.9%. The AIMS implementation was deemed successful by 98% of the anesthesiologists. Users rated the AIMS usability in general as very good and the data-entry forms in particular as comfortable. Due to the complexity and the high costs of implementation of an anesthesia information management system it was not possible to compare various system designs (for example with or without context-sensitive mandatory data entry-fields). Therefore, it is possible that a different or simpler design would have yielded the same or even better results. This refers also to the evaluation of usability, since users did not have the opportunity to work with different design approaches or even different computer programs. Using context-sensitive mandatory fields in an anesthesia information management system was associated with high record completeness rate and data concordance. In addition, the system's usability was rated as very good by its users. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Investigation of the Usability of Computerized Critical Care Information Systems in Germany.

PubMed

von Dincklage, Falk; Suchodolski, Klaudiusz; Lichtner, Gregor; Friesdorf, Wolfgang; Podtschaske, Beatrice; Ragaller, Maximilian

2017-01-01

The term "usability" describes how effectively, efficiently, and with what level of user satisfaction an information system can be used to accomplish specific goals. Computerized critical care information systems (CCISs) with high usability increase quality of care and staff satisfaction, while reducing medication errors. Conversely, systems lacking usability can interrupt clinical workflow, facilitate errors, and increase charting time. The aim of this study was to investigate and compare usability across CCIS currently used in Germany. In this study, German intensive care unit (ICU) nurses and physicians completed a specialized, previously validated, web-based questionnaire. The questionnaire assessed CCIS usability based on three rating models: an overall rating of the systems, a model rating technical usability, and a model rating task-specific usability. We analyzed results from 535 survey participants and compared eight different CCIS commonly used in Germany. Our results showed that usability strongly differs across the compared systems. The system ICUData had the best overall rating and technical usability, followed by the platforms ICM and MetaVision. The same three systems performed best in the rating of task-specific usability without significant differences between each other. Across all systems, overall ratings were more dependent on ease-of-use aspects than on aspects of utility/functionality, and the general scope of the functions offered was rated better than how well the functions are realized. Our results suggest that manufacturers should shift some of their effort away from the development of new features and focus more on improving the ease-of-use and quality of existing features.

Engineering the electronic health record for safety: a multi-level video-based approach to diagnosing and preventing technology-induced error arising from usability problems.

PubMed

Borycki, Elizabeth M; Kushniruk, Andre W; Kuwata, Shigeki; Kannry, Joseph

2011-01-01

Electronic health records (EHRs) promise to improve and streamline healthcare through electronic entry and retrieval of patient data. Furthermore, based on a number of studies showing their positive benefits, they promise to reduce medical error and make healthcare safer. However, a growing body of literature has clearly documented that if EHRS are not designed properly and with usability as an important goal in their design, rather than reducing error, EHR deployment has the potential to actually increase medical error. In this paper we describe our approach to engineering (and reengineering) EHRs in order to increase their beneficial potential while at the same time improving their safety. The approach described in this paper involves an integration of the methods of usability analysis with video analysis of end users interacting with EHR systems and extends the evaluation of the usability of EHRs to include the assessment of the impact of these systems on work practices. Using clinical simulations, we analyze human-computer interaction in real healthcare settings (in a portable, low-cost and high fidelity manner) and include both artificial and naturalistic data collection to identify potential usability problems and sources of technology-induced error prior to widespread system release. Two case studies where the methods we have developed and refined have been applied at different levels of user-computer interaction are described.

Evaluating the Usability of a Museum Web Site.

ERIC Educational Resources Information Center

Harms, Ilse; Schweibenz, Werner

This paper presents a research project conducted by the Department of Information Science in cooperation with the Saarland Museum, the art museum of the Federal State of Saarland, Germany. The study had two aims. The first was to evaluate some methods of usability engineering for the Web, and the second was to evaluate the usability of the…

Enhancing the Usability of an Optical Reader System to Support Point-of-Care Rapid Diagnostic Testing: An Iterative Design Approach.

PubMed

Hohenstein, Jess; O'Dell, Dakota; Murnane, Elizabeth L; Lu, Zhengda; Erickson, David; Gay, Geri

2017-11-21

In today's health care environment, increasing costs and inadequate medical resources have created a worldwide need for more affordable diagnostic tools that are also portable, fast, and easy to use. To address this issue, numerous research and commercial efforts have focused on developing rapid diagnostic technologies; however, the efficacy of existing systems has been hindered by usability problems or high production costs, making them infeasible for deployment in at-home, point-of-care (POC), or resource-limited settings. The aim of this study was to create a low-cost optical reader system that integrates with any smart device and accepts any type of rapid diagnostic test strip to provide fast and accurate data collection, sample analysis, and diagnostic result reporting. An iterative design methodology was employed by a multidisciplinary research team to engineer three versions of a portable diagnostic testing device that were evaluated for usability and overall user receptivity. Repeated design critiques and usability studies identified a number of system requirements and considerations (eg, software compatibility, biomatter contamination, and physical footprint) that we worked to incrementally incorporate into successive system variants. Our final design phase culminated in the development of Tidbit, a reader that is compatible with any Wi-Fi-enabled device and test strip format. The Tidbit includes various features that support intuitive operation, including a straightforward test strip insertion point, external indicator lights, concealed electronic components, and an asymmetric shape, which inherently signals correct device orientation. Usability testing of the Tidbit indicates high usability for potential user communities. This study presents the design process, specification, and user reception of the Tidbit, an inexpensive, easy-to-use, portable optical reader for fast, accurate quantification of rapid diagnostic test results. Usability testing suggests that the reader is usable among and can benefit a wide group of potential users, including in POC contexts. Generally, the methodology of this study demonstrates the importance of testing these types of systems with potential users and exemplifies how iterative design processes can be employed by multidisciplinary research teams to produce compelling technological solutions. ©Jess Hohenstein, Dakota O'Dell, Elizabeth L Murnane, Zhengda Lu, David Erickson, Geri Gay. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 21.11.2017.

Supporting cancer patients in illness management: usability evaluation of a mobile app.

PubMed

Mirkovic, Jelena; Kaufman, David R; Ruland, Cornelia M

2014-08-13

Mobile phones and tablets currently represent a significant presence in people's everyday lives. They enable access to different information and services independent of current place and time. Such widespread connectivity offers significant potential in different app areas including health care. Our goal was to evaluate the usability of the Connect Mobile app. The mobile app enables mobile access to the Connect system, an online system that supports cancer patients in managing health-related issues. Along with symptom management, the system promotes better patient-provider communication, collaboration, and shared decision making. The Connect Mobile app enables access to the Connect system over both mobile phones and tablets. The study consisted of usability tests of a high fidelity prototype with 7 cancer patients where the objectives were to identify existing design and functionality issues and to provide patients with a real look-and-feel of the mobile system. In addition, we conducted semistructured interviews to obtain participants' feedback about app usefulness, identify the need for new system features and design requirements, and measure the acceptance of the mobile app and its features within everyday health management. The study revealed a total of 27 design issues (13 for mobile apps and 14 for tablet apps), which were mapped to source events (ie, errors, requests for help, participants' concurrent feedback, and moderator observation). We also applied usability heuristics to identify violations of usability principles. The majority of violations were related to enabling ease of input, screen readability, and glanceability (15 issues), as well as supporting an appropriate match between systems and the real world (7 issues) and consistent mapping of system functions and interactions (4 issues). Feedback from participants also showed the cancer patients' requirements for support systems and how these needs are influenced by different context-related factors, such as type of access terminal (eg, desktop computer, tablet, mobile phone) and phases of illness. Based on the observed results, we proposed design and functionality recommendations that can be used for the development of mobile apps for cancer patients to support their health management process. Understanding and addressing users' requirements is one of the main prerequisites for developing useful and effective technology-based health interventions. The results of this study outline different user requirements related to the design of the mobile patient support app for cancer patients. The results will be used in the iterative development of the Connect Mobile app and can also inform other developers and researchers in development, integration, and evaluation of mobile health apps and services that support cancer patients in managing their health-related issues.

Evaluating Automatic Speech Recognition-Based Language Learning Systems: A Case Study

ERIC Educational Resources Information Center

van Doremalen, Joost; Boves, Lou; Colpaert, Jozef; Cucchiarini, Catia; Strik, Helmer

2016-01-01

The purpose of this research was to evaluate a prototype of an automatic speech recognition (ASR)-based language learning system that provides feedback on different aspects of speaking performance (pronunciation, morphology and syntax) to students of Dutch as a second language. We carried out usability reviews, expert reviews and user tests to…

Using Eye Trackers for Usability Evaluation of Health Information Technology: A Systematic Literature Review

PubMed Central

Yang, Yushi

2015-01-01

Background Eye-tracking technology has been used to measure human cognitive processes and has the potential to improve the usability of health information technology (HIT). However, it is still unclear how the eye-tracking method can be integrated with other traditional usability methodologies to achieve its full potential. Objective The objective of this study was to report on HIT evaluation studies that have used eye-tracker technology, and to envision the potential use of eye-tracking technology in future research. Methods We used four reference databases to initially identify 5248 related papers, which resulted in only 9 articles that met our inclusion criteria. Results Eye-tracking technology was useful in finding usability problems in many ways, but is still in its infancy for HIT usability evaluation. Limited types of HITs have been evaluated by eye trackers, and there has been a lack of evaluation research in natural settings. Conclusions More research should be done in natural settings to discover the real contextual-based usability problems of clinical and mobile HITs using eye-tracking technology with more standardized methodologies and guidance. PMID:27026079

A Cognitive Framework for Understanding Barriers to the Productive Use of a Diabetes Home Telemedicine System

PubMed Central

Kaufman, David R.; Starren, Justin; Patel, Vimla L.; Morin, Philip C.; Hilliman, Charlyn; Pevzner, Jenia; Weinstock, Ruth S.; Goland, Robin; Shea, Steven

2003-01-01

Telemedicine has the potential to transcend geographic and socio-cultural barriers to the delivery of high quality health care to the medically underserved populations. However, there are significant cognitive and usability barriers. This paper presents a multifaceted cognitive evaluation of the IDEATel diabetes education and telemedicine program. The evaluation included a cognitive walkthrough analysis to characterize task complexity and identify potential problems as well as field usability testing in patients’ homes. The study revealed dimensions of the interface that impeded optimal access to system resources. In addition, we found significant obstacles corresponding to perceptual-motoric skills, mental models of the system, and health literacy. The objective of this work is to contribute to a design framework so that participants with a wide range of skills can better manage their chronic illnesses. PMID:14728194

An investigation of the efficacy of collaborative virtual reality systems for moderated remote usability testing.

PubMed

Chalil Madathil, Kapil; Greenstein, Joel S

2017-11-01

Collaborative virtual reality-based systems have integrated high fidelity voice-based communication, immersive audio and screen-sharing tools into virtual environments. Such three-dimensional collaborative virtual environments can mirror the collaboration among usability test participants and facilitators when they are physically collocated, potentially enabling moderated usability tests to be conducted effectively when the facilitator and participant are located in different places. We developed a virtual collaborative three-dimensional remote moderated usability testing laboratory and employed it in a controlled study to evaluate the effectiveness of moderated usability testing in a collaborative virtual reality-based environment with two other moderated usability testing methods: the traditional lab approach and Cisco WebEx, a web-based conferencing and screen sharing approach. Using a mixed methods experimental design, 36 test participants and 12 test facilitators were asked to complete representative tasks on a simulated online shopping website. The dependent variables included the time taken to complete the tasks; the usability defects identified and their severity; and the subjective ratings on the workload index, presence and satisfaction questionnaires. Remote moderated usability testing methodology using a collaborative virtual reality system performed similarly in terms of the total number of defects identified, the number of high severity defects identified and the time taken to complete the tasks with the other two methodologies. The overall workload experienced by the test participants and facilitators was the least with the traditional lab condition. No significant differences were identified for the workload experienced with the virtual reality and the WebEx conditions. However, test participants experienced greater involvement and a more immersive experience in the virtual environment than in the WebEx condition. The ratings for the virtual environment condition were not significantly different from those for the traditional lab condition. The results of this study suggest that participants were productive and enjoyed the virtual lab condition, indicating the potential of a virtual world based approach as an alternative to conventional approaches for synchronous usability testing. Copyright © 2017 Elsevier Ltd. All rights reserved.

Usability Guidelines for Product Recommenders Based on Example Critiquing Research

NASA Astrophysics Data System (ADS)

Pu, Pearl; Faltings, Boi; Chen, Li; Zhang, Jiyong; Viappiani, Paolo

Over the past decade, our group has developed a suite of decision tools based on example critiquing to help users find their preferred products in e-commerce environments. In this chapter, we survey important usability research work relative to example critiquing and summarize the major results by deriving a set of usability guidelines. Our survey is focused on three key interaction activities between the user and the system: the initial preference elicitation process, the preference revision process, and the presentation of the systems recommendation results. To provide a basis for the derivation of the guidelines, we developed a multi-objective framework of three interacting criteria: accuracy, confidence, and effort (ACE). We use this framework to analyze our past work and provide a specific context for each guideline: when the system should maximize its ability to increase users' decision accuracy, when to increase user confidence, and when to minimize the interaction effort for the users. Due to the general nature of this multi-criteria model, the set of guidelines that we propose can be used to ease the usability engineering process of other recommender systems, especially those used in e-commerce environments. The ACE framework presented here is also the first in the field to evaluate the performance of preference-based recommenders from a user-centric point of view.

Accessibility and usability OCW data: The UTPL OCW.

PubMed

Rodríguez, Germania; Perez, Jennifer; Cueva, Samanta; Torres, Rommel

2017-08-01

This data article provides a data description on article entitled "A framework for improving web accessibility and usability of Open Course Ware sites" [3] This Data in Brief presents the data obtained from the accessibility and usability evaluation of the UTPL OCW. The data obtained from the framework evaluation consists of the manual evaluation of the standards criteria and the automatic evaluation of the tools Google PageSpeed and Google Analytics. In addition, this article presents the synthetized tables from standards that are used by the framework to evaluate the accessibility and usability of OCW, and the questionnaires required to extract the data. As a result, the article also provides the data required to reproduce the evaluation of other OCW.

Testing the usability of two automated home-based patient-management systems.

PubMed

Farzanfar, Ramesh; Finkelstein, Joseph; Friedman, Robert H

2004-04-01

To explore to what extent observation and semistructured in-depth interviews provide effective tools for usability testing of two automated home-based systems aimed at monitoring patients' health status at home and improving self-care. Telephone-Linked Care for Diet Adherence in Dyslipidemia (TLC-DietAid) used computer telephony to interact with users and Home Asthma Telemonitoring System (HAT System) used a combination of Personal Digital Assistant (palmtops) and the Internet for similar purposes. Both systems were evaluated in two separate pilot studies. Our pilot studies uncovered "medium-specific" and "content-specific" issues that addressed either the process of the interaction or its content. The results demonstrated that patient-users tended to evaluate each system on the basis of how it fit into everyday life and corresponded to personal preferences. The methodology also allowed the system designers to understand users' concerns and the context of adoption in order to introduce necessary changes to the design to address such concerns.

Usability and feasibility of mobile phone diaries in an experimental physical exercise study.

PubMed

Heinonen, Reetta; Luoto, Riitta; Lindfors, Pirjo; Nygård, Clas-Håkan

2012-03-01

Wireless and mobile phone technology as a method of data collection will increase alongside conventional methods. The aim of the present study was to evaluate the use of a mobile phone application for recording symptoms and physical activity exertion during an experimental physical exercise study. An experimental study on the effects of physical exercise on the well-being of menopausal women included 158 subjects between 44 and 63 years of age. The women were randomized into intervention and control groups. All participants in both groups reported daily symptoms by responding to morning and evening questionnaires via mobile phones. The usability of the mobile phone as a data collector tool was evaluated with the System Usability Scale (SUS) questionnaire 2 months after the intervention. The feasibility evaluation was based on the frequency of responses and open questions. The response rates were about 70% to both morning and evening questionnaires. The average frequency of responses (n =158) to morning questionnaires was 125 (±40)/170 (±14) and to evening questionnaires was 118 (±40)/171 (±14). The response rate did not differ between the intervention and the control groups. The SUS score was on average 75.4 (range, 0-100; n =107). A mobile phone diary is a feasible and usable tool for data collection in clinical trials.

Health figures: an open source JavaScript library for health data visualization.

PubMed

Ledesma, Andres; Al-Musawi, Mohammed; Nieminen, Hannu

2016-03-22

The way we look at data has a great impact on how we can understand it, particularly when the data is related to health and wellness. Due to the increased use of self-tracking devices and the ongoing shift towards preventive medicine, better understanding of our health data is an important part of improving the general welfare of the citizens. Electronic Health Records, self-tracking devices and mobile applications provide a rich variety of data but it often becomes difficult to understand. We implemented the hFigures library inspired on the hGraph visualization with additional improvements. The purpose of the library is to provide a visual representation of the evolution of health measurements in a complete and useful manner. We researched the usefulness and usability of the library by building an application for health data visualization in a health coaching program. We performed a user evaluation with Heuristic Evaluation, Controlled User Testing and Usability Questionnaires. In the Heuristics Evaluation the average response was 6.3 out of 7 points and the Cognitive Walkthrough done by usability experts indicated no design or mismatch errors. In the CSUQ usability test the system obtained an average score of 6.13 out of 7, and in the ASQ usability test the overall satisfaction score was 6.64 out of 7. We developed hFigures, an open source library for visualizing a complete, accurate and normalized graphical representation of health data. The idea is based on the concept of the hGraph but it provides additional key features, including a comparison of multiple health measurements over time. We conducted a usability evaluation of the library as a key component of an application for health and wellness monitoring. The results indicate that the data visualization library was helpful in assisting users in understanding health data and its evolution over time.

Heuristics to Evaluate Interactive Systems for Children with Autism Spectrum Disorder (ASD).

PubMed

Khowaja, Kamran; Salim, Siti Salwah; Asemi, Adeleh

2015-01-01

In this paper, we adapted and expanded a set of guidelines, also known as heuristics, to evaluate the usability of software to now be appropriate for software aimed at children with autism spectrum disorder (ASD). We started from the heuristics developed by Nielsen in 1990 and developed a modified set of 15 heuristics. The first 5 heuristics of this set are the same as those of the original Nielsen set, the next 5 heuristics are improved versions of Nielsen's, whereas the last 5 heuristics are new. We present two evaluation studies of our new heuristics. In the first, two groups compared Nielsen's set with the modified set of heuristics, with each group evaluating two interactive systems. The Nielsen's heuristics were assigned to the control group while the experimental group was given the modified set of heuristics, and a statistical analysis was conducted to determine the effectiveness of the modified set, the contribution of 5 new heuristics and the impact of 5 improved heuristics. The results show that the modified set is significantly more effective than the original, and we found a significant difference between the five improved heuristics and their corresponding heuristics in the original set. The five new heuristics are effective in problem identification using the modified set. The second study was conducted using a system which was developed to ascertain if the modified set was effective at identifying usability problems that could be fixed before the release of software. The post-study analysis revealed that the majority of the usability problems identified by the experts were fixed in the updated version of the system.

Usability and Utility of a Mobile Application for Marksmanship Training

DTIC Science & Technology

2015-01-01

Right of Canada, as represented by the Minister of National Defence, 2015 © Sa Majesté la Reine (en droit du Canada), telle que représentée par le...Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574-594. BenMoussa, C. (2003). Workers on...Behavioral and Social Sciences. Jumisko-Pyykko, S., & Vainio, T. (2011). Framing the Context of Use for Mobile HCI. International Journal of Mobile

A Usability Evaluation of Academic Virtual Reference Services

ERIC Educational Resources Information Center

Chow, Anthony S.; Croxton, Rebecca A.

2014-01-01

This study examined the usability of five virtual reference services--instant messenger chat, e-mail, telephone, text messaging, and Skype videoconferencing--by having 31 undergraduate and graduate students evaluate the usability of the virtual reference services of two different universities. The study's results suggest that user preference and…

Usability Assessment of E-Café Operational Management Simulation Game

ERIC Educational Resources Information Center

Chang, Chiung-sui; Huang, Ya-Ping

2013-01-01

To ensure the quality of digital simulation game, we utilized the usability evaluation heuristic in the design and development processes of e-café operational management game-based learning material for students. The application of usability evaluations during this study is described. Additionally, participant selection, data collection and…

An Evaluation of the Usability of a Computerized Decision Support System for Nursing Homes

PubMed Central

Fossum, M.; Ehnfors, M.; Fruhling, A.; Ehrenberg, A.

2011-01-01

Background Computerized decision support systems (CDSSs) have the potential to significantly improve the quality of nursing care of older people by enhancing the decision making of nursing personnel. Despite this potential, health care organizations have been slow to incorporate CDSSs into nursing home practices. Objective This study describes facilitators and barriers that impact the ability of nursing personnel to effectively use a clinical CDSS for planning and treating pressure ulcers (PUs) and malnutrition and for following the suggested risk assessment guidelines for the care of nursing home residents. Methods We employed a qualitative descriptive design using varied methods, including structured group interviews, cognitive walkthrough observations and a graphical user interface (GUI) usability evaluation. Group interviews were conducted with 25 nursing personnel from four nursing homes in southern Norway. Five nursing personnel participated in cognitive walkthrough observations and the GUI usability evaluation. Text transcripts were analyzed using qualitative content analysis. Results Group interview participants reported that ease of use, usefulness and a supportive work environment were key facilitators of CDSS use. The barriers identified were lack of training, resistance to using computers and limited integration of the CDSS with the facility’s electronic health record (EHR) system. Key findings from the usability evaluation also identified the difficulty of using the CDSS within the EHR and the poorly designed GUI integration as barriers. Conclusion Overall, we found disconnect between two types of nursing personnel. Those who were comfortable with computer technology reported positive feedback about the CDSS, while others expressed resistance to using the CDSS for various reasons. This study revealed that organizations must invest more resources in educating nursing personnel on the seriousness of PUs and poor nutrition in the elderly, providing specialized CDSS training and ensuring that nursing personnel have time in the workday to use the CDSS. PMID:23616886

Usability Study and Heuristic Evaluation of the Applied Robotics for Installations and Base Operations (ARIBO) Driverless Vehicle Reservation Application ARIBO Mobile

DTIC Science & Technology

2017-03-01

ARL-TN-0814 ● MAR 2017 US Army Research Laboratory Usability Study and Heuristic Evaluation of the Applied Robotics for...ARL-TN-0814 ● MAR 2017 US Army Research Laboratory Usability Study and Heuristic Evaluation of the Applied Robotics for...Heuristic Evaluation of the Applied Robotics for Installations and Base Operations (ARIBO) Driverless Vehicle Reservation Application ARIBO Mobile 5a

Translation, Cross-Cultural Adaptation, and Validation of the Malay Version of the System Usability Scale Questionnaire for the Assessment of Mobile Apps.

PubMed

Mohamad Marzuki, Muhamad Fadhil; Yaacob, Nor Azwany; Yaacob, Najib Majdi

2018-05-14

A mobile app is a programmed system designed to be used by a target user on a mobile device. The usability of such a system refers not only to the extent to which product can be used to achieve the task that it was designed for, but also its effectiveness and efficiency, as well as user satisfaction. The System Usability Scale is one of the most commonly used questionnaires used to assess the usability of a system. The original 10-item version of System Usability Scale was developed in English and thus needs to be adapted into local languages to assess the usability of a mobile apps developed in other languages. The aim of this study is to translate and validate (with cross-cultural adaptation) the English System Usability Scale questionnaire into Malay, the main language spoken in Malaysia. The development of a translated version will allow the usability of mobile apps to be assessed in Malay. Forward and backward translation of the questionnaire was conducted by groups of Malay native speakers who spoke English as their second language. The final version was obtained after reconciliation and cross-cultural adaptation. The content of the Malay System Usability Scale questionnaire for mobile apps was validated by 10 experts in mobile app development. The efficacy of the questionnaire was further probed by testing the face validity on 10 mobile phone users, followed by reliability testing involving 54 mobile phone users. The content validity index was determined to be 0.91, indicating good relevancy of the 10 items used to assess the usability of a mobile app. Calculation of the face validity index resulted in a value of 0.94, therefore indicating that the questionnaire was easily understood by the users. Reliability testing showed a Cronbach alpha value of .85 (95% CI 0.79-0.91) indicating that the translated System Usability Scale questionnaire is a reliable tool for the assessment of usability of a mobile app. The Malay System Usability Scale questionnaire is a valid and reliable tool to assess the usability of mobile app in Malaysia. ©Muhamad Fadhil Mohamad Marzuki, Nor Azwany Yaacob, Najib Majdi Yaacob. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 14.05.2018.

Developing an interactive mobile phone self-report system for self-management of hypertension. Part 2: content validity and usability.

PubMed

Bengtsson, Ulrika; Kjellgren, Karin; Höfer, Stefan; Taft, Charles; Ring, Lena

2014-10-01

Self-management support tools using technology may improve adherence to hypertension treatment. There is a need for user-friendly tools facilitating patients' understanding of the interconnections between blood pressure, wellbeing and lifestyle. This study aimed to examine comprehension, comprehensiveness and relevance of items, and further to evaluate the usability and reliability of an interactive hypertension-specific mobile phone self-report system. Areas important in supporting self-management and candidate items were derived from five focus group interviews with patients and healthcare professionals (n = 27), supplemented by a literature review. Items and response formats were drafted to meet specifications for mobile phone administration and were integrated into a mobile phone data-capture system. Content validity and usability were assessed iteratively in four rounds of cognitive interviews with patients (n = 21) and healthcare professionals (n = 4). Reliability was examined using a test-retest. Focus group analyses yielded six areas covered by 16 items. The cognitive interviews showed satisfactory item comprehension, relevance and coverage; however, one item was added. The mobile phone self-report system was reliable and perceived easy to use. The mobile phone self-report system appears efficiently to capture information relevant in patients' self-management of hypertension. Future studies need to evaluate the effectiveness of this tool in improving self-management of hypertension in clinical practice.

How Information Visualization Systems Change Users' Understandings of Complex Data

ERIC Educational Resources Information Center

Allendoerfer, Kenneth Robert

2009-01-01

User-centered evaluations of information systems often focus on the usability of the system rather its usefulness. This study examined how a using an interactive knowledge-domain visualization (KDV) system affected users' understanding of a domain. Interactive KDVs allow users to create graphical representations of domains that depict important…

Usability Evaluation of an Augmented Reality System for Teaching Euclidean Vectors

ERIC Educational Resources Information Center

Martin-Gonzalez, Anabel; Chi-Poot, Angel; Uc-Cetina, Victor

2016-01-01

Augmented reality (AR) is one of the emerging technologies that has demonstrated to be an efficient technological tool to enhance learning techniques. In this paper, we describe the development and evaluation of an AR system for teaching Euclidean vectors in physics and mathematics. The goal of this pedagogical tool is to facilitate user's…

In the Loop: The Organization of Team-Based Communication in a Patient-Centered Clinical Collaboration System.

PubMed

Kurahashi, Allison M; Weinstein, Peter B; Jamieson, Trevor; Stinson, Jennifer N; Cafazzo, Joseph A; Lokuge, Bhadra; Morita, Plinio P; Cohen, Eyal; Rapoport, Adam; Bezjak, Andrea; Husain, Amna

2016-03-24

We describe the development and evaluation of a secure Web-based system for the purpose of collaborative care called Loop. Loop assembles the team of care with the patient as an integral member of the team in a secure space. The objectives of this paper are to present the iterative design of the separate views for health care providers (HCPs) within each patient's secure space and examine patients', caregivers', and HCPs' perspectives on this separate view for HCP-only communication. The overall research program includes cycles of ethnography, prototyping, usability testing, and pilot testing. This paper describes the usability testing phase that directly informed development. A descriptive qualitative approach was used to analyze participant perspectives that emerged during usability testing. During usability testing, we sampled 89 participants from three user groups: 23 patients, 19 caregivers, and 47 HCPs. Almost all perspectives from the three user groups supported the need for an HCP-only communication view. In an earlier prototype, the visual presentation caused confusion among HCPs when reading and composing messages about whether a message was visible to the patient. Usability testing guided us to design a more deliberate distinction between posting in the Patient and Team view and the Health Care Provider Only view at the time of composing a message, which once posted is distinguished by an icon. The team made a decision to incorporate an HCP-only communication view based on findings during earlier phases of work. During usability testing we tested the separate communication views, and all groups supported this partition. We spent considerable effort designing the partition; however, preliminary findings from the next phase of evaluation, pilot testing, show that the Patient and Team communication is predominantly being used. This demonstrates the importance of a subsequent phase of the clinical trial of Loop to validate the concept and design.

Identification of design features to enhance utilization and acceptance of systems for Internet-based decision support at the point of care.

PubMed

Gadd, C S; Baskaran, P; Lobach, D F

1998-01-01

Extensive utilization of point-of-care decision support systems will be largely dependent on the development of user interaction capabilities that make them effective clinical tools in patient care settings. This research identified critical design features of point-of-care decision support systems that are preferred by physicians, through a multi-method formative evaluation of an evolving prototype of an Internet-based clinical decision support system. Clinicians used four versions of the system--each highlighting a different functionality. Surveys and qualitative evaluation methodologies assessed clinicians' perceptions regarding system usability and usefulness. Our analyses identified features that improve perceived usability, such as telegraphic representations of guideline-related information, facile navigation, and a forgiving, flexible interface. Users also preferred features that enhance usefulness and motivate use, such as an encounter documentation tool and the availability of physician instruction and patient education materials. In addition to identifying design features that are relevant to efforts to develop clinical systems for point-of-care decision support, this study demonstrates the value of combining quantitative and qualitative methods of formative evaluation with an iterative system development strategy to implement new information technology in complex clinical settings.

A usability evaluation exploring the design of American Nurses Association state web sites.

PubMed

Alexander, Gregory L; Wakefield, Bonnie J; Anbari, Allison B; Lyons, Vanessa; Prentice, Donna; Shepherd, Marilyn; Strecker, E Bradley; Weston, Marla J

2014-08-01

National leaders are calling for opportunities to facilitate the Future of Nursing. Opportunities can be encouraged through state nurses association Web sites, which are part of the American Nurses Association, that are well designed, with appropriate content, and in a language professional nurses understand. The American Nurses Association and constituent state nurses associations provide information about nursing practice, ethics, credentialing, and health on Web sites. We conducted usability evaluations to determine compliance with heuristic and ethical principles for Web site design. We purposefully sampled 27 nursing association Web sites and used 68 heuristic and ethical criteria to perform systematic usability assessments of nurse association Web sites. Web site analysis included seven double experts who were all RNs trained in usability analysis. The extent to which heuristic and ethical criteria were met ranged widely from one state that met 0% of the criteria for "help and documentation" to states that met greater than 92% of criteria for "visibility of system status" and "aesthetic and minimalist design." Suggested improvements are simple yet make an impact on a first-time visitor's impression of the Web site. For example, adding internal navigation and tracking features and providing more details about the application process through help and frequently asked question documentation would facilitate better use. Improved usability will improve effectiveness, efficiency, and consumer satisfaction with these Web sites.

An Evaluation of the Design and Usability of a Novel Robotic Bilateral Arm Rehabilitation Device for Patients with Stroke.

PubMed

Pei, Yu-Cheng; Chen, Jean-Lon; Wong, Alice M K; Tseng, Kevin C

2017-01-01

Case series. IV (case series). Robot-assisted therapy for upper limb rehabilitation is an emerging research topic and its design process must integrate engineering, neurological pathophysiology, and clinical needs. This study developed/evaluated the usefulness of a novel rehabilitation device, the MirrorPath , designed for the upper limb rehabilitation of patients with hemiplegic stroke. The process follows Tseng's methodology for innovative product design and development, namely two stages, device development and usability assessment. During the development process, the design was guided by patients' rehabilitation needs as defined by patients and their therapists. The design applied synchronic movement of the bilateral upper limbs, an approach that is compatible with the bilateral movement therapy and proprioceptive neuromuscular facilitation theories. MirrorPath consists of a robotic device that guides upper limb movement linked to a control module containing software controlling the robotic movement. Five healthy subjects were recruited in the pretest, and 4 patients, 4 caregivers, and 4 therapists were recruited in the formal test for usability. All recruited subjects were allocated to the test group, completed the evaluation, and their data were all analyzed. The total system usability scale score obtained from the patients, caregivers, and therapists was 71.8 ± 11.9, indicating a high level of usability and product acceptance. Following a standard development process, we could yield a design that meets clinical needs. This low-cost device provides a feasible platform for carrying out robot-assisted bilateral movement therapy of patients with hemiplegic stroke. identifier NCT02698605.

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results

PubMed Central

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-01-01

Background In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. Objective In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. Methods The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users’ perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). Results The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in ‘Quality of Work Life’, ‘Perceived Usefulness’, ‘Perceived Ease of Use’, and ‘User Control’, respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Conclusions The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. PMID:24567081

Testing the Feasibility and Usability of a Novel Smartphone-Based Self-Management Support System for Dialysis Patients: A Pilot Study.

PubMed

Hayashi, Aki; Yamaguchi, Satoko; Waki, Kayo; Fujiu, Katsuhito; Hanafusa, Norio; Nishi, Takahiro; Tomita, Hyoe; Kobayashi, Haruka; Fujita, Hideo; Kadowaki, Takashi; Nangaku, Masaomi; Ohe, Kazuhiko

2017-04-20

Diet and fluid restrictions that need continuous self-management are among the most difficult aspects of dialysis treatment. Smartphone applications may be useful for supporting self-management. Our objective is to investigate the feasibility and usability of a novel smartphone-based self-management support system for dialysis patients. We developed the Self-Management and Recording System for Dialysis (SMART-D), which supports self-monitoring of three mortality-related factors that can be modified by lifestyle: interdialytic weight gain and predialysis serum potassium and phosphorus concentrations. Data is displayed graphically, with all data evaluated automatically to determine whether they achieve the values suggested by the Japanese Society for Dialysis Therapy guidelines. In a pilot study, 9 dialysis patients used SMART-D system for 2 weeks. A total of 7 of them completed questionnaires rating their assessment of SMART-D's usability and their satisfaction with the system. In addition, the Kidney Disease Quality of Life scale was compared before and after the study period. All 9 participants were able to use SMART-D with no major problems. Completion rates for body weight, pre- and postdialysis weight, and serum potassium and phosphorus concentrations were, respectively, 89% (SD 23), 95% (SD 7), and 78% (SD 44). Of the 7 participants who completed the usability survey, all were motivated by the sense of security derived from using the system, and 6 of the 7 (86%) reported that using SMART-D helped improve their lifestyle and self-management. Using SMART-D was feasible, and the system was well regarded by patients. Further study with larger scale cohorts and longer study and follow-up periods is needed to evaluate the effects of SMART-D on clinical outcomes and quality of life. ©Aki Hayashi, Satoko Yamaguchi, Kayo Waki, Katsuhito Fujiu, Norio Hanafusa, Takahiro Nishi, Hyoe Tomita, Haruka Kobayashi, Hideo Fujita, Takashi Kadowaki, Masaomi Nangaku, Kazuhiko Ohe. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 20.04.2017.

Testing the Feasibility and Usability of a Novel Smartphone-Based Self-Management Support System for Dialysis Patients: A Pilot Study

PubMed Central

Fujiu, Katsuhito; Hanafusa, Norio; Nishi, Takahiro; Tomita, Hyoe; Kobayashi, Haruka; Fujita, Hideo; Kadowaki, Takashi; Nangaku, Masaomi; Ohe, Kazuhiko

2017-01-01

Background Diet and fluid restrictions that need continuous self-management are among the most difficult aspects of dialysis treatment. Smartphone applications may be useful for supporting self-management. Objective Our objective is to investigate the feasibility and usability of a novel smartphone-based self-management support system for dialysis patients. Methods We developed the Self-Management and Recording System for Dialysis (SMART-D), which supports self-monitoring of three mortality-related factors that can be modified by lifestyle: interdialytic weight gain and predialysis serum potassium and phosphorus concentrations. Data is displayed graphically, with all data evaluated automatically to determine whether they achieve the values suggested by the Japanese Society for Dialysis Therapy guidelines. In a pilot study, 9 dialysis patients used SMART-D system for 2 weeks. A total of 7 of them completed questionnaires rating their assessment of SMART-D’s usability and their satisfaction with the system. In addition, the Kidney Disease Quality of Life scale was compared before and after the study period. Results All 9 participants were able to use SMART-D with no major problems. Completion rates for body weight, pre- and postdialysis weight, and serum potassium and phosphorus concentrations were, respectively, 89% (SD 23), 95% (SD 7), and 78% (SD 44). Of the 7 participants who completed the usability survey, all were motivated by the sense of security derived from using the system, and 6 of the 7 (86%) reported that using SMART-D helped improve their lifestyle and self-management. Conclusions Using SMART-D was feasible, and the system was well regarded by patients. Further study with larger scale cohorts and longer study and follow-up periods is needed to evaluate the effects of SMART-D on clinical outcomes and quality of life. PMID:28428168

Development of a Video Coding Scheme for Analyzing the Usability and Usefulness of Health Information Systems.

PubMed

Kushniruk, Andre W; Borycki, Elizabeth M

2015-01-01

Usability has been identified as a key issue in health informatics. Worldwide numerous projects have been carried out in an attempt to increase and optimize health system usability. Usability testing, involving observing end users interacting with systems, has been widely applied and numerous publications have appeared describing such studies. However, to date, fewer works have been published describing methodological approaches to analyzing the rich data stream that results from usability testing. This includes analysis of video, audio and screen recordings. In this paper we describe our work in the development and application of a coding scheme for analyzing the usability of health information systems. The phases involved in such analyses are described.

Usability study of the EduMod eLearning Program for contouring nodal stations of the head and neck.

PubMed

Deraniyagala, Rohan; Amdur, Robert J; Boyer, Arthur L; Kaylor, Scott

2015-01-01

A major strategy for improving radiation oncology education and competence evaluation is to develop eLearning programs that reproduce the real work environment. A valuable measure of the quality of an eLearning program is "usability," which is a multidimensional endpoint defined from the end user's perspective. The gold standard for measuring usability is the Software Usability Measurement Inventory (SUMI). The purpose of this study is to use the SUMI to measure usability of an eLearning course that uses innovative software to teach and test contouring of nodal stations of the head and neck. This is a prospective institutional review board-approved study in which all participants gave written informed consent. The study population was radiation oncology residents from 8 different programs across the United States. The subjects had to pass all sections of the same 2 eLearning modules and then complete the SUMI usability evaluation instrument. We reached the accrual goal of 25 participants. Usability results for the EduMod eLearning course, "Nodal Stations of the Head and Neck," were compared with a large database of scores of other major software programs. Results were evaluated in 5 domains: Affect, Helpfulness, Control, Learnability, and Global Usability. In all 5 domains, usability scores for the study modules were higher than the database mean and statistically superior in 4 domains. This is the first study to evaluate usability of an eLearning program related to radiation oncology. Usability of 2 representative modules related to contouring nodal stations of the head and neck was highly favorable, with scores that were superior to the industry standard in multiple domains. These results support the continued development of this type of eLearning program for teaching and testing radiation oncology technical skills. Copyright © 2015 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.

Using smartphone technology to deliver a virtual pedestrian environment: usability and validation.

PubMed

Schwebel, David C; Severson, Joan; He, Yefei

2017-09-01

Various programs effectively teach children to cross streets more safely, but all are labor- and cost-intensive. Recent developments in mobile phone technology offer opportunity to deliver virtual reality pedestrian environments to mobile smartphone platforms. Such an environment may offer a cost- and labor-effective strategy to teach children to cross streets safely. This study evaluated usability, feasibility, and validity of a smartphone-based virtual pedestrian environment. A total of 68 adults completed 12 virtual crossings within each of two virtual pedestrian environments, one delivered by smartphone and the other a semi-immersive kiosk virtual environment. Participants completed self-report measures of perceived realism and simulator sickness experienced in each virtual environment, plus self-reported demographic and personality characteristics. All participants followed system instructions and used the smartphone-based virtual environment without difficulty. No significant simulator sickness was reported or observed. Users rated the smartphone virtual environment as highly realistic. Convergent validity was detected, with many aspects of pedestrian behavior in the smartphone-based virtual environment matching behavior in the kiosk virtual environment. Anticipated correlations between personality and kiosk virtual reality pedestrian behavior emerged for the smartphone-based system. A smartphone-based virtual environment can be usable and valid. Future research should develop and evaluate such a training system.

76 FR 8715 - Proposed Collection; Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2011-02-15

.... Title; Associated Form; and OMB Number: Electronic Health Record (EHR) Usability Survey; OMB Control... longitudinal measure of how end-users perceive the usability of the Department of Defense (DoD) suite of... usability of a system. This usability attribute describes the ease with which people can use the system to...

Sensor-oriented feature usability evaluation in fingerprint segmentation

NASA Astrophysics Data System (ADS)

Li, Ying; Yin, Yilong; Yang, Gongping

2013-06-01

Existing fingerprint segmentation methods usually process fingerprint images captured by different sensors with the same feature or feature set. We propose to improve the fingerprint segmentation result in view of an important fact that images from different sensors have different characteristics for segmentation. Feature usability evaluation, which means to evaluate the usability of features to find the personalized feature or feature set for different sensors to improve the performance of segmentation. The need for feature usability evaluation for fingerprint segmentation is raised and analyzed as a new issue. To address this issue, we present a decision-tree-based feature-usability evaluation method, which utilizes a C4.5 decision tree algorithm to evaluate and pick the best suitable feature or feature set for fingerprint segmentation from a typical candidate feature set. We apply the novel method on the FVC2002 database of fingerprint images, which are acquired by four different respective sensors and technologies. Experimental results show that the accuracy of segmentation is improved, and time consumption for feature extraction is dramatically reduced with selected feature(s).

Usability and acceptability of balance exergames in older adults: A scoping review.

PubMed

Nawaz, Ather; Skjæret, Nina; Helbostad, Jorunn Lægdheim; Vereijken, Beatrix; Boulton, Elisabeth; Svanaes, Dag

2016-12-01

Serious games (exergames) have the potential to be effective for postural balance and increasing muscle strength. Several games have been developed to increase physical fitness and balance among older adults. However, it is unclear to which degree usability and acceptability of exergames for older adults have been evaluated. The aim of this study was to summarize usability evaluation and acceptability of studies in older adults. We conducted a scoping review on studies focusing on usability of exergames for older adults. The result shows that older adults consider usability and acceptability of exercise video games good. The review shows that longitudinal studies mainly use off-the-shelf exergame and evaluated game effectiveness and acceptability, whereas cross-sectional studies focus on interactional experience. Studies varied in their approaches to measure usability and acceptability of exergames for older adults. There is a need for a systematic developmental approach to involve older adults in development of exergames for longitudinal studies. © The Author(s) 2015.

Web usability evaluation on BloobIS website by using hallway usability testing method and ISO 9241:11

NASA Astrophysics Data System (ADS)

Dwi Susanto, Tony; Ingesti Prasetyo, Anisa; Astuti, Hanim Maria

2018-03-01

At this moment, the need for web as an information media is highly important. Not only confined in the infotainment area, government, and education, but health as well uses the web media as a medium for providing information effectively. BloobIS is a web based application which integrates blood supply and distribution information at the Blood Transfusion Unit. Knowing how easy information is on BloobIS is marked by how convenient the website is used. Up until now, the BloobIS website is nearing completion but testing has not yet been performed to users and is on the testing and development phase in the Development Life Cycle software. Thus, an evaluation namely the quality control software which focuses on the perspective of BloobIs web usability is required. Hallway Usability Testing and ISO 9241:11 are the methods chosen to measure the BloobIS application usability. The expected outputs of the quality control software focusing on the usability evaluation are being able to rectify the usability deficiencies on the BloobIs web and provide recommendations to develop the web as a basic BloobIS web quality upgraed which sets a goal to amplify the satisfaction of web users based on usability factors in ISO 9241:11.

Museum Personnel's Opinions on Mobile Guidance Systems

ERIC Educational Resources Information Center

Yoshimura, Hirokazu; Sekiguchi, Hiromi; Yabumoto, Yoshitaka

2007-01-01

While opinions from the general public are certainly important, opinions from the museum staff are also necessary to improve user service systems. this article introduces two groups of museum staff who have evaluated the usability of mobile guidance systems in Japanese museums. One group is the research team who used the PDA system in the National…

Applying Human Factors Evaluation and Design Guidance to a Nuclear Power Plant Digital Control System

DOE Office of Scientific and Technical Information (OSTI.GOV)

Thomas Ulrich; Ronald Boring; William Phoenix

2012-08-01

The United States (U.S.) nuclear industry, like similar process control industries, has moved toward upgrading its control rooms. The upgraded control rooms typically feature digital control system (DCS) displays embedded in the panels. These displays gather information from the system and represent that information on a single display surface. In this manner, the DCS combines many previously separate analog indicators and controls into a single digital display, whereby the operators can toggle between multiple windows to monitor and control different aspects of the plant. The design of the DCS depends on the function of the system it monitors, but revolvesmore » around presenting the information most germane to an operator at any point in time. DCSs require a carefully designed human system interface. This report centers on redesigning existing DCS displays for an example chemical volume control system (CVCS) at a U.S. nuclear power plant. The crucial nature of the CVCS, which controls coolant levels and boration in the primary system, requires a thorough human factors evaluation of its supporting DCS. The initial digital controls being developed for the DCSs tend to directly mimic the former analog controls. There are, however, unique operator interactions with a digital vs. analog interface, and the differences have not always been carefully factored in the translation of an analog interface to a replacement DCS. To ensure safety, efficiency, and usability of the emerging DCSs, a human factors usability evaluation was conducted on a CVCS DCS currently being used and refined at an existing U.S. nuclear power plant. Subject matter experts from process control engineering, software development, and human factors evaluated the DCS displays to document potential usability issues and propose design recommendations. The evaluation yielded 167 potential usability issues with the DCS. These issues should not be considered operator performance problems but rather opportunities identified by experts to improve upon the design of the DCS. A set of nine design recommendations was developed to address these potential issues. The design principles addressed the following areas: (1) color, (2) pop-up window structure, (3) navigation, (4) alarms, (5) process control diagram, (6) gestalt grouping, (7) typography, (8) terminology, and (9) data entry. Visuals illustrating the improved DCS displays accompany the design recommendations. These nine design principles serve as the starting point to a planned general DCS style guide that can be used across the U.S. nuclear industry to aid in the future design of effective DCS interfaces.« less

Heuristics to Evaluate Interactive Systems for Children with Autism Spectrum Disorder (ASD)

PubMed Central

Khowaja, Kamran; Salim, Siti Salwah

2015-01-01

In this paper, we adapted and expanded a set of guidelines, also known as heuristics, to evaluate the usability of software to now be appropriate for software aimed at children with autism spectrum disorder (ASD). We started from the heuristics developed by Nielsen in 1990 and developed a modified set of 15 heuristics. The first 5 heuristics of this set are the same as those of the original Nielsen set, the next 5 heuristics are improved versions of Nielsen's, whereas the last 5 heuristics are new. We present two evaluation studies of our new heuristics. In the first, two groups compared Nielsen’s set with the modified set of heuristics, with each group evaluating two interactive systems. The Nielsen’s heuristics were assigned to the control group while the experimental group was given the modified set of heuristics, and a statistical analysis was conducted to determine the effectiveness of the modified set, the contribution of 5 new heuristics and the impact of 5 improved heuristics. The results show that the modified set is significantly more effective than the original, and we found a significant difference between the five improved heuristics and their corresponding heuristics in the original set. The five new heuristics are effective in problem identification using the modified set. The second study was conducted using a system which was developed to ascertain if the modified set was effective at identifying usability problems that could be fixed before the release of software. The post-study analysis revealed that the majority of the usability problems identified by the experts were fixed in the updated version of the system. PMID:26196385

Enhancing Learning Management Systems Utility for Blind Students: A Task-Oriented, User-Centered, Multi-Method Evaluation Technique

ERIC Educational Resources Information Center

Babu, Rakesh; Singh, Rahul

2013-01-01

This paper presents a novel task-oriented, user-centered, multi-method evaluation (TUME) technique and shows how it is useful in providing a more complete, practical and solution-oriented assessment of the accessibility and usability of Learning Management Systems (LMS) for blind and visually impaired (BVI) students. Novel components of TUME…

To twist, roll, stroke or poke? A study of input devices for menu navigation in the cockpit.

PubMed

Stanton, Neville A; Harvey, Catherine; Plant, Katherine L; Bolton, Luke

2013-01-01

Modern interfaces within the aircraft cockpit integrate many flight management system (FMS) functions into a single system. The success of a user's interaction with an interface depends upon the optimisation between the input device, tasks and environment within which the system is used. In this study, four input devices were evaluated using a range of Human Factors methods, in order to assess aspects of usability including task interaction times, error rates, workload, subjective usability and physical discomfort. The performance of the four input devices was compared using a holistic approach and the findings showed that no single input device produced consistently high performance scores across all of the variables evaluated. The touch screen produced the highest number of 'best' scores; however, discomfort ratings for this device were high, suggesting that it is not an ideal solution as both physical and cognitive aspects of performance must be accounted for in design. This study evaluated four input devices for control of a screen-based flight management system. A holistic approach was used to evaluate both cognitive and physical performance. Performance varied across the dependent variables and between the devices; however, the touch screen produced the largest number of 'best' scores.

Supporting Cancer Patients in Illness Management: Usability Evaluation of a Mobile App

PubMed Central

Kaufman, David R; Ruland, Cornelia M

2014-01-01

Background Mobile phones and tablets currently represent a significant presence in people’s everyday lives. They enable access to different information and services independent of current place and time. Such widespread connectivity offers significant potential in different app areas including health care. Objective Our goal was to evaluate the usability of the Connect Mobile app. The mobile app enables mobile access to the Connect system, an online system that supports cancer patients in managing health-related issues. Along with symptom management, the system promotes better patient-provider communication, collaboration, and shared decision making. The Connect Mobile app enables access to the Connect system over both mobile phones and tablets. Methods The study consisted of usability tests of a high fidelity prototype with 7 cancer patients where the objectives were to identify existing design and functionality issues and to provide patients with a real look-and-feel of the mobile system. In addition, we conducted semistructured interviews to obtain participants’ feedback about app usefulness, identify the need for new system features and design requirements, and measure the acceptance of the mobile app and its features within everyday health management. Results The study revealed a total of 27 design issues (13 for mobile apps and 14 for tablet apps), which were mapped to source events (ie, errors, requests for help, participants' concurrent feedback, and moderator observation). We also applied usability heuristics to identify violations of usability principles. The majority of violations were related to enabling ease of input, screen readability, and glanceability (15 issues), as well as supporting an appropriate match between systems and the real world (7 issues) and consistent mapping of system functions and interactions (4 issues). Feedback from participants also showed the cancer patients’ requirements for support systems and how these needs are influenced by different context-related factors, such as type of access terminal (eg, desktop computer, tablet, mobile phone) and phases of illness. Based on the observed results, we proposed design and functionality recommendations that can be used for the development of mobile apps for cancer patients to support their health management process. Conclusions Understanding and addressing users’ requirements is one of the main prerequisites for developing useful and effective technology-based health interventions. The results of this study outline different user requirements related to the design of the mobile patient support app for cancer patients. The results will be used in the iterative development of the Connect Mobile app and can also inform other developers and researchers in development, integration, and evaluation of mobile health apps and services that support cancer patients in managing their health-related issues. PMID:25119490

How Do Clinical Information Systems Affect the Cognitive Demands of General Practitioners?: Usability Study with a Focus on Cognitive Workload.

PubMed

Ariza, Ferran; Kalra, Dipak; Potts, Henry Ww

2015-11-20

Clinical information systems in the National Health Service do not need to conform to any explicit usability requirements. Poor usability can increase the mental workload experienced by clinicians and cause fatigue, increase error rates and impact the overall patient safety. Mental workload can be used as a measure of usability. To assess the subjective cognitive workload experienced by general practitioners (GPs) with their systems. To raise awareness of the importance of usability in system design among users, designers, developers and policymakers. We used a modified version of the NASA Task Load Index, adapted for web. We developed a set of common clinical scenarios and computer tasks on an online survey. We emailed the study link to 199 clinical commissioning groups and 1,646 GP practices in England. Sixty-seven responders completed the survey. The respondents had spent an average of 17 years in general practice, had experience of using a mean of 1.5 GP computer systems and had used their current system for a mean time of 6.7 years. The mental workload score was not different among systems. There were significant differences among the task scores, but these differences were not specific to particular systems. The overall score and task scores were related to the length of experience with their present system. Four tasks imposed a higher mental workload on GPs: 'repeat prescribing', 'find episode', 'drug management' and 'overview records'. Further usability studies on GP systems should focus on these tasks. Users, policymakers, designers and developers should remain aware of the importance of usability in system design.What does this study add?• Current GP systems in England do not need to conform to explicit usability requirements. Poor usability can increase the mental workload of clinicians and lead to errors.• Some clinical computer tasks incur more cognitive workload than others and should be considered carefully during the design of a system.• GPs did not report overall very high levels of subjective cognitive workload when undertaking common clinical tasks with their systems.• Further usability studies on GP systems should focus on the tasks incurring higher cognitive workload.• Users, policymakers, and designers and developers should remain aware of the importance of usability in system design.

Seamless recording of glucometer measurements among older experienced diabetic patients - A study of perception and usability.

PubMed

Rasche, Peter; Mertens, Alexander; Miron-Shatz, Talya; Berzon, Corinne; Schlick, Christopher M; Jahn, Michael; Becker, Stefan

2018-01-01

Self-measurement and documentation of blood-glucose are critical elements of diabetes management, particularly in regimes including insulin. In this study, we analyze the usability of iBG-STAR, the first blood glucose meter connectable to a smartphone. This technology records glucometer measurements, removing the burden of documentation from diabetic patients. This study assesses the potential for implementation of iBG-STAR in routine care. Twelve long-term diabetic patients (4 males; median age of 66.5 years) were enrolled in the study. N = 4/12 reported diabetic polyneuropathy. Reported subjective mental workload for all tasks related to iBG-STAR was on average lower than 12 points, corresponding to the verbal code 'nearly no effort needed'. A "Post Study System Usability Questionnaire", evaluated the glucometer at an average value of 2.06 (SD = 1.02) on a 7-Likert-scale (1 = 'I fully agree' to 7 = 'I completely disagree') for usability. These results represent a positive user-experience. Patients with polyneuropathy may experience physical difficulties in completing the tasks, thereby affecting usability. Technologically savvy patients (n = 6) with a positive outlook on diabetes assessed the product as a suitable tool for themselves and would recommend to other diabetic patients. The main barrier to regular use was treating physicians' inability to retrieve digitally recorded data. This barrier was due to a shortcoming in interoperability of mobile devices and medical information systems.

Debunking Health IT Usability Myths

PubMed Central

Staggers, N.; Xiao, Y.; Chapman, L.

2013-01-01

Poor usability is a threat to patient safety and linked to productivity loss, workflow disruption, user frustration, sub-optimal product use and system de-installations. Although usability is receiving more attention nationally and internationally, myths about usability persist. This editorial debunks five common myths about usability (1) usability only concerns the look and feel of a product and is, therefore, only a minor concern, (2) usability is not measurable, (3) usability stifles innovation, (4) vendors are solely responsible for product usability, and (5) usability methods are not practical for use in healthcare. PMID:23874361

HRI usability evaluation of interaction modes for a teleoperated agricultural robotic sprayer.

PubMed

Adamides, George; Katsanos, Christos; Parmet, Yisrael; Christou, Georgios; Xenos, Michalis; Hadzilacos, Thanasis; Edan, Yael

2017-07-01

Teleoperation of an agricultural robotic system requires effective and efficient human-robot interaction. This paper investigates the usability of different interaction modes for agricultural robot teleoperation. Specifically, we examined the overall influence of two types of output devices (PC screen, head mounted display), two types of peripheral vision support mechanisms (single view, multiple views), and two types of control input devices (PC keyboard, PS3 gamepad) on observed and perceived usability of a teleoperated agricultural sprayer. A modular user interface for teleoperating an agricultural robot sprayer was constructed and field-tested. Evaluation included eight interaction modes: the different combinations of the 3 factors. Thirty representative participants used each interaction mode to navigate the robot along a vineyard and spray grape clusters based on a 2 × 2 × 2 repeated measures experimental design. Objective metrics of the effectiveness and efficiency of the human-robot collaboration were collected. Participants also completed questionnaires related to their user experience with the system in each interaction mode. Results show that the most important factor for human-robot interface usability is the number and placement of views. The type of robot control input device was also a significant factor in certain dependents, whereas the effect of the screen output type was only significant on the participants' perceived workload index. Specific recommendations for mobile field robot teleoperation to improve HRI awareness for the agricultural spraying task are presented. Copyright © 2017 Elsevier Ltd. All rights reserved.

Development of the IMB Model and an Evidence-Based Diabetes Self-management Mobile Application.

PubMed

Jeon, Eunjoo; Park, Hyeoun-Ae

2018-04-01

This study developed a diabetes self-management mobile application based on the information-motivation-behavioral skills (IMB) model, evidence extracted from clinical practice guidelines, and requirements identified through focus group interviews (FGIs) with diabetes patients. We developed a diabetes self-management (DSM) app in accordance with the following four stages of the system development life cycle. The functional and knowledge requirements of the users were extracted through FGIs with 19 diabetes patients. A system diagram, data models, a database, an algorithm, screens, and menus were designed. An Android app and server with an SSL protocol were developed. The DSM app algorithm and heuristics, as well as the usability of the DSM app were evaluated, and then the DSM app was modified based on heuristics and usability evaluation. A total of 11 requirement themes were identified through the FGIs. Sixteen functions and 49 knowledge rules were extracted. The system diagram consisted of a client part and server part, 78 data models, a database with 10 tables, an algorithm, and a menu structure with 6 main menus, and 40 user screens were developed. The DSM app was Android version 4.4 or higher for Bluetooth connectivity. The proficiency and efficiency scores of the algorithm were 90.96% and 92.39%, respectively. Fifteen issues were revealed through the heuristic evaluation, and the app was modified to address three of these issues. It was also modified to address five comments received by the researchers through the usability evaluation. The DSM app was developed based on behavioral change theory through IMB models. It was designed to be evidence-based, user-centered, and effective. It remains necessary to fully evaluate the effect of the DSM app on the DSM behavior changes of diabetes patients.

Development of the IMB Model and an Evidence-Based Diabetes Self-management Mobile Application

PubMed Central

Jeon, Eunjoo

2018-01-01

Objectives This study developed a diabetes self-management mobile application based on the information-motivation-behavioral skills (IMB) model, evidence extracted from clinical practice guidelines, and requirements identified through focus group interviews (FGIs) with diabetes patients. Methods We developed a diabetes self-management (DSM) app in accordance with the following four stages of the system development life cycle. The functional and knowledge requirements of the users were extracted through FGIs with 19 diabetes patients. A system diagram, data models, a database, an algorithm, screens, and menus were designed. An Android app and server with an SSL protocol were developed. The DSM app algorithm and heuristics, as well as the usability of the DSM app were evaluated, and then the DSM app was modified based on heuristics and usability evaluation. Results A total of 11 requirement themes were identified through the FGIs. Sixteen functions and 49 knowledge rules were extracted. The system diagram consisted of a client part and server part, 78 data models, a database with 10 tables, an algorithm, and a menu structure with 6 main menus, and 40 user screens were developed. The DSM app was Android version 4.4 or higher for Bluetooth connectivity. The proficiency and efficiency scores of the algorithm were 90.96% and 92.39%, respectively. Fifteen issues were revealed through the heuristic evaluation, and the app was modified to address three of these issues. It was also modified to address five comments received by the researchers through the usability evaluation. Conclusions The DSM app was developed based on behavioral change theory through IMB models. It was designed to be evidence-based, user-centered, and effective. It remains necessary to fully evaluate the effect of the DSM app on the DSM behavior changes of diabetes patients. PMID:29770246

Web usability testing with a Hispanic medically underserved population.

PubMed

Moore, Mary; Bias, Randolph G; Prentice, Katherine; Fletcher, Robin; Vaughn, Terry

2009-04-01

Skilled website developers value usability testing to assure user needs are met. When the target audience differs substantially from the developers, it becomes essential to tailor both design and evaluation methods. In this study, researchers carried out a multifaceted usability evaluation of a website (Healthy Texas) designed for Hispanic audiences with lower computer literacy and lower health literacy. METHODS INCLUDED: (1) heuristic evaluation by a usability engineer, (2) remote end-user testing using WebEx software; and (3) face-to-face testing in a community center where use of the website was likely. Researchers found standard usability testing methods needed to be modified to provide interpreters, increased flexibility for time on task, presence of a trusted intermediary such as a librarian, and accommodation for family members who accompanied participants. Participants offered recommendations for website redesign, including simplified language, engaging and relevant graphics, culturally relevant examples, and clear navigation. User-centered design is especially important when website developers are not representative of the target audience. Failure to conduct appropriate usability testing with a representative audience can substantially reduce use and value of the website. This thorough course of usability testing identified improvements that benefit all users but become crucial when trying to reach an underserved audience.

Field Evaluation of Programmable Thermostats

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sachs, O.; Tiefenbeck, V.; Duvier, C.

2012-12-01

Prior research suggests that poor programmable thermostats usability may prevent their effective use to save energy. We hypothesized that home occupants with a high-usability thermostats would be more likely to use them to save energy than people with a basic thermostat. We randomly installed a high-usability thermostat in half the 77 apartments of an affordable housing complex, installing a basic thermostat in the other half. During the heating season, we collected space temperature and furnace on-off data to evaluate occupant interaction with the thermostats, foremost nighttime setbacks. We found that thermostat usability did not influence energy-saving behaviors, finding no significantmore » difference in temperature maintained among apartments with high- and low-usability thermostats.« less

Evaluation of user interface and workflow design of a bedside nursing clinical decision support system.

PubMed

Yuan, Michael Juntao; Finley, George Mike; Long, Ju; Mills, Christy; Johnson, Ron Kim

2013-01-31

Clinical decision support systems (CDSS) are important tools to improve health care outcomes and reduce preventable medical adverse events. However, the effectiveness and success of CDSS depend on their implementation context and usability in complex health care settings. As a result, usability design and validation, especially in real world clinical settings, are crucial aspects of successful CDSS implementations. Our objective was to develop a novel CDSS to help frontline nurses better manage critical symptom changes in hospitalized patients, hence reducing preventable failure to rescue cases. A robust user interface and implementation strategy that fit into existing workflows was key for the success of the CDSS. Guided by a formal usability evaluation framework, UFuRT (user, function, representation, and task analysis), we developed a high-level specification of the product that captures key usability requirements and is flexible to implement. We interviewed users of the proposed CDSS to identify requirements, listed functions, and operations the system must perform. We then designed visual and workflow representations of the product to perform the operations. The user interface and workflow design were evaluated via heuristic and end user performance evaluation. The heuristic evaluation was done after the first prototype, and its results were incorporated into the product before the end user evaluation was conducted. First, we recruited 4 evaluators with strong domain expertise to study the initial prototype. Heuristic violations were coded and rated for severity. Second, after development of the system, we assembled a panel of nurses, consisting of 3 licensed vocational nurses and 7 registered nurses, to evaluate the user interface and workflow via simulated use cases. We recorded whether each session was successfully completed and its completion time. Each nurse was asked to use the National Aeronautics and Space Administration (NASA) Task Load Index to self-evaluate the amount of cognitive and physical burden associated with using the device. A total of 83 heuristic violations were identified in the studies. The distribution of the heuristic violations and their average severity are reported. The nurse evaluators successfully completed all 30 sessions of the performance evaluations. All nurses were able to use the device after a single training session. On average, the nurses took 111 seconds (SD 30 seconds) to complete the simulated task. The NASA Task Load Index results indicated that the work overhead on the nurses was low. In fact, most of the burden measures were consistent with zero. The only potentially significant burden was temporal demand, which was consistent with the primary use case of the tool. The evaluation has shown that our design was functional and met the requirements demanded by the nurses' tight schedules and heavy workloads. The user interface embedded in the tool provided compelling utility to the nurse with minimal distraction.

A low cost technique to evaluate usable product for small manufacturing companies: a case study on Garcia robot.

PubMed

An, Vatana; Soares, Marcelo

2012-01-01

A usability evaluation technique to evaluate user interfaces is introduces. The technique is effective and affordable for small manufacturing companies. By using this technique, an integration of users' feedbacks and some usability concepts, a product can be 3 times easier to use among potential users and more than 5 times easier to use among motivated users. In addition, the technique can be implemented with the company's employees as participants.

Mobile applications for diabetics: a systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older.

PubMed

Arnhold, Madlen; Quade, Mandy; Kirch, Wilhelm

2014-04-09

A multitude of mhealth (mobile health) apps have been developed in recent years to support effective self-management of patients with diabetes mellitus type 1 or 2. We carried out a systematic review of all currently available diabetes apps for the operating systems iOS and Android. We considered the number of newly released diabetes apps, range of functions, target user groups, languages, acquisition costs, user ratings, available interfaces, and the connection between acquisition costs and user ratings. Additionally, we examined whether the available applications serve the special needs of diabetes patients aged 50 or older by performing an expert-based usability evaluation. We identified relevant keywords, comparative categories, and their specifications. Subsequently, we performed the app review based on the information given in the Google Play Store, the Apple App Store, and the apps themselves. In addition, we carried out an expert-based usability evaluation based on a representative 10% sample of diabetes apps. In total, we analyzed 656 apps finding that 355 (54.1%) offered just one function and 348 (53.0%) provided a documentation function. The dominating app language was English (85.4%, 560/656), patients represented the main user group (96.0%, 630/656), and the analysis of the costs revealed a trend toward free apps (53.7%, 352/656). The median price of paid apps was €1.90. The average user rating was 3.6 stars (maximum 5). Our analyses indicated no clear differences in the user rating between free and paid apps. Only 30 (4.6%) of the 656 available diabetes apps offered an interface to a measurement device. We evaluated 66 apps within the usability evaluation. On average, apps were rated best regarding the criterion "comprehensibility" (4.0 out of 5.0), while showing a lack of "fault tolerance" (2.8 out of 5.0). Of the 66 apps, 48 (72.7%) offered the ability to read the screen content aloud. The number of functions was significantly negative correlated with usability. The presence of documentation and analysis functions reduced the usability score significantly by 0.36 and 0.21 points. A vast number of diabetes apps already exist, but the majority offer similar functionalities and combine only one to two functions in one app. Patients and physicians alike should be involved in the app development process to a greater extent. We expect that the data transmission of health parameters to physicians will gain more importance in future applications. The usability of diabetes apps for patients aged 50 or older was moderate to good. But this result applied mainly to apps offering a small range of functions. Multifunctional apps performed considerably worse in terms of usability. Moreover, the presence of a documentation or analysis function resulted in significantly lower usability scores. The operability of accessibility features for diabetes apps was quite limited, except for the feature "screen reader".

Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics.

PubMed

Tsopra, Rosy; Jais, Jean-Philippe; Venot, Alain; Duclos, Catherine

2014-02-01

It is important to consider the way in which information is presented by the interfaces of clinical decision support systems, to favor the adoption of these systems by physicians. Interface design can focus on decision processes (guided navigation) or usability principles. The aim of this study was to compare these two approaches in terms of perceived usability, accuracy rate, and confidence in the system. We displayed clinical practice guidelines for antibiotic treatment via two types of interface, which we compared in a crossover design. General practitioners were asked to provide responses for 10 clinical cases and the System Usability Scale (SUS) for each interface. We assessed SUS scores, the number of correct responses, and the confidence level for each interface. SUS score and percentage confidence were significantly higher for the interface designed according to usability principles (81 vs 51, p=0.00004, and 88.8% vs 80.7%, p=0.004). The percentage of correct responses was similar for the two interfaces. The interface designed according to usability principles was perceived to be more usable and inspired greater confidence among physicians than the guided navigation interface. Consideration of usability principles in the construction of an interface--in particular 'effective information presentation', 'consistency', 'efficient interactions', 'effective use of language', and 'minimizing cognitive load'--seemed to improve perceived usability and confidence in the system.

Usability evaluation model for mobile e-book applications

NASA Astrophysics Data System (ADS)

Matraf, Munya Saleh Ba; Hussain, Azham

2017-10-01

Evaluation for mobile e-book applications are limited and did not address all the important usability measurements. Hence, this study aimed to identify the characteristics that affect user satisfaction on the usability of mobile e-book applications. Five characteristics that have a significant effect on the user satisfaction of mobile e-book applications have been identified namely readability, effectiveness, accessibility, efficiency, and navigation. A usability evaluation was conducted on three mobile e-book applications namely Adobe Acrobat Reader, Ebook Reader, and Amazon Kindle. 30 students from Universiti Utara Malaysia evaluated the mobile e-book applications and their satisfaction was measured using questionnaire. The outcomes discovered that the five characteristics have a significant positive relationship with user satisfaction. This provides insights into the main characteristics that increase user satisfaction.

Increasing Complexity in Rule-Based Clinical Decision Support: The Symptom Assessment and Management Intervention.

PubMed

Lobach, David F; Johns, Ellis B; Halpenny, Barbara; Saunders, Toni-Ann; Brzozowski, Jane; Del Fiol, Guilherme; Berry, Donna L; Braun, Ilana M; Finn, Kathleen; Wolfe, Joanne; Abrahm, Janet L; Cooley, Mary E

2016-11-08

Management of uncontrolled symptoms is an important component of quality cancer care. Clinical guidelines are available for optimal symptom management, but are not often integrated into the front lines of care. The use of clinical decision support (CDS) at the point-of-care is an innovative way to incorporate guideline-based symptom management into routine cancer care. The objective of this study was to develop and evaluate a rule-based CDS system to enable management of multiple symptoms in lung cancer patients at the point-of-care. This study was conducted in three phases involving a formative evaluation, a system evaluation, and a contextual evaluation of clinical use. In Phase 1, we conducted iterative usability testing of user interface prototypes with patients and health care providers (HCPs) in two thoracic oncology clinics. In Phase 2, we programmed complex algorithms derived from clinical practice guidelines into a rules engine that used Web services to communicate with the end-user application. Unit testing of algorithms was conducted using a stack-traversal tree-spanning methodology to identify all possible permutations of pathways through each algorithm, to validate accuracy. In Phase 3, we evaluated clinical use of the system among patients and HCPs in the two clinics via observations, structured interviews, and questionnaires. In Phase 1, 13 patients and 5 HCPs engaged in two rounds of formative testing, and suggested improvements leading to revisions until overall usability scores met a priori benchmarks. In Phase 2, symptom management algorithms contained between 29 and 1425 decision nodes, resulting in 19 to 3194 unique pathways per algorithm. Unit testing required 240 person-hours, and integration testing required 40 person-hours. In Phase 3, both patients and HCPs found the system usable and acceptable, and offered suggestions for improvements. A rule-based CDS system for complex symptom management was systematically developed and tested. The complexity of the algorithms required extensive development and innovative testing. The Web service-based approach allowed remote access to CDS knowledge, and could enable scaling and sharing of this knowledge to accelerate availability, and reduce duplication of effort. Patients and HCPs found the system to be usable and useful. ©David F Lobach, Ellis B Johns, Barbara Halpenny, Toni-Ann Saunders, Jane Brzozowski, Guilherme Del Fiol, Donna L Berry, Ilana M Braun, Kathleen Finn, Joanne Wolfe, Janet L Abrahm, Mary E Cooley. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 08.11.2016.

User perspectives on the usability of a regional health information exchange

PubMed Central

Ho, Yun-Xian; Cala, Cather Marie; Blakemore, Dana; Chen, Qingxia; Frisse, Mark E; Johnson, Kevin B

2011-01-01

Objective We assessed the usability of a health information exchange (HIE) in a densely populated metropolitan region. This grant-funded HIE had been deployed rapidly to address the imminent needs of the patient population and the need to draw wider participation from regional entities. Design We conducted a cross-sectional survey of individuals given access to the HIE at participating organizations and examined some of the usability and usage factors related to the technology acceptance model. Measurements We probed user perceptions using the Questionnaire for User Interaction Satisfaction, an author-generated Trust scale, and user characteristic questions (eg, age, weekly system usage time). Results Overall, users viewed the system favorably (ratings for all usability items were greater than neutral (one-sample Wilcoxon test, p<0.0014, Bonferroni-corrected for 35 tests). System usage was regressed on usability, trust, and demographic and user characteristic factors. Three usability factors were positively predictive of system usage: overall reactions (p<0 0.01), learning (p<0.05), and system functionality (p<0.01). Although trust is an important component in collaborative relationships, we did not find that user trust of other participating healthcare entities was significantly predictive of usage. An analysis of respondents' comments revealed ways to improve the HIE. Conclusion We used a rapid deployment model to develop an HIE and found that perceptions of system usability were positive. We also found that system usage was predicted well by some aspects of usability. Results from this study suggest that a rapid development approach may serve as a viable model for developing usable HIEs serving communities with limited resources. PMID:21622933

The added value of user involvement during the development of a feedback system regarding physical functioning for community-dwelling elderly people

PubMed Central

Vermeulen, Joan; Neyens, Jacques C. L; Spreeuwenberg, Marieke D; Van Rossum, Erik; Hewson, David; De Witte, Luc P

2012-01-01

Background The number of frail elderly people is increasing. Unfortunately, the number of caregivers is not increasing at the same pace, which affects older people, caregivers and healthcare systems. Because of these developments, self-management is becoming more important in healthcare. To support community-dwelling elderly people in their self-management, a system was developed that monitors their physical functioning. This system provides feedback to elderly people and their caregivers regarding physical indicators of frailty. The feedback is provided to elderly people via the screen of a mobile phone. It is important that elderly people understand the content of the feedback and are able to use the mobile phone properly. If not, it is unlikely that the system can support self-management. Many interactive health technologies that have been developed do not fulfil their promises. An important reason for this is that human and other non-technology issues are not sufficiently taken into consideration during the development process. Objective To collaborate with elderly people during the development and evaluation of a feedback system for community-dwelling elderly people regarding physical functioning. Methods An iterative user-centered design that consists of five phases was used to develop and evaluate the feedback system. These five phases were: 1) Selection of users, 2) Analysis of users and their context, 3) Identification of user needs, 4) Development of a prototype, and 5) Evaluation of the prototype. Three representatives of a target group panel for elderly people were selected in phase 1. They shared their needs and preferences during three expert group meetings that took place during the development process. This resulted in the development of a prototype which was first evaluated in a heuristic evaluation. Once adjustments were made, 11 elderly people evaluated the adjusted prototype using a think aloud procedure. They rated the usability and acceptability of the developed interface on a scale from 1 till 7 using an adapted version of the Post-Study System Usability Questionnaire (PSSUQ). Results A feedback system was developed that provides feedback regarding physical indicators of frailty via a touch screen mobile phone. The interface uses colours, smiley’s, and spoken/written messages to provide feedback that is easy to understand. The heuristic evaluation revealed that there were some problems with consistency and the use of user language. The think aloud evaluation showed that the 11 elderly people were able to navigate through the interface without much difficulty despite some small problems related to the lay-out of the interface. The mean score on an adapted version of the PSSUQ was 5.90 (SD 1.09) which indicates high user satisfaction and good usability. Conclusions The involvement of end-users significantly influenced the lay-out of the interface that was developed. This resulted in an interface that was accepted by the target group. Evaluation of the prototype revealed that the usability of the interface was good. The feedback system will only succeed in supporting self-management when elderly people are able to use the interface and understand the feedback. The input of elderly people during the development process contributed to this.

Preliminary study of online machine translation use of nursing literature: quality evaluation and perceived usability

PubMed Central

2012-01-01

Background Japanese nurses are increasingly required to read published international research in clinical, educational, and research settings. Language barriers are a significant obstacle, and online machine translation (MT) is a tool that can be used to address this issue. We examined the quality of Google Translate® (English to Japanese and Korean to Japanese), which is a representative online MT, using a previously verified evaluation method. We also examined the perceived usability and current use of online MT among Japanese nurses. Findings Randomly selected nursing abstracts were translated and then evaluated for intelligibility and usability by 28 participants, including assistants and research associates from nursing universities throughout Japan. They answered a questionnaire about their online MT use. From simple comparison of mean scores between two language pairs, translation quality was significantly better, with respect to both intelligibility and usability, for Korean-Japanese than for English-Japanese. Most respondents perceived a language barrier. Online MT had been used by 61% of the respondents and was perceived as not useful enough. Conclusion Nursing articles translated from Korean into Japanese by an online MT system could be read at an acceptable level of comprehension, but the same could not be said for English-Japanese translations. Respondents with experience using online MT used it largely to grasp the overall meanings of the original text. Enrichment in technical terms appeared to be the key to better usability. Users will be better able to use MT outputs if they improve their foreign language proficiency as much as possible. Further research is being conducted with a larger sample size and detailed analysis. PMID:23151362

Evaluation of Smartphone Applications for Cardiopulmonary Resuscitation Training in South Korea

PubMed Central

Cho, Yongtak; Song, Yeongtak; Lim, Tae Ho; Kang, Hyunggoo

2016-01-01

Objective. There are many smartphone-based applications (apps) for cardiopulmonary resuscitation (CPR) training. We investigated the conformity and the learnability/usability of these apps for CPR training and real-life supports. Methods. We conducted a mixed-method, sequential explanatory study to assess CPR training apps downloaded on two apps stores in South Korea. Apps were collected with inclusion criteria as follows, Korean-language instruction, training features, and emergency supports for real-life incidents, and analyzed with two tests; 15 medical experts evaluated the apps' contents according to current Basic Life Support guidelines in conformity test, and 15 nonmedical individuals examined the apps using System Usability Scale (SUS) in the learnability/usability test. Results. Out of 79 selected apps, five apps were included and analyzed. For conformity (ICC, 0.95, p < 0.001), means of all apps were greater than 12 of 20 points, indicating that they were well designed according to current guidelines. Three of the five apps yielded acceptable level (greater than 68 of 100 points) for learnability/usability. Conclusion. All the included apps followed current BLS guidelines and a majority offered acceptable learnability/usability for layperson. Current and developmental smartphone-based CPR training apps should include accurate CPR information and be easy to use for laypersons that are potential rescuers in real-life incidents. For Clinical Trials. This is a clinical trial, registered at the Clinical Research Information Service (CRIS, cris.nih.go.kr), number KCT0001840. PMID:27668257

Usability of a barcode scanning system as a means of data entry on a PDA for self-report health outcome questionnaires: a pilot study in individuals over 60 years of age

PubMed Central

Boissy, Patrick; Jacobs, Karen; Roy, Serge H

2006-01-01

Background Throughout the medical and paramedical professions, self-report health status questionnaires are used to gather patient-reported outcome measures. The objective of this pilot study was to evaluate in individuals over 60 years of age the usability of a PDA-based barcode scanning system with a text-to-speech synthesizer to collect data electronically from self-report health outcome questionnaires. Methods Usability of the system was tested on a sample of 24 community-living older adults (7 men, 17 women) ranging in age from 63 to 93 years. After receiving a brief demonstration on the use of the barcode scanner, participants were randomly assigned to complete two sets of 16 questions using the bar code wand scanner for one set and a pen for the other. Usability was assessed using directed interviews with a usability questionnaire and performance-based metrics (task times, errors, sources of errors). Results Overall, participants found barcode scanning easy to learn, easy to use, and pleasant. Participants were marginally faster in completing the 16 survey questions when using pen entry (20/24 participants). The mean response time with the barcode scanner was 31 seconds longer than traditional pen entry for a subset of 16 questions (p = 0.001). The responsiveness of the scanning system, expressed as first scan success rate, was less than perfect, with approximately one-third of first scans requiring a rescan to successfully capture the data entry. The responsiveness of the system can be explained by a combination of factors such as the location of the scanning errors, the type of barcode used as an answer field in the paper version, and the optical characteristics of the barcode scanner. Conclusion The results presented in this study offer insights regarding the feasibility, usability and effectiveness of using a barcode scanner with older adults as an electronic data entry method on a PDA. While participants in this study found their experience with the barcode scanning system enjoyable and learned to become proficient in its use, the responsiveness of the system constitutes a barrier to wide-scale use of such a system. Optimizing the graphical presentation of the information on paper should significantly increase the system's responsiveness. PMID:17184533

Palliative Care Planner: A Pilot Study to Evaluate Acceptability and Usability of an Electronic Health Records System-integrated, Needs-targeted App Platform.

PubMed

Cox, Christopher E; Jones, Derek M; Reagan, Wen; Key, Mary D; Chow, Vinca; McFarlin, Jessica; Casarett, David; Creutzfeldt, Claire J; Docherty, Sharron L

2018-01-01

The quality and patient-centeredness of intensive care unit (ICU)-based palliative care delivery is highly variable. To develop and pilot an app platform for clinicians and ICU patients and their family members that enhances the delivery of needs-targeted palliative care. In the development phase of the study, we developed an electronic health record (EHR) system-integrated mobile web app system prototype, PCplanner (Palliative Care Planner). PCplanner screens the EHR for ICU patients meeting any of five prompts (triggers) for palliative care consultation, allows families to report their unmet palliative care needs, and alerts clinicians to these needs. The evaluation phase included a prospective before/after study conducted at a large academic medical center. Two control populations were enrolled in the before period to serve as context for the intervention. First, 25 ICU patients who received palliative care consults served as patient-level controls. Second, 49 family members of ICU patients who received mechanical ventilation for at least 48 hours served as family-level controls. Afterward, 14 patients, 18 family members, and 10 clinicians participated in the intervention evaluation period. Family member outcomes measured at baseline and 4 days later included acceptability (Client Satisfaction Questionnaire [CSQ]), usability (Systems Usability Scale [SUS]), and palliative care needs, assessed with the adapted needs of social nature, existential concerns, symptoms, and therapeutic interaction (NEST) scale; the Patient-Centeredness of Care Scale (PCCS); and the Perceived Stress Scale (PSS). Patient outcomes included frequency of goal concordant treatment, hospital length of stay, and discharge disposition. Family members reported high PCplanner acceptability (mean CSQ, 14.1 [SD, 1.4]) and usability (mean SUS, 21.1 [SD, 1.7]). PCplanner family member recipients experienced a 12.7-unit reduction in NEST score compared with a 3.4-unit increase among controls (P = 0.002), as well as improved mean scores on the PCCS (6.6 [SD, 5.8]) and the PSS (-0.8 [SD, 1.9]). The frequency of goal-concordant treatment increased over the course of the intervention (n = 14 [SD, 79%] vs. n = 18 [SD, 100%]). Compared with palliative care controls, intervention patients received palliative care consultation sooner (3.9 [SD, 2.7] vs. 6.9 [SD, 7.1] mean days), had a shorter mean hospital length of stay (20.5 [SD, 9.1] vs. 22.3 [SD, 16.0] patient number), and received hospice care more frequently (5 [36%] vs. 5 [20%]), although these differences were not statistically significant. PCplanner represents an acceptable, usable, and clinically promising systems-based approach to delivering EHR-triggered, needs-targeted ICU-based palliative care within a standard clinical workflow. A clinical trial in a larger population is needed to evaluate its efficacy.

Implementation of the Agitated Behavior Scale in the Electronic Health Record.

PubMed

Wilson, Helen John; Dasgupta, Kritis; Michael, Kathleen

The purpose of the study was to implement an Agitated Behavior Scale through an electronic health record and to evaluate the usability of the scale in a brain injury unit at a rehabilitation hospital. A quality improvement project was conducted in the brain injury unit at a large rehabilitation hospital with registered nurses as participants using convenience sampling. The project consisted of three phases and included education, implementation of the scale in the electronic health record, and administration of the survey questionnaire, which utilized the system usability scale. The Agitated Behavior Scale was found to be usable, and there was 92.2% compliance with the use of the electronic Electronic Agitated Behavior Scale. The Agitated Behavior Scale was effectively implemented in the electronic health record and was found to be usable in the assessment of agitation. Utilization of the scale through the electronic health record on a daily basis will allow for an early identification of agitation in patients with traumatic brain injury and enable prompt interventions to manage agitation.

User Interface Requirements for Web-Based Integrated Care Pathways: Evidence from the Evaluation of an Online Care Pathway Investigation Tool.

PubMed

Balatsoukas, Panos; Williams, Richard; Davies, Colin; Ainsworth, John; Buchan, Iain

2015-11-01

Integrated care pathways (ICPs) define a chronological sequence of steps, most commonly diagnostic or treatment, to be followed in providing care for patients. Care pathways help to ensure quality standards are met and to reduce variation in practice. Although research on the computerisation of ICP progresses, there is still little knowledge on what are the requirements for designing user-friendly and usable electronic care pathways, or how users (normally health care professionals) interact with interfaces that support design, analysis and visualisation of ICPs. The purpose of the study reported in this paper was to address this gap by evaluating the usability of a novel web-based tool called COCPIT (Collaborative Online Care Pathway Investigation Tool). COCPIT supports the design, analysis and visualisation of ICPs at the population level. In order to address the aim of this study, an evaluation methodology was designed based on heuristic evaluations and a mixed method usability test. The results showed that modular visualisation and direct manipulation of information related to the design and analysis of ICPs is useful for engaging and stimulating users. However, designers should pay attention to issues related to the visibility of the system status and the match between the system and the real world, especially in relation to the display of statistical information about care pathways and the editing of clinical information within a care pathway. The paper concludes with recommendations for interface design.

45 CFR 155.230 - General standards for Exchange notices.

Code of Federal Regulations, 2014 CFR

2014-10-01

...). (c) Re-evaluation of appropriateness and usability. The Exchange must re-evaluate the appropriateness and usability of applications, forms, and notices. (d) Electronic notices. (1) The individual market...

45 CFR 155.230 - General standards for Exchange notices.

Code of Federal Regulations, 2013 CFR

2013-10-01

...). (c) Re-evaluation of appropriateness and usability. The Exchange must re-evaluate the appropriateness and usability of applications, forms, and notices. (d) Electronic notices. (1) The individual market...

Design and pilot evaluation of a system to develop computer-based site-specific practice guidelines from decision models.

PubMed

Sanders, G D; Nease, R F; Owens, D K

2000-01-01

Local tailoring of clinical practice guidelines (CPGs) requires experts in medicine and evidence synthesis unavailable in many practice settings. The authors' computer-based system enables developers and users to create, disseminate, and tailor CPGs, using normative decision models (DMs). ALCHEMIST, a web-based system, analyzes a DM, creates a CPG in the form of an annotated algorithm, and displays for the guideline user the optimal strategy. ALCHEMIST'S interface enables remote users to tailor the guideline by changing underlying input variables and observing the new annotated algorithm that is developed automatically. In a pilot evaluation of the system, a DM was used to evaluate strategies for staging non-small-cell lung cancer. Subjects (n = 15) compared the automatically created CPG with published guidelines for this staging and critiqued both using a previously developed instrument to rate the CPGs' usability, accountability, and accuracy on a scale of 0 (worst) to 2 (best), with higher scores reflecting higher quality. The mean overall score for the ALCHEMIST CPG was 1.502, compared with the published-CPG score of 0.987 (p = 0.002). The ALCHEMIST CPG scores for usability, accountability, and accuracy were 1.683, 1.393, and 1.430, respectively; the published CPG scores were 1.192, 0.941, and 0.830 (each comparison p < 0.05). On a scale of 1 (worst) to 5 (best), users' mean ratings of ALCHEMIST'S ease of use, usefulness of content, and presentation format were 4.76, 3.98, and 4.64, respectively. The results demonstrate the feasibility of a web-based system that automatically analyzes a DM and creates a CPG as an annotated algorithm, enabling remote users to develop site-specific CPGs. In the pilot evaluation, the ALCHEMIST guidelines met established criteria for quality and compared favorably with national CPGs. The high usability and usefulness ratings suggest that such systems can be a good tool for guideline development.

Usability factors of mobile health application for chronic diseases

NASA Astrophysics Data System (ADS)

Zahra, Fatima; Hussain, Azham; Mohd, Haslina

2016-08-01

M-health has changed the conventional delivery system of health-care, permitting continuous, pervasive Health-care anywhere, anytime. Chronic disease apps are increasing, as many health workers, patients and clinicians already embracing smartphones in their comprehensive and diverse practices. There are lots of challenges and requirements that need to be addressed for mobile health applications to prevent or eliminate design problems and minimize potential threats for users, the proposed factors for chronic disease mobile applications can be used as a guide for app developers While, the usability testing, and evaluations of chronic disease apps have not yet touched the accuracy level of other web based applications. This study is being conducted to learn about challenges of m-health apps and to identify the factors that affect the usability of such applications.

Teaching clinical reasoning and decision-making skills to nursing students: Design, development, and usability evaluation of a serious game.

PubMed

Johnsen, Hege Mari; Fossum, Mariann; Vivekananda-Schmidt, Pirashanthie; Fruhling, Ann; Slettebø, Åshild

2016-10-01

Serious games (SGs) are a type of simulation technology that may provide nursing students with the opportunity to practice their clinical reasoning and decision-making skills in a safe and authentic environment. Despite the growing number of SGs developed for healthcare professionals, few SGs are video based or address the domain of home health care. This paper aims to describe the design, development, and usability evaluation of a video based SG for teaching clinical reasoning and decision-making skills to nursing students who care for patients with chronic obstructive pulmonary disease (COPD) in home healthcare settings. A prototype SG was developed. A unified framework of usability called TURF (Task, User, Representation, and Function) and SG theory were employed to ensure a user-centered design. The educational content was based on the clinical decision-making model, Bloom's taxonomy, and a Bachelor of Nursing curriculum. A purposeful sample of six participants evaluated the SG prototype in a usability laboratory. Cognitive walkthrough evaluations, a questionnaire, and individual interviews were used for the usability evaluation. The data were analyzed using qualitative deductive content analysis based on the TURF framework elements and related usability heuristics. The SG was perceived as being realistic, clinically relevant, and at an adequate level of complexity for the intended users. Usability issues regarding functionality and the user-computer interface design were identified. However, the SG was perceived as being easy to learn, and participants suggested that the SG could serve as a supplement to traditional training in laboratory and clinical settings. Using video based scenarios with an authentic COPD patient and a home healthcare registered nurse as actors contributed to increased realism. Using different theoretical approaches in the SG design was considered an advantage of the design process. The SG was perceived as being useful, usable, and satisfying. The achievement of the desired functionality and the minimization of user-computer interface issues emphasize the importance of conducting a usability evaluation during the SG development process. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

Identification of design features to enhance utilization and acceptance of systems for Internet-based decision support at the point of care.

PubMed Central

Gadd, C. S.; Baskaran, P.; Lobach, D. F.

1998-01-01

Extensive utilization of point-of-care decision support systems will be largely dependent on the development of user interaction capabilities that make them effective clinical tools in patient care settings. This research identified critical design features of point-of-care decision support systems that are preferred by physicians, through a multi-method formative evaluation of an evolving prototype of an Internet-based clinical decision support system. Clinicians used four versions of the system--each highlighting a different functionality. Surveys and qualitative evaluation methodologies assessed clinicians' perceptions regarding system usability and usefulness. Our analyses identified features that improve perceived usability, such as telegraphic representations of guideline-related information, facile navigation, and a forgiving, flexible interface. Users also preferred features that enhance usefulness and motivate use, such as an encounter documentation tool and the availability of physician instruction and patient education materials. In addition to identifying design features that are relevant to efforts to develop clinical systems for point-of-care decision support, this study demonstrates the value of combining quantitative and qualitative methods of formative evaluation with an iterative system development strategy to implement new information technology in complex clinical settings. Images Figure 1 PMID:9929188

Usability Testing of a Multimedia e-Learning Resource for Electrolyte and Acid-Base Disorders

ERIC Educational Resources Information Center

Davids, Mogamat Razeen; Chikte, Usuf; Grimmer-Somers, Karen; Halperin, Mitchell L.

2014-01-01

The usability of computer interfaces may have a major influence on learning. Design approaches that optimize usability are commonplace in the software development industry but are seldom used in the development of e-learning resources, especially in medical education. We conducted a usability evaluation of a multimedia resource for teaching…

47 CFR 6.7 - Product design, development, and evaluation.

Code of Federal Regulations, 2014 CFR

2014-10-01

...) Manufacturers and service providers shall evaluate the accessibility, usability, and compatibility of equipment... identify barriers to accessibility and usability as part of such a product design and development process...

47 CFR 6.7 - Product design, development, and evaluation.

Code of Federal Regulations, 2013 CFR

2013-10-01

...) Manufacturers and service providers shall evaluate the accessibility, usability, and compatibility of equipment... identify barriers to accessibility and usability as part of such a product design and development process...

Usability testing of a monitoring and feedback tool to stimulate physical activity.

PubMed

van der Weegen, Sanne; Verwey, Renée; Tange, Huibert J; Spreeuwenberg, Marieke D; de Witte, Luc P

2014-01-01

A MONITORING AND FEEDBACK TOOL TO STIMULATE PHYSICAL ACTIVITY, CONSISTING OF AN ACTIVITY SENSOR, SMARTPHONE APPLICATION (APP), AND WEBSITE FOR PATIENTS AND THEIR PRACTICE NURSES, HAS BEEN DEVELOPED: the 'It's LiFe!' tool. In this study the usability of the tool was evaluated by technology experts and end users (people with chronic obstructive pulmonary disease or type 2 diabetes, with ages from 40-70 years), to improve the user interfaces and content of the tool. THE STUDY HAD FOUR PHASES: 1) a heuristic evaluation with six technology experts; 2) a usability test in a laboratory by five patients; 3) a pilot in real life wherein 20 patients used the tool for 3 months; and 4) a final lab test by five patients. In both lab tests (phases 2 and 4) qualitative data were collected through a thinking-aloud procedure and video recordings, and quantitative data through questions about task complexity, text comprehensiveness, and readability. In addition, the post-study system usability questionnaire (PSSUQ) was completed for the app and the website. In the pilot test (phase 3), all patients were interviewed three times and the Software Usability Measurement Inventory (SUMI) was completed. After each phase, improvements were made, mainly to the layout and text. The main improvement was a refresh button for active data synchronization between activity sensor, app, and server, implemented after connectivity problems in the pilot test. The mean score on the PSSUQ for the website improved from 5.6 (standard deviation [SD] 1.3) to 6.5 (SD 0.5), and for the app from 5.4 (SD 1.5) to 6.2 (SD 1.1). Satisfaction in the pilot was not very high according to the SUMI. The use of laboratory versus real-life tests and expert-based versus user-based tests revealed a wide range of usability issues. The usability of the It's LiFe! tool improved considerably during the study.

Tracking Patient Education Documentation across Time and Care Settings

PubMed Central

Janousek, Lisa; Heermann, Judith; Eilers, June

2005-01-01

Results of a formative evaluation of a patient education documentation system will be presented. Both quantitative and qualitative approaches to data collection are being used. The goal of integrating patient education documentation into the electronic patient record is to facilitate seamless, multidisciplinary patient/family education across time and settings. The system is being piloted by oncology services at The Nebraska Medical Center. The evaluation addresses the usability and comprehensiveness of the system. PMID:16779280

How you spend your pennies ... factors affecting the efficiency of human waste disposal systems (re-usable and disposable) and their cost.

PubMed

Rollnick, M

1991-05-01

Both re-usable and disposable systems have their merits and problems. The disposable system, being fully integrated, appears to be steadily gaining market share compared with the re-usable system. Since its introduction, the success of the re-usable system has been limited by the use of pans not designed for automatic processing. Where the 'perfection' pan has been superseded by 'open' shaped receptacles and those used in commode chairs, cleaning effectiveness can be improved by a factor of 10. For this and other reasons, nursing involvement in the re-usable system can be high while the 'perfection' pan is in use. A work study exercise to compare nursing involvement in re-usable and disposable systems is under way. When selecting a human waste disposal system for any site, it is vital that all disciplines discuss and decide objectives. The equipment usage, space, site conditions, availability and quality of supplies (eg water, electricity), the costs of maintenance, nursing time and other expenditure must be considered. The disposable system is capable of high process rates (more than double that of the fastest re-usable system). Its capital cost is currently about 1,000 pounds less than an average re-usable system, but in the busiest wards, revenue costs may be higher. In such wards the space for disposable receptacle storage can be as much as five to ten times machine volume. The design of macerators is generally simpler (having less components) than washer-disinfectors. Monitoring and maintenance involvement are likewise expected to be lower, particularly in hospitals with modern drainage systems.(ABSTRACT TRUNCATED AT 250 WORDS)

Heuristic evaluation and thinking aloud test of a digitized questionnaire for diabetes outpatient clinics.

PubMed

Schaarup, Clara; Hejlesen, Ole K

2014-01-01

Diabetes is a chronic disease characterized by hyperglycaemia. The number of patients with diabetes is expected to exceed 592 million in 2035. The growing number of diabetics is a great burden for the Danish healthcare system. The Danish government desires a modern and efficient healthcare system with a high patient security and a coherent continuity of care. To achieve these outcomes medical record-keeping, paper questionnaires and notes must be digitized. The current system enforces that the diabetics fill out questionnaires in paper form after which the healthcare personnel enter the same information in the electronic health record. In this study, an online questionnaire was designed and the usability was evaluated using the following parameters: learnability, efficiency, memorability, errors, and satisfaction. The parameters were evaluated by using the discount usability engineering method. 5 double specialists and 6 patients diagnosed with diabetes provided the data of the study. The results indicated that simple and obvious figures were preferred in the online questionnaire, as well as error preventing in the form of validation fields. This study inspire to further development in the digitizing process.

Heuristic evaluation of eNote: an electronic notes system.

PubMed

Bright, Tiffani J; Bakken, Suzanne; Johnson, Stephen B

2006-01-01

eNote is an electronic health record (EHR) system based on semi-structured narrative documents. A heuristic evaluation was conducted with a sample of five usability experts. eNote performed highly in: 1)consistency with standards and 2)recognition rather than recall. eNote needs improvement in: 1)help and documentation, 2)aesthetic and minimalist design, 3)error prevention, 4)helping users recognize, diagnosis, and recover from errors, and 5)flexibility and efficiency of use. The heuristic evaluation was an efficient method of evaluating our interface.

The Cultural Evolution of Human Communication Systems in Different Sized Populations: Usability Trumps Learnability

PubMed Central

Fay, Nicolas; Ellison, T. Mark

2013-01-01

This study examines the intergenerational transfer of human communication systems. It tests if human communication systems evolve to be easy to learn or easy to use (or both), and how population size affects learnability and usability. Using an experimental-semiotic task, we find that human communication systems evolve to be easier to use (production efficiency and reproduction fidelity), but harder to learn (identification accuracy) for a second generation of naïve participants. Thus, usability trumps learnability. In addition, the communication systems that evolve in larger populations exhibit distinct advantages over those that evolve in smaller populations: the learnability loss (from the Initial signs) is more muted and the usability benefits are more pronounced. The usability benefits for human communication systems that evolve in a small and large population is explained through guided variation reducing sign complexity. The enhanced performance of the communication systems that evolve in larger populations is explained by the operation of a content bias acting on the larger pool of competing signs. The content bias selects for information-efficient iconic signs that aid learnability and enhance usability. PMID:23967243

The cultural evolution of human communication systems in different sized populations: usability trumps learnability.

PubMed

Fay, Nicolas; Ellison, T Mark

2013-01-01

This study examines the intergenerational transfer of human communication systems. It tests if human communication systems evolve to be easy to learn or easy to use (or both), and how population size affects learnability and usability. Using an experimental-semiotic task, we find that human communication systems evolve to be easier to use (production efficiency and reproduction fidelity), but harder to learn (identification accuracy) for a second generation of naïve participants. Thus, usability trumps learnability. In addition, the communication systems that evolve in larger populations exhibit distinct advantages over those that evolve in smaller populations: the learnability loss (from the Initial signs) is more muted and the usability benefits are more pronounced. The usability benefits for human communication systems that evolve in a small and large population is explained through guided variation reducing sign complexity. The enhanced performance of the communication systems that evolve in larger populations is explained by the operation of a content bias acting on the larger pool of competing signs. The content bias selects for information-efficient iconic signs that aid learnability and enhance usability.

Examining Older Adults’ Perceptions of Usability and Acceptability of Remote Monitoring Systems to Manage Chronic Heart Failure

PubMed Central

Evangelista, Lorraine S.; Moser, Debra K.; Lee, Jung-Ah; Moore, Alison A.; Ghasemzadeh, Hassan; Sarrafzadeh, Majid; Mangione, Carol M.

2015-01-01

Objective: This study was conducted to evaluate the feasibility, usability, and acceptability of using remote monitoring systems (RMS) in monitoring health status (e.g., vital signs, symptom distress) in older adults (≥55) with chronic heart failure (HF). Method: Twenty-one patients (52.4% women, mean age 73.1 ± 9.3) were trained to measure and transmit health data with an RMS. Data transmissions were tracked for 12 weeks. Results: All participants initiated use of RMS within 1 week; 71%, 14%, and 14% of patients transmitted daily health data 100%, ≥75%, and <75% of the time, respectively, for 12 weeks. Overall usability and acceptability of the RMS were 4.08 ± 0.634 and 4.10 ± 0.563, respectively (when scored on a range of 1-5, where 1 = strongly disagree and 5 = strongly agree). Discussion: Findings show that an RMS-based intervention can be successfully implemented in a group of older patients with chronic HF. PMID:28138479

Usability of online psychoeducation for siblings of people with psychosis.

PubMed

Sin, Jacqueline; Henderson, Claire; Norman, Ian

2014-10-01

The E Sibling Project aims to address the needs of siblings of individuals affected by psychosis through provision of a comprehensive online intervention. The online intervention comprises four core elements, including: information on psychosis; various coping and promoting well-being strategies; siblings' blogs and discussion forum with peers; and an "Ask the Experts" function. After the intervention-prototype was developed, we tested its feasibility, usability and acceptability by siblings. We evaluated the usability of the intervention-prototype using a non-randomised usability study with siblings of individuals diagnosed with psychosis. The usability study adapted Poulson et al's framework to collect subjective feedback from participants on ease of use, perceived usefulness and acceptability, together with objective usage data on the intervention. Twenty siblings were recruited to the usability test; 19 tried out the resource-prototype over a 4-week period and 17 completed the online evaluation after using the intervention. In total, 906 page-views were made by the participants and each spent about two hours using the resource. Participants evaluated the intervention as helpful, relevant and useful in terms of content, design and usability. Developments are needed to improve the navigation and intuitiveness of the resource. Using an internet-based information-giving and peer support intervention to promote wellbeing and coping is feasible and acceptable to siblings of people with psychosis.

The STEP database through the end-users eyes--USABILITY STUDY.

PubMed

Salunke, Smita; Tuleu, Catherine

2015-08-15

The user-designed database of Safety and Toxicity of Excipients for Paediatrics ("STEP") is created to address the shared need of drug development community to access the relevant information of excipients effortlessly. Usability testing was performed to validate if the database satisfies the need of the end-users. Evaluation framework was developed to assess the usability. The participants performed scenario based tasks and provided feedback and post-session usability ratings. Failure Mode Effect Analysis (FMEA) was performed to prioritize the problems and improvements to the STEP database design and functionalities. The study revealed several design vulnerabilities. Tasks such as limiting the results, running complex queries, location of data and registering to access the database were challenging. The three critical attributes identified to have impact on the usability of the STEP database included (1) content and presentation (2) the navigation and search features (3) potential end-users. Evaluation framework proved to be an effective method for evaluating database effectiveness and user satisfaction. This study provides strong initial support for the usability of the STEP database. Recommendations would be incorporated into the refinement of the database to improve its usability and increase user participation towards the advancement of the database. Copyright © 2015 Elsevier B.V. All rights reserved.

Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics

PubMed Central

Tsopra, Rosy; Jais, Jean-Philippe; Venot, Alain; Duclos, Catherine

2014-01-01

Context It is important to consider the way in which information is presented by the interfaces of clinical decision support systems, to favor the adoption of these systems by physicians. Interface design can focus on decision processes (guided navigation) or usability principles. Objective The aim of this study was to compare these two approaches in terms of perceived usability, accuracy rate, and confidence in the system. Materials and methods We displayed clinical practice guidelines for antibiotic treatment via two types of interface, which we compared in a crossover design. General practitioners were asked to provide responses for 10 clinical cases and the System Usability Scale (SUS) for each interface. We assessed SUS scores, the number of correct responses, and the confidence level for each interface. Results SUS score and percentage confidence were significantly higher for the interface designed according to usability principles (81 vs 51, p=0.00004, and 88.8% vs 80.7%, p=0.004). The percentage of correct responses was similar for the two interfaces. Discussion/conclusion The interface designed according to usability principles was perceived to be more usable and inspired greater confidence among physicians than the guided navigation interface. Consideration of usability principles in the construction of an interface—in particular ‘effective information presentation’, ‘consistency’, ‘efficient interactions’, ‘effective use of language’, and ‘minimizing cognitive load’—seemed to improve perceived usability and confidence in the system. PMID:24008427

Operating manual-based usability evaluation of medical devices: an effective patient safety screening method.

PubMed

Turley, James P; Johnson, Todd R; Smith, Danielle Paige; Zhang, Jaijie; Brixey, Juliana J

2006-04-01

Use of medical devices often directly contributes to medical errors. Because it is difficult or impossible to change the design of existing devices, the best opportunity for improving medical device safety is during the purchasing process. However, most hospital personnel are not familiar with the usability evaluation methods designed to identify aspects of a user interface that do not support intuitive and safe use. A review of medical device operating manuals is proposed as a more practical method of usability evaluation. Operating manuals for five volumetric infusion pumps from three manufacturers were selected for this study (January-April 2003). Each manual's safety message content was evaluated to determine whether the message indicated a device design characteristic that violated known usability principles (heuristics) or indicated a violation of an affordance of the device. "Minimize memory load," with 65 violations, was the heuristic violated most frequently across pumps. Variations between pumps, including the frequency and severity of violations for each, were noted. Results suggest that manual review can provide a proxy for heuristic evaluation of the actual medical device. This method, intended to be a component of prepurchasing evaluation, can complement more formal usability evaluation methods and be used to select a subset of devices for more extensive and formal testing.

Alternative images for perpendicular parking : a usability test of a multi-camera parking assistance system.

DOT National Transportation Integrated Search

2004-10-01

The parking assistance system evaluated consisted of four outward facing cameras whose images could be presented on a monitor on the center console. The images presented varied in the location of the virtual eye point of the camera (the height above ...

Content Appraisal as a Method for Measuring the Effectiveness & Usability of Online Content

ERIC Educational Resources Information Center

Nadeau, Jenifer; Heidorn, Neely; Broady, Nick; Whittle, Jennifer

2012-01-01

Content appraisal is a simple, qualitative system to identify modifications to make website material more useful to clientele. This system provides a comprehensive evaluation of content, focuses on content strategy, identifies weaknesses, and provides recommendations for improvement. The criteria examined included knowledge level,…

Baseline Evaluations to Support Control Room Modernization at Nuclear Power Plants

DOE Office of Scientific and Technical Information (OSTI.GOV)

Boring, Ronald L.; Joe, Jeffrey C.

2015-02-01

For any major control room modernization activity at a commercial nuclear power plant (NPP) in the U.S., a utility should carefully follow the four phases prescribed by the U.S. Nuclear Regulatory Commission in NUREG-0711, Human Factors Engineering Program Review Model. These four phases include Planning and Analysis, Design, Verification and Validation, and Implementation and Operation. While NUREG-0711 is a useful guideline, it is written primarily from the perspective of regulatory review, and it therefore does not provide a nuanced account of many of the steps the utility might undertake as part of control room modernization. The guideline is largely summative—intendedmore » to catalog final products—rather than formative—intended to guide the overall modernization process. In this paper, we highlight two crucial formative sub-elements of the Planning and Analysis phase specific to control room modernization that are not covered in NUREG-0711. These two sub-elements are the usability and ergonomics baseline evaluations. A baseline evaluation entails evaluating the system as-built and currently in use. The usability baseline evaluation provides key insights into operator performance using the control system currently in place. The ergonomics baseline evaluation identifies possible deficiencies in the physical configuration of the control system. Both baseline evaluations feed into the design of the replacement system and subsequent summative benchmarking activities that help ensure that control room modernization represents a successful evolution of the control system.« less

Developing Software to “Track and Catch” Missed Follow-up of Abnormal Test Results in a Complex Sociotechnical Environment

PubMed Central

Smith, M.; Murphy, D.; Laxmisan, A.; Sittig, D.; Reis, B.; Esquivel, A.; Singh, H.

2013-01-01

Summary Background Abnormal test results do not always receive timely follow-up, even when providers are notified through electronic health record (EHR)-based alerts. High workload, alert fatigue, and other demands on attention disrupt a provider’s prospective memory for tasks required to initiate follow-up. Thus, EHR-based tracking and reminding functionalities are needed to improve follow-up. Objectives The purpose of this study was to develop a decision-support software prototype enabling individual and system-wide tracking of abnormal test result alerts lacking follow-up, and to conduct formative evaluations, including usability testing. Methods We developed a working prototype software system, the Alert Watch And Response Engine (AWARE), to detect abnormal test result alerts lacking documented follow-up, and to present context-specific reminders to providers. Development and testing took place within the VA’s EHR and focused on four cancer-related abnormal test results. Design concepts emphasized mitigating the effects of high workload and alert fatigue while being minimally intrusive. We conducted a multifaceted formative evaluation of the software, addressing fit within the larger socio-technical system. Evaluations included usability testing with the prototype and interview questions about organizational and workflow factors. Participants included 23 physicians, 9 clinical information technology specialists, and 8 quality/safety managers. Results Evaluation results indicated that our software prototype fit within the technical environment and clinical workflow, and physicians were able to use it successfully. Quality/safety managers reported that the tool would be useful in future quality assurance activities to detect patients who lack documented follow-up. Additionally, we successfully installed the software on the local facility’s “test” EHR system, thus demonstrating technical compatibility. Conclusion To address the factors involved in missed test results, we developed a software prototype to account for technical, usability, organizational, and workflow needs. Our evaluation has shown the feasibility of the prototype as a means of facilitating better follow-up for cancer-related abnormal test results. PMID:24155789

Developing software to "track and catch" missed follow-up of abnormal test results in a complex sociotechnical environment.

PubMed

Smith, M; Murphy, D; Laxmisan, A; Sittig, D; Reis, B; Esquivel, A; Singh, H

2013-01-01

Abnormal test results do not always receive timely follow-up, even when providers are notified through electronic health record (EHR)-based alerts. High workload, alert fatigue, and other demands on attention disrupt a provider's prospective memory for tasks required to initiate follow-up. Thus, EHR-based tracking and reminding functionalities are needed to improve follow-up. The purpose of this study was to develop a decision-support software prototype enabling individual and system-wide tracking of abnormal test result alerts lacking follow-up, and to conduct formative evaluations, including usability testing. We developed a working prototype software system, the Alert Watch And Response Engine (AWARE), to detect abnormal test result alerts lacking documented follow-up, and to present context-specific reminders to providers. Development and testing took place within the VA's EHR and focused on four cancer-related abnormal test results. Design concepts emphasized mitigating the effects of high workload and alert fatigue while being minimally intrusive. We conducted a multifaceted formative evaluation of the software, addressing fit within the larger socio-technical system. Evaluations included usability testing with the prototype and interview questions about organizational and workflow factors. Participants included 23 physicians, 9 clinical information technology specialists, and 8 quality/safety managers. Evaluation results indicated that our software prototype fit within the technical environment and clinical workflow, and physicians were able to use it successfully. Quality/safety managers reported that the tool would be useful in future quality assurance activities to detect patients who lack documented follow-up. Additionally, we successfully installed the software on the local facility's "test" EHR system, thus demonstrating technical compatibility. To address the factors involved in missed test results, we developed a software prototype to account for technical, usability, organizational, and workflow needs. Our evaluation has shown the feasibility of the prototype as a means of facilitating better follow-up for cancer-related abnormal test results.

An Expert System for Diagnosing Eye Diseases using Forward Chaining Method

NASA Astrophysics Data System (ADS)

Munaiseche, C. P. C.; Kaparang, D. R.; Rompas, P. T. D.

2018-02-01

Expert System is a system that seeks to adopt human knowledge to the computer, so that the computer can solve problems which are usually done by experts. The purpose of medical expert system is to support the diagnosis process of physicians. It considers facts and symptoms to provide diagnosis. This implies that a medical expert system uses knowledge about diseases and facts about the patients to suggest diagnosis. The aim of this research is to design an expert system application for diagnosing eye diseases using forward chaining method and to figure out user acceptance to this application through usability testing. Eye is selected because it is one of the five senses which is very sensitive and important. The scope of the work is extended to 16 types of eye diseases with 41 symptoms of the disease, arranged in 16 rules. The computer programming language employed was the PHP programming language and MySQL as the Relational Database Management System (RDBMS). The results obtained showed that the expert system was able to successfully diagnose eye diseases corresponding to the selected symptoms entered as query and the system evaluation through usability testing showed the expert system for diagnosis eye diseases had very good rate of usability, which includes learnability, efficiency, memorability, errors, and satisfaction so that the system can be received in the operational environment.

A Wearable Sensor-Based Exercise Biofeedback System: Mixed Methods Evaluation of Formulift

PubMed Central

Slevin, Patrick; Ward, Tomas; Caulfield, Brian

2018-01-01

Background Formulift is a newly developed mobile health (mHealth) app that connects to a single inertial measurement unit (IMU) worn on the left thigh. The IMU captures users’ movements as they exercise, and the app analyzes the data to count repetitions in real time and classify users’ exercise technique. The app also offers feedback and guidance to users on exercising safely and effectively. Objective The aim of this study was to assess the Formulift system with three different and realistic types of potential users (beginner gym-goers, experienced gym-goers, and qualified strength and conditioning [S&C] coaches) under a number of categories: (1) usability, (2) functionality, (3) the perceived impact of the system, and (4) the subjective quality of the system. It was also desired to discover suggestions for future improvements to the system. Methods A total of 15 healthy volunteers participated (12 males; 3 females; age: 23.8 years [SD 1.80]; height: 1.79 m [SD 0.07], body mass: 78.4 kg [SD 9.6]). Five participants were beginner gym-goers, 5 were experienced gym-goers, and 5 were qualified and practicing S&C coaches. IMU data were first collected from each participant to create individualized exercise classifiers for them. They then completed a number of nonexercise-related tasks with the app. Following this, a workout was completed using the system, involving squats, deadlifts, lunges, and single-leg squats. Participants were then interviewed about their user experience and completed the System Usability Scale (SUS) and the user version of the Mobile Application Rating Scale (uMARS). Thematic analysis was completed on all interview transcripts, and survey results were analyzed. Results Qualitative and quantitative analysis found the system has “good” to “excellent” usability. The system achieved a mean (SD) SUS usability score of 79.2 (8.8). Functionality was also deemed to be good, with many users reporting positively on the systems repetition counting, technique classification, and feedback. A number of bugs were found, and other suggested changes to the system were also made. The overall subjective quality of the app was good, with a median star rating of 4 out of 5 (interquartile range, IQR: 3-5). Participants also reported that the system would aid their technique, provide motivation, reassure them, and help them avoid injury. Conclusions This study demonstrated an overall positive evaluation of Formulift in the categories of usability, functionality, perceived impact, and subjective quality. Users also suggested a number of changes for future iterations of the system. These findings are the first of their kind and show great promise for wearable sensor-based exercise biofeedback systems. PMID:29386171

Design and Evaluation of Virtual Reality-Based Therapy Games with Dual Focus on Therapeutic Relevance and User Experience for Children with Cerebral Palsy.

PubMed

Ni, Lian Ting; Fehlings, Darcy; Biddiss, Elaine

2014-06-01

Virtual reality (VR)-based therapy for motor rehabilitation of children with cerebral palsy (CP) is growing in prevalence. Although mainstream active videogames typically offer children an appealing user experience, they are not designed for therapeutic relevance. Conversely, rehabilitation-specific games often struggle to provide an immersive experience that sustains interest. This study aims to design and evaluate two VR-based therapy games for upper and lower limb rehabilitation and to evaluate their efficacy with dual focus on therapeutic relevance and user experience. Three occupational therapists, three physiotherapists, and eight children (8-12 years old), with CP Level I-III on the Gross Motor Function Classification System, evaluated two games for the Microsoft(®) (Redmond, WA) Kinect™ for Windows and completed the System Usability Scale (SUS), Physical Activity Enjoyment Scale (PACES), and custom feedback questionnaires. Children and therapists unanimously agreed on the enjoyment and therapeutic value of the games. Median scores on the PACES were high (6.24±0.95 on the 7-point scale). Therapists considered the system to be of average usability (50th percentile on the SUS). The most prevalent usability issue was detection errors distinguishing the child's movements from the supporting therapist's. The ability to adjust difficulty settings and to focus on targeted goals (e.g., elbow/shoulder extension, weight shifting) was highly valued by therapists. Engaging both therapists and children in a user-centered design approach enabled the development of two VR-based therapy games for upper and lower limb rehabilitation that are dually (a) engaging to the child and (b) therapeutically relevant.

Evaluating the Usability and Accessibility of LMS "Blackboard" at King Saud University

ERIC Educational Resources Information Center

Alturki, Uthman T.; Aldraiweesh, Ahmed; Kinshuck

2016-01-01

King Saud University is in the process of adopting and implementing the interactive Blackboard Learning Management Systems (LMSs) with features that allow members of staff and teachers from different faculties to access, upload assignments, send quizzes, download content, and evaluate the academic progress of the members of faculty. However, many…

Usability of Videogame-Based Dexterity Training in the Early Rehabilitation Phase of Stroke Patients: A Pilot Study.

PubMed

Vanbellingen, Tim; Filius, Suzanne J; Nyffeler, Thomas; van Wegen, Erwin E H

2017-01-01

Approximately 70-80% of stroke survivors have limited activities of daily living, mainly due to dexterous problems. Videogame-based training (VBT) along with virtual reality seems to be beneficial to train upper limb function. To evaluate the usability of VBT using the Leap Motion Controller (LMC) to train fine manual dexterity in the early rehabilitation phase of stroke patients as an add-on to conventional therapy. Additionally, this study aimed to estimate the feasibility and potential efficacy of the VBT. During 3 months, 64 stroke patients were screened for eligibility, 13 stroke patients were included (4 women and 9 men; age range: 24-91 years; mean time post stroke: 28.2 days). Nine sessions of 30 min VBT, three times per week as an add-on to conventional therapy with stroke inpatients. Primary outcome was the usability of the system measured with the System Usability Scale. Secondary outcomes concerning feasibility were the compliance rate calculated from the total time spent on the intervention (TT) compared to planned time, the opinion of participants via open-end questions, and the level of active participation measured with the Pittsburgh Rehabilitation Participation Scale. Regarding the potential efficacy secondary outcomes were: functional dexterity measured with the Nine Hole Peg Test (NHPT), subjective dexterity measured with the Dexterity Questionnaire 24, grip strength measured with the Jamar dynamometer, and motor impairment of the upper limb measured with the Fugl-Meyer Upper Extremity (FM-UE) scale. Primarily, the usability of the system was good to excellent. The patient's perception of usability remained stable over a mean period of 3 weeks of VBT. Secondly, the compliance rate was good, and the level of active participation varied between good and very good. The opinion of the participants revealed that despite individual differences, the overall impression of the therapy and device was good. Patients showed significant improvements in hand dexterity. No changes were found in motor impairment of the upper limb (FM-UE) during intervention. VBT using LMC is a usable rehabilitation tool to train dexterity in the early rehabilitation phase of stroke inpatients.

The Relationship between National Culture and the Usability of an E-Learning System

ERIC Educational Resources Information Center

Downey, Steve; Wentling, Rose Mary; Wentling, Tim; Wadsworth, Andrew

2004-01-01

This study sought to measure the relationship between national culture and the usability of an e-Learning system by using Hofstede's cultural dimensions and Nielson's usability attributes. The study revealed that high uncertainty avoidance cultures found the system more frustrating to use. The study also revealed that individuals from cultures…

[Relations between health information systems and patient safety].

PubMed

Nøhr, Christian

2012-11-05

Health information systems have the potential to reduce medical errors, and indeed many studies have shown a significant reduction. However, if the systems are not designed and implemented properly, there is evidence that suggest that new types of errors will arise--i.e., technology-induced errors. Health information systems will need to undergo a more rigorous evaluation. Usability evaluation and simulation test with humans in the loop can help to detect and prevent technology-induced errors before they are deployed in real health-care settings.

PHIT for Duty, a Mobile Application for Stress Reduction, Sleep Improvement, and Alcohol Moderation.

PubMed

Kizakevich, Paul N; Eckhoff, Randall; Brown, Janice; Tueller, Stephen J; Weimer, Belinda; Bell, Stacey; Weeks, Adam; Hourani, Laurel L; Spira, James L; King, Laurel A

2018-03-01

Post-traumatic stress and other problems often occur after combat, deployment, and other military operations. Because techniques such as mindfulness meditation show efficacy in improving mental health, our team developed a mobile application (app) for individuals in the armed forces with subclinical psychological problems as secondary prevention of more significant disease. Based on the Personal Health Intervention Toolkit (PHIT), a mobile app framework for personalized health intervention studies, PHIT for Duty integrates mindfulness-based relaxation, behavioral education in sleep quality and alcohol use, and psychometric and psychophysiological data capture. We evaluated PHIT for Duty in usability and health assessment studies to establish app quality for use in health research. Participants (N = 31) rated usability on a 1 (very hard) to 5 (very easy) scale and also completed the System Usability Scale (SUS) questionnaire (N = 9). Results were (mean ± SD) overall (4.5 ± 0.6), self-report instruments (4.5 ± 0.7), pulse sensor (3.7 ± 1.2), sleep monitor (4.4 ± 0.7), sleep monitor comfort (3.7 ± 1.1), and wrist actigraphy comfort (2.7 ± 0.9). The average SUS score was 85 ± 12, indicating a rank of 95%. A comparison of PHIT-based assessments to traditional paper forms demonstrated a high overall correlation (r = 0.87). These evaluations of usability, health assessment accuracy, physiological sensing, system acceptability, and overall functionality have shown positive results and affirmation for using the PHIT framework and PHIT for Duty application in mobile health research.

The usability of ventilators: a comparative evaluation of use safety and user experience.

PubMed

Morita, Plinio P; Weinstein, Peter B; Flewwelling, Christopher J; Bañez, Carleene A; Chiu, Tabitha A; Iannuzzi, Mario; Patel, Aastha H; Shier, Ashleigh P; Cafazzo, Joseph A

2016-08-20

The design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices. We carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator. Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX). Nine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen's d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p = 0.044) and V500 (p = 0.020). Participants reported higher system usability for G5 when compared to PB980 (p = 0.035) and higher system usability for SERVO-U when compared to G5 (p < 0.001), PB980 (p < 0.001), and V500 (p < 0.001). Participants reported lower workload for G5 when compared to PB980 (p < 0.001) and lower workload for SERVO-U when compared to PB980 (p < 0.001) and V500 (p < 0.001). G5 scored better on two of nine possible comparisons; SERVO-U scored better on seven of nine possible comparisons. Aspects influencing participants' performance and perception include the low sensitivity of G5's touchscreen and the positive effect from the quality of SERVO-U's user interface design. This study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users' expectations.

Approaches for Evaluating the Usability of Assistive Technology Product Prototypes

ERIC Educational Resources Information Center

Choi, Young Mi; Sprigle, Stephen H.

2011-01-01

User input is an important component to help guide designers in producing a more usable product. Evaluation of prototypes is one method of obtaining this input, but methods for evaluating assistive technology prototypes during design have not been adequately described or evaluated. This project aimed to compare different methods of evaluating…

Research-IQ: Development and Evaluation of an Ontology-anchored Integrative Query Tool

PubMed Central

Borlawsky, Tara B.; Lele, Omkar; Payne, Philip R. O.

2011-01-01

Investigators in the translational research and systems medicine domains require highly usable, efficient and integrative tools and methods that allow for the navigation of and reasoning over emerging large-scale data sets. Such resources must cover a spectrum of granularity from bio-molecules to population phenotypes. Given such information needs, we report upon the initial design and evaluation of an ontology-anchored integrative query tool, Research-IQ, which employs a combination of conceptual knowledge engineering and information retrieval techniques to enable the intuitive and rapid construction of queries, in terms of semi-structured textual propositions, that can subsequently be applied to integrative data sets. Our initial results, based upon both quantitative and qualitative evaluations of the efficacy and usability of Research-IQ, demonstrate its potential to increase clinical and translational research throughput. PMID:21821150

Mobile Applications for Participation at the Shopping Mall: Content Analysis and Usability for Persons with Physical Disabilities and Communication or Cognitive Limitations

PubMed Central

Auger, Claudine; Leduc, Emilie; Labbé, Delphine; Guay, Cassioppée; Fillion, Brigitte; Bottari, Carolina; Swaine, Bonnie

2014-01-01

The aim of this exploratory study was to determine the important features in content and usability of existing mobile applications evaluating environmental barriers and facilitators (EBF) to participation for persons with physical disabilities presenting mild communication or cognitive limitations. A rigorous process based on a user-centered design approach led to the identification of two relevant mobile applications to evaluate the EBF. An accessibility expert, the research team as well as five users then tested the mobile applications in a shopping mall. A thematic content analysis of the research team’s and users’ comments established 10 categories of key features that adequately respond to the needs of the clientele targeted in this study. In terms of content, granularity and contextualization of the information provided were considered important. With respect to usability, relevant features were place finding, rating system, presentation of results, compatibility, user-friendliness, aesthetics, credibility of the information as well as connectivity/interactiveness. The research team and the users agreed on some aspects such as aesthetics, but had different perspectives on features such as the rating system or the connectivity/interactiveness of the application. The users proposed new features suggesting that the existing mobile applications did not correspond to all their needs. PMID:25513999

Mobile applications for participation at the shopping mall: content analysis and usability for persons with physical disabilities and communication or cognitive limitations.

PubMed

Auger, Claudine; Leduc, Emilie; Labbé, Delphine; Guay, Cassioppée; Fillion, Brigitte; Bottari, Carolina; Swaine, Bonnie

2014-12-01

The aim of this exploratory study was to determine the important features in content and usability of existing mobile applications evaluating environmental barriers and facilitators (EBF) to participation for persons with physical disabilities presenting mild communication or cognitive limitations. A rigorous process based on a user-centered design approach led to the identification of two relevant mobile applications to evaluate the EBF. An accessibility expert, the research team as well as five users then tested the mobile applications in a shopping mall. A thematic content analysis of the research team's and users' comments established 10 categories of key features that adequately respond to the needs of the clientele targeted in this study. In terms of content, granularity and contextualization of the information provided were considered important. With respect to usability, relevant features were place finding, rating system, presentation of results, compatibility, user-friendliness, aesthetics, credibility of the information as well as connectivity/interactiveness. The research team and the users agreed on some aspects such as aesthetics, but had different perspectives on features such as the rating system or the connectivity/interactiveness of the application. The users proposed new features suggesting that the existing mobile applications did not correspond to all their needs.

Usability evaluation of a medication reconciliation tool: Embedding safety probes to assess users' detection of medication discrepancies.

PubMed

Russ, Alissa L; Jahn, Michelle A; Patel, Himalaya; Porter, Brian W; Nguyen, Khoa A; Zillich, Alan J; Linsky, Amy; Simon, Steven R

2018-06-01

An electronic medication reconciliation tool was previously developed by another research team to aid provider-patient communication for medication reconciliation. To evaluate the usability of this tool, we integrated artificial safety probes into standard usability methods. The objective of this article is to describe this method of using safety probes, which enabled us to evaluate how well the tool supports users' detection of medication discrepancies. We completed a mixed-method usability evaluation in a simulated setting with 30 participants: 20 healthcare professionals (HCPs) and 10 patients. We used factual scenarios but embedded three artificial safety probes: (1) a missing medication (i.e., omission); (2) an extraneous medication (i.e., commission); and (3) an inaccurate dose (i.e., dose discrepancy). We measured users' detection of each probe to estimate the probability that a HCP or patient would detect these discrepancies. Additionally, we recorded participants' detection of naturally occurring discrepancies. Each safety probe was detected by ≤50% of HCPs. Patients' detection rates were generally higher. Estimates indicate that a HCP and patient, together, would detect 44.8% of these medication discrepancies. Additionally, HCPs and patients detected 25 and 45 naturally-occurring discrepancies, respectively. Overall, detection of medication discrepancies was low. Findings indicate that more advanced interface designs are warranted. Future research is needed on how technologies can be designed to better aid HCPs' and patients' detection of medication discrepancies. This is one of the first studies to evaluate the usability of a collaborative medication reconciliation tool and assess HCPs' and patients' detection of medication discrepancies. Results demonstrate that embedded safety probes can enhance standard usability methods by measuring additional, clinically-focused usability outcomes. The novel safety probes we used may serve as an initial, standard set for future medication reconciliation research. More prevalent use of safety probes could strengthen usability research for a variety of health information technologies. Published by Elsevier Inc.

Usability Assessment of Secure Messaging for Clinical Document Sharing between Health Care Providers and Patients.

PubMed

Jahn, Michelle A; Porter, Brian W; Patel, Himalaya; Zillich, Alan J; Simon, Steven R; Russ, Alissa L

2018-04-01

 Web-based patient portals feature secure messaging systems that enable health care providers and patients to communicate information. However, little is known about the usability of these systems for clinical document sharing.  This article evaluates the usability of a secure messaging system for providers and patients in terms of its ability to support sharing of electronic clinical documents.  We conducted usability testing with providers and patients in a human-computer interaction laboratory at a Midwestern U.S. hospital. Providers sent a medication list document to a fictitious patient via secure messaging. Separately, patients retrieved the clinical document from a secure message and returned it to a fictitious provider. We collected use errors, task completion, task time, and satisfaction.  Twenty-nine individuals participated: 19 providers (6 physicians, 6 registered nurses, and 7 pharmacists) and 10 patients. Among providers, 11 (58%) attached and sent the clinical document via secure messaging without requiring assistance, in a median (range) of 4.5 (1.8-12.7) minutes. No patients completed tasks without moderator assistance. Patients accessed the secure messaging system within 3.6 (1.2-15.0) minutes; retrieved the clinical document within 0.8 (0.5-5.7) minutes; and sent the attached clinical document in 6.3 (1.5-18.1) minutes. Although median satisfaction ratings were high, with 5.8 for providers and 6.0 for patients (scale, 0-7), we identified 36 different use errors. Physicians and pharmacists requested additional features to support care coordination via health information technology, while nurses requested features to support efficiency for their tasks.  This study examined the usability of clinical document sharing, a key feature of many secure messaging systems. Our results highlight similarities and differences between provider and patient end-user groups, which can inform secure messaging design to improve learnability and efficiency. The observations suggest recommendations for improving the technical aspects of secure messaging for clinical document sharing. Schattauer GmbH Stuttgart.

Mobile Applications for Diabetics: A Systematic Review and Expert-Based Usability Evaluation Considering the Special Requirements of Diabetes Patients Age 50 Years or Older

PubMed Central

Quade, Mandy; Kirch, Wilhelm

2014-01-01

Background A multitude of mhealth (mobile health) apps have been developed in recent years to support effective self-management of patients with diabetes mellitus type 1 or 2. Objective We carried out a systematic review of all currently available diabetes apps for the operating systems iOS and Android. We considered the number of newly released diabetes apps, range of functions, target user groups, languages, acquisition costs, user ratings, available interfaces, and the connection between acquisition costs and user ratings. Additionally, we examined whether the available applications serve the special needs of diabetes patients aged 50 or older by performing an expert-based usability evaluation. Methods We identified relevant keywords, comparative categories, and their specifications. Subsequently, we performed the app review based on the information given in the Google Play Store, the Apple App Store, and the apps themselves. In addition, we carried out an expert-based usability evaluation based on a representative 10% sample of diabetes apps. Results In total, we analyzed 656 apps finding that 355 (54.1%) offered just one function and 348 (53.0%) provided a documentation function. The dominating app language was English (85.4%, 560/656), patients represented the main user group (96.0%, 630/656), and the analysis of the costs revealed a trend toward free apps (53.7%, 352/656). The median price of paid apps was €1.90. The average user rating was 3.6 stars (maximum 5). Our analyses indicated no clear differences in the user rating between free and paid apps. Only 30 (4.6%) of the 656 available diabetes apps offered an interface to a measurement device. We evaluated 66 apps within the usability evaluation. On average, apps were rated best regarding the criterion “comprehensibility” (4.0 out of 5.0), while showing a lack of “fault tolerance” (2.8 out of 5.0). Of the 66 apps, 48 (72.7%) offered the ability to read the screen content aloud. The number of functions was significantly negative correlated with usability. The presence of documentation and analysis functions reduced the usability score significantly by 0.36 and 0.21 points. Conclusions A vast number of diabetes apps already exist, but the majority offer similar functionalities and combine only one to two functions in one app. Patients and physicians alike should be involved in the app development process to a greater extent. We expect that the data transmission of health parameters to physicians will gain more importance in future applications. The usability of diabetes apps for patients aged 50 or older was moderate to good. But this result applied mainly to apps offering a small range of functions. Multifunctional apps performed considerably worse in terms of usability. Moreover, the presence of a documentation or analysis function resulted in significantly lower usability scores. The operability of accessibility features for diabetes apps was quite limited, except for the feature “screen reader”. PMID:24718852

Assessment of the usability of a digital learning technology prototype for monitoring intracranial pressure 1

PubMed Central

de Carvalho, Lilian Regina; Évora, Yolanda Dora Martinez; Zem-Mascarenhas, Silvia Helena

2016-01-01

ABSTRACT Objective: to assess the usability of a digital learning technology prototype as a new method for minimally invasive monitoring of intracranial pressure. Method: descriptive study using a quantitative approach on assessing the usability of a prototype based on Nielsen's ten heuristics. Four experts in the area of Human-Computer interaction participated in the study. Results: the evaluation delivered eight violated heuristics and 31 usability problems in the 32 screens of the prototype. Conclusion: the suggestions of the evaluators were critical for developing an intuitive, user-friendly interface and will be included in the final version of the digital learning technology. PMID:27579932

Development of a Prototype Continuity of Care Record with Context-Specific Links to Meet the Information Needs of Case Managers for Persons Living with HIV

PubMed Central

Cimino, James J.; Bakken, Suzanne

2012-01-01

Objectives (1) To develop a prototype Continuity of Care Record (CCR) with context-specific links to electronic HIV information resources; and (2) to assess case managers’ perceptions regarding the usability of the prototype. Methods We integrated context-specific links to HIV case management information resources into a prototype CCR using the Infobutton Manager and Librarian Infobutton Tailoring Environment (LITE). Case managers (N=9) completed a think-aloud protocol and the Computer System Usability Questionnaire (CSUQ) to evaluate the usability of the prototype. Verbalizations from the think-aloud protocol were summarized using thematic analysis. CSUQ data were analyzed with descriptive statistics. Results Although participants expressed positive comments regarding the usability of the prototype, the think-aloud protocol also identified the need for improvement in resource labels and for additional resources. On a scale ranging from 1 (strongly agree) to 7 (strongly disagree), the average CSUQ overall satisfaction was 2.25 indicating that users (n=9) were generally satisfied with the system. Mean CSUQ factor scores were: System Usefulness (M=2.13), Information Quality (M=2.46), and Interface Quality (M=2.26). Conclusion Our novel application of the Infobutton Manager and LITE in the context of case management for persons living with HIV in community-based settings resulted in a prototype CCR with infobuttons that met the majority of case managers’ information needs and received relatively positive usability ratings. Findings from this study inform future integration of context-specific links into CCRs and electronic health records and support their use for meeting end-users information needs. PMID:22632821

Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results.

PubMed

Yen, Po-Yin; Sousa, Karen H; Bakken, Suzanne

2014-10-01

In a previous study, we developed the Health Information Technology Usability Evaluation Scale (Health-ITUES), which is designed to support customization at the item level. Such customization matches the specific tasks/expectations of a health IT system while retaining comparability at the construct level, and provides evidence of its factorial validity and internal consistency reliability through exploratory factor analysis. In this study, we advanced the development of Health-ITUES to examine its construct validity and predictive validity. The health IT system studied was a web-based communication system that supported nurse staffing and scheduling. Using Health-ITUES, we conducted a cross-sectional study to evaluate users' perception toward the web-based communication system after system implementation. We examined Health-ITUES's construct validity through first and second order confirmatory factor analysis (CFA), and its predictive validity via structural equation modeling (SEM). The sample comprised 541 staff nurses in two healthcare organizations. The CFA (n=165) showed that a general usability factor accounted for 78.1%, 93.4%, 51.0%, and 39.9% of the explained variance in 'Quality of Work Life', 'Perceived Usefulness', 'Perceived Ease of Use', and 'User Control', respectively. The SEM (n=541) supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. The results of CFA and SEM provide additional evidence for the construct and predictive validity of Health-ITUES. The customizability of Health-ITUES has the potential to support comparisons at the construct level, while allowing variation at the item level. We also illustrate application of Health-ITUES across stages of system development. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

USEQ: A Short Questionnaire for Satisfaction Evaluation of Virtual Rehabilitation Systems.

PubMed

Gil-Gómez, José-Antonio; Manzano-Hernández, Pilar; Albiol-Pérez, Sergio; Aula-Valero, Carmen; Gil-Gómez, Hermenegildo; Lozano-Quilis, José-Antonio

2017-07-07

New emerging technologies have proven their efficacy in aiding people in their rehabilitation. The tests that are usually used to evaluate usability (in general) or user satisfaction (in particular) of this technology are not specifically focused on virtual rehabilitation and patients. The objective of this contribution is to present and evaluate the USEQ (User Satisfaction Evaluation Questionnaire). The USEQ is a questionnaire that is designed to properly evaluate the satisfaction of the user (which constitutes part of usability) in virtual rehabilitation systems. Forty patients with balance disorders completed the USEQ after their first session with ABAR (Active Balance Rehabilitation), which is a virtual rehabilitation system that is designed for the rehabilitation of balance disorders. Internal consistency analysis and exploratory factor analysis were carried out to identify the factor structure of the USEQ. The six items of USEQ were significantly associated with each other, and the Cronbach alpha coefficient for the questionnaire was 0.716. In an analysis of the principal components, a one-factor solution was considered to be appropriate. The findings of the study suggest that the USEQ is a reliable questionnaire with adequate internal consistency. With regard to patient perception, the patients found the USEQ to be an easy-to-understand questionnaire with a convenient number of questions.

Randomised controlled trial to evaluate the efficacy and usability of a computerised phone-based lifestyle coaching system for primary and secondary prevention of stroke.

PubMed

Spassova, Lübomira; Vittore, Debora; Droste, Dirk W; Rösch, Norbert

2016-02-09

One of the most effective current approaches to preventing stroke events is the reduction of lifestyle risk factors, such as unhealthy diet, physical inactivity and smoking. In this study, we assessed the efficacy and usability of the phone-based Computer-aided Prevention System (CAPSYS) in supporting the reduction of lifestyle-related risk factors. A single-centre two-arm clinical trial was performed between January 2013 and February 2014, based on individual follow-up periods of six months with 94 patients at high risk of stroke, randomly assigned to an intervention group (IC: 48; advised to use the CAPSYS system) or a standard care group (SC: 46). Study parameters, such as blood pressure, blood values (HDL, LDL, HbA1c, glycaemia and triglycerides), weight, height, physical activity as well as nutrition and smoking habits were captured through questionnaires and medical records at baseline and post-intervention and analysed to detect significant changes. The usability of the intervention was assessed based on the standardised System Usability Scale (SUS) complemented by a more system-specific user satisfaction and feedback questionnaire. The statistical evaluation of primary measures revealed significant decreases of systolic blood pressure (mean of the differences = -9 mmHg; p = 0.03; 95% CI = [-17.29, -0.71]), LDL (pseudo-median of the differences = -7.9 mg/dl; p = 0.04; 95% CI = [-18.5, -0.5]) and triglyceride values (pseudo-median of the differences = -12.5 mg/dl; p = 0.04; 95% CI = [-26, -0.5]) in the intervention group, while no such changes could be observed in the control group. Furthermore, we detected a statistically significant increase in self-reported fruit and vegetable consumption (pseudo-median of the differences = 5.4 servings/week; p = 0.04; 95% CI = [0.5, 10.5]) and a decrease in sweets consumption (pseudo-median of the differences = -2 servings/week; p = 0.04; 95% CI = [-4, -0.00001]) in the intervention group. The usability assessment showed that the CAPSYS system was, in general, highly accepted by the users (average SUS score: 80.1). The study provided encouraging results indicating that a computerised phone-based lifestyle coaching system, such as CAPSYS, can support the usual treatment in reducing cerebro-cardiovascular risk factors and that such an approach is well applicable in practice. ClinicalTrials.gov Identifier: NCT02444715.

Evaluation of User Interface and Workflow Design of a Bedside Nursing Clinical Decision Support System

PubMed Central

Yuan, Michael Juntao; Finley, George Mike; Mills, Christy; Johnson, Ron Kim

2013-01-01

Background Clinical decision support systems (CDSS) are important tools to improve health care outcomes and reduce preventable medical adverse events. However, the effectiveness and success of CDSS depend on their implementation context and usability in complex health care settings. As a result, usability design and validation, especially in real world clinical settings, are crucial aspects of successful CDSS implementations. Objective Our objective was to develop a novel CDSS to help frontline nurses better manage critical symptom changes in hospitalized patients, hence reducing preventable failure to rescue cases. A robust user interface and implementation strategy that fit into existing workflows was key for the success of the CDSS. Methods Guided by a formal usability evaluation framework, UFuRT (user, function, representation, and task analysis), we developed a high-level specification of the product that captures key usability requirements and is flexible to implement. We interviewed users of the proposed CDSS to identify requirements, listed functions, and operations the system must perform. We then designed visual and workflow representations of the product to perform the operations. The user interface and workflow design were evaluated via heuristic and end user performance evaluation. The heuristic evaluation was done after the first prototype, and its results were incorporated into the product before the end user evaluation was conducted. First, we recruited 4 evaluators with strong domain expertise to study the initial prototype. Heuristic violations were coded and rated for severity. Second, after development of the system, we assembled a panel of nurses, consisting of 3 licensed vocational nurses and 7 registered nurses, to evaluate the user interface and workflow via simulated use cases. We recorded whether each session was successfully completed and its completion time. Each nurse was asked to use the National Aeronautics and Space Administration (NASA) Task Load Index to self-evaluate the amount of cognitive and physical burden associated with using the device. Results A total of 83 heuristic violations were identified in the studies. The distribution of the heuristic violations and their average severity are reported. The nurse evaluators successfully completed all 30 sessions of the performance evaluations. All nurses were able to use the device after a single training session. On average, the nurses took 111 seconds (SD 30 seconds) to complete the simulated task. The NASA Task Load Index results indicated that the work overhead on the nurses was low. In fact, most of the burden measures were consistent with zero. The only potentially significant burden was temporal demand, which was consistent with the primary use case of the tool. Conclusions The evaluation has shown that our design was functional and met the requirements demanded by the nurses’ tight schedules and heavy workloads. The user interface embedded in the tool provided compelling utility to the nurse with minimal distraction. PMID:23612350

Situated Usability Testing for Security Systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Greitzer, Frank L.

2011-03-02

While usability testing is well established, assessing the usability of security software, tools, or methods deserves more careful consideration. It has been argued that dealing with security has become too difficult for individuals or organizations to manage effectively or to use conveniently. As difficult as it is for system administrators and developers to deal with, security is even more challenging for casual users. Indeed, it is much too easy for casual/home users to configure the security of their systems in non-optimal ways that leave their systems inadvertently insecure. This is exacerbated by the fact that casual users are focused onmore » matters other than security, and likely would prefer not even to think about security. This brief report argues that when security and/or privacy are part of the equation, traditional methods for usability testing should be re-considered. The purpose of this brief report is to argue for and outline a method associated with a new approach to usability testing for examining usable security issues.« less

Usability and Instructional Design Heuristics for E-Learning Evaluation.

ERIC Educational Resources Information Center

Reeves, Thomas C.; Benson, Lisa; Elliott, Dean; Grant, Michael; Holschuh, Doug; Kim, Beaumie; Kim, Hyeonjin; Lauber, Erick; Loh, Sebastian

Heuristic evaluation is a methodology for investigating the usability of software originally developed by Nielsen (1993, 2000). Nielsen's protocol was modified and refined for evaluating e-learning programs by participants in a doctoral seminar held at the University of Georgia in 2001. The modifications primarily involved expanding Nielsen's…

Systematic Heuristic Evaluation of Computerized Consultation Order Templates: Clinicians' and Human Factors Engineers' Perspectives.

PubMed

Savoy, April; Patel, Himalaya; Flanagan, Mindy E; Weiner, Michael; Russ, Alissa L

2017-08-01

We assessed the usability of consultation order templates and identified problems to prioritize in design efforts for improving referral communication. With a sample of 26 consultation order templates, three evaluators performed a usability heuristic evaluation. The evaluation used 14 domain-independent heuristics and the following three supplemental references: 1 new domain-specific heuristic, 6 usability goals, and coded clinicians' statements regarding ease of use for 10 sampled templates. Evaluators found 201 violations, a mean of 7.7 violations per template. Minor violations outnumbered major violations almost twofold, 115 (57%) to 62 (31%). Approximately 68% of violations were linked to 5 heuristics: aesthetic and minimalist design (17%), error prevention (16%), consistency and standards (14%), recognition rather than recall (11%), and meet referrers' information needs (10%). Severe violations were attributed mostly to meet referrers' information needs and recognition rather than recall. Recorded violations yielded potential negative consequences for efficiency, effectiveness, safety, learnability, and utility. Evaluators and clinicians demonstrated 80% agreement in usability assessment. Based on frequency and severity of usability heuristic violations, the consultation order templates reviewed may impede clinical efficiency and risk patient safety. Results support the following design considerations: communicate consultants' requirements, facilitate information seeking, and support communication. While the most frequent heuristic violations involved interaction design and presentation, the most severe violations lacked information desired by referring clinicians. Violations related to templates' inability to support referring clinicians' information needs had the greatest potential negative impact on efficiency and safety usability goals. Heuristics should be prioritized in future design efforts.

Evaluation of the usability of a serious game aiming to teach facial expressions to schizophrenic patients.

PubMed

Isleyen, Filiz; Gulkesen, K Hakan; Cinemre, Buket; Samur, M Kemal; Zayim, Nese; Sen Kaya, Semiha

2014-01-01

In some psychological disorders such as autism and schizophrenia, loss of facial expression recognition skill may complicate patient's daily life. Information technology may help to develop facial expression recognition skill by educational software and games. We designed and developed an interactive web-based educational program with which we performed a usability study before investigating its effectiveness on the schizophrenia patients' ability of emotion perception. The purpose of this study is to describe the usability evaluation for a web-based game set that has been designed to teach facial expressions to schizophrenic patients. The usability study was done at two steps; first, we applied heuristic evaluation and the violations were rated in a scale from most to least severe and the major problems were solved. In the second step, think-aloud method was used and the web site was assessed by five schizophrenic patients. Eight experts participated in the heuristic evaluation, in which a total of 60 violations were identified with a mean severity of 2.77 (range: 0-4). All of the major problems (severity over 2.5) were listed and the usability problems were solved by the development team. After solving the problems, five users with a diagnosis of schizophrenia used the web site with the same scenario. They reported to have experienced minor, but different problems. In conclusion, we suggest that a combination of heuristic evaluation and think-aloud method may be an effective and efficient way for usability evaluations for the serious games that have been designed for special patient groups.

The Usability of SEEQ in Quality Evaluation of Arabic Secondary Education in Malaysia

ERIC Educational Resources Information Center

al-Muslim, M.; Arifin, Zamri

2015-01-01

Evaluation of the quality of Arabic education in Malaysia needs to be conducted on a continuous basis to achieve the objectives of Arabic education. Student Evaluation of Education Quality (SEEQ) was proposed as one of the evaluation instruments of the quality of Arabic education. This study aimed to evaluate the usability of SEEQ in the context…

Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia.

PubMed

Ben-Zeev, Dror; Kaiser, Susan M; Brenner, Christopher J; Begale, Mark; Duffecy, Jennifer; Mohr, David C

2013-12-01

Mobile Health (mHealth) approaches can support the rehabilitation of individuals with psychiatric conditions. In the current article, we describe the development of a smartphone illness self-management system for people with schizophrenia. The research was conducted with consumers and practitioners at a community-based rehabilitation agency. Stage 1: 904 individuals with schizophrenia or schizoaffective disorder completed a survey reporting on their current use of mobile devices and interest in mHealth services. Eight practitioners completed a survey examining their attitudes and expectations from an mHealth intervention, and identified needs and potential obstacles. Stage 2: A multidisciplinary team incorporated consumer and practitioner input and employed design principles for the development of e-resources for people with schizophrenia to produce an mHealth intervention. Stage 3: 12 consumers participated in laboratory usability sessions. They performed tasks involved in operating the new system, and provided "think aloud" commentary and post-session usability ratings. 570 (63%) of survey respondents reported owning a mobile device and many expressed interest in receiving mHealth services. Most practitioners believed that consumers could learn to use and would benefit from an mHealth intervention. In response, we developed a smartphone system that targets medication adherence, mood regulation, sleep, social functioning, and coping with symptoms. Usability testing revealed several design vulnerabilities, and the system was adapted to address consumer needs and preferences accordingly. Through a comprehensive development process, we produced an mHealth illness self-management intervention that is likely to be used successfully, and is ready for deployment and systemic evaluation in real-world conditions. PsycINFO Database Record (c) 2013 APA, all rights reserved.

Quality of human-computer interaction - results of a national usability survey of hospital-IT in Germany

PubMed Central

2011-01-01

Background Due to the increasing functionality of medical information systems, it is hard to imagine day to day work in hospitals without IT support. Therefore, the design of dialogues between humans and information systems is one of the most important issues to be addressed in health care. This survey presents an analysis of the current quality level of human-computer interaction of healthcare-IT in German hospitals, focused on the users' point of view. Methods To evaluate the usability of clinical-IT according to the design principles of EN ISO 9241-10 the IsoMetrics Inventory, an assessment tool, was used. The focus of this paper has been put on suitability for task, training effort and conformity with user expectations, differentiated by information systems. Effectiveness has been evaluated with the focus on interoperability and functionality of different IT systems. Results 4521 persons from 371 hospitals visited the start page of the study, while 1003 persons from 158 hospitals completed the questionnaire. The results show relevant variations between different information systems. Conclusions Specialised information systems with defined functionality received better assessments than clinical information systems in general. This could be attributed to the improved customisation of these specialised systems for specific working environments. The results can be used as reference data for evaluation and benchmarking of human computer engineering in clinical health IT context for future studies. PMID:22070880

Design and usability study of an iconic user interface to ease information retrieval of medical guidelines.

PubMed

Griffon, Nicolas; Kerdelhué, Gaétan; Hamek, Saliha; Hassler, Sylvain; Boog, César; Lamy, Jean-Baptiste; Duclos, Catherine; Venot, Alain; Darmoni, Stéfan J

2014-10-01

Doc'CISMeF (DC) is a semantic search engine used to find resources in CISMeF-BP, a quality controlled health gateway, which gathers guidelines available on the internet in French. Visualization of Concepts in Medicine (VCM) is an iconic language that may ease information retrieval tasks. This study aimed to describe the creation and evaluation of an interface integrating VCM in DC in order to make this search engine much easier to use. Focus groups were organized to suggest ways to enhance information retrieval tasks using VCM in DC. A VCM interface was created and improved using the ergonomic evaluation approach. 20 physicians were recruited to compare the VCM interface with the non-VCM one. Each evaluator answered two different clinical scenarios in each interface. The ability and time taken to select a relevant resource were recorded and compared. A usability analysis was performed using the System Usability Scale (SUS). The VCM interface contains a filter based on icons, and icons describing each resource according to focus group recommendations. Some ergonomic issues were resolved before evaluation. Use of VCM significantly increased the success of information retrieval tasks (OR=11; 95% CI 1.4 to 507). Nonetheless, it took significantly more time to find a relevant resource with VCM interface (101 vs 65 s; p=0.02). SUS revealed 'good' usability with an average score of 74/100. VCM was successfully implemented in DC as an option. It increased the success rate of information retrieval tasks, despite requiring slightly more time, and was well accepted by end-users. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

User needs analysis and usability assessment of DataMed - a biomedical data discovery index.

PubMed

Dixit, Ram; Rogith, Deevakar; Narayana, Vidya; Salimi, Mandana; Gururaj, Anupama; Ohno-Machado, Lucila; Xu, Hua; Johnson, Todd R

2017-11-30

To present user needs and usability evaluations of DataMed, a Data Discovery Index (DDI) that allows searching for biomedical data from multiple sources. We conducted 2 phases of user studies. Phase 1 was a user needs analysis conducted before the development of DataMed, consisting of interviews with researchers. Phase 2 involved iterative usability evaluations of DataMed prototypes. We analyzed data qualitatively to document researchers' information and user interface needs. Biomedical researchers' information needs in data discovery are complex, multidimensional, and shaped by their context, domain knowledge, and technical experience. User needs analyses validate the need for a DDI, while usability evaluations of DataMed show that even though aggregating metadata into a common search engine and applying traditional information retrieval tools are promising first steps, there remain challenges for DataMed due to incomplete metadata and the complexity of data discovery. Biomedical data poses distinct problems for search when compared to websites or publications. Making data available is not enough to facilitate biomedical data discovery: new retrieval techniques and user interfaces are necessary for dataset exploration. Consistent, complete, and high-quality metadata are vital to enable this process. While available data and researchers' information needs are complex and heterogeneous, a successful DDI must meet those needs and fit into the processes of biomedical researchers. Research directions include formalizing researchers' information needs, standardizing overviews of data to facilitate relevance judgments, implementing user interfaces for concept-based searching, and developing evaluation methods for open-ended discovery systems such as DDIs. © The Author 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Establishment and Usability Evaluation of an Interactive AR Learning System on Conservation of Fish

ERIC Educational Resources Information Center

Lin, Hao-Chiang Koong; Hsieh, Min-Chai; Wang, Cheng-Hung; Sie, Zong-Yuan; Chang, Shei-Hsi

2011-01-01

In this study, we develop an interactive AR Learning System based on Augmented Reality and interactive touch-screen. The learning content knowledge is about conservation of fish in Taiwan. The system combines the game by the concept of AR book which allows children to learn about the importance of conservation of fish. A mechanism is designed to…

Evaluating User Perceptions of Mobile Medication Management Applications With Older Adults: A Usability Study

PubMed Central

Gates, Allison

2014-01-01

Background Medication nonadherence has a significant impact on the health and wellbeing of individuals with chronic disease. Several mobile medication management applications are available to help users track, remember, and read about their medication therapy. Objective The objective of this study was to explore the usability and usefulness of existing medication management applications for older adults. Methods We recruited 35 participants aged 50 and over to participate in a 2-hour usability session. The average age ranged from 52-78 years (mean 67 years) and 71% (25/35) of participants were female. Each participant was provided with an iPad loaded with four medication management applications: MyMedRec, DrugHub, Pillboxie, and PocketPharmacist. These applications were evaluated using the 10 item System Usability Scale (SUS) and visual analog scale. An investigator-moderated 30-minute discussion followed, and was recorded. We used a grounded theory (GT) approach to analyze qualitative data. Results When assessing mobile medication management applications, participants struggled to think of a need for the applications in their own lives. Many were satisfied with their current management system and proposed future use only if cognition and health declined. Most participants felt capable of using the applications after a period of time and training, but were frustrated by their initial experiences with the applications. The early experiences of participants highlighted the benefits of linear navigation and clear wording (eg, “undo” vs “cancel”) when designing for older users. While there was no order effect, participants attributed their poor performance to the order in which they tried the applications. They also described being a part of a technology generation that did not encounter the computer until adulthood. Of the four applications, PocketPharmacist was found to be the least usable with a score of 42/100 (P<.0001) though it offered a drug interaction feature that was among the favorite features of participants. The usability scores for MyMedRec (56/100), DrugHub (57/100), and Pillboxie (52/100) were not significantly different and participants preferred MyMedRec and DrugHub for their simple, linear interfaces. Conclusions With training, adults aged 50 and over can be capable and interested in using mHealth applications for their medication management. However, in order to adopt such technology, they must find a need that their current medication management system cannot fill. Interface diversity and multimodal reminder methods should be considered to increase usability for older adults. Lastly, regulation or the involvement of older adults in development may help to alleviate generation bias and mistrust for applications. PMID:25099993

Defining Usability: How Library Practice Differs from Published Research

ERIC Educational Resources Information Center

Chen, Yu-Hui; Germain, Carol Anne; Rorissa, Abebe

2011-01-01

Library/information science professionals need a clearly articulated definition of usability/Web usability to implement intuitive websites. In this study, the authors analyzed usability definitions provided by the ARL library professionals and those found in the library/information science and computer science-information systems literature.…

NASA Johnson Space Center Usability Testing and Analysis facility (UTAF) Overview

NASA Technical Reports Server (NTRS)

Whitmore, Mihriban; Holden, Kritina L.

2005-01-01

The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility performs research for NASA's HumanSystems Integration Program, under the HumanSystems Research and Technology Division. Specifically, the UTAF provides human factors support for space vehicles, including the International Space Station, the Space Shuttle, and the forthcoming Crew Exploration Vehicle. In addition, there are ongoing collaborative research efforts with external corporations and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes and requirements. This presentation will provide an overview of ongoing activities, and will address how the UTAF projects will evolve to meet new space initiatives.

Usability issues concerning child restraint system harness design.

PubMed

Rudin-Brown, Christina M; Kumagai, Jason K; Angel, Harry A; Iwasa-Madge, Kim M; Noy, Y Ian

2003-05-01

A study was conducted to assess usability issues relating to child restraint system (CRS) harness design. Four convertible child restraint systems representing a wide variety of design features were used. Forty-two participants installed two child test dummies in both forward- and rear-facing configurations either inside or outside a test vehicle. Observer-scored checklists determined the degree to which each harness was installed correctly. Participant-scored questionnaires evaluated the 'ease-of-use' of various design features. While the percentage of correct installations exceeded 83% for all designs when installed in the forward-facing configuration, in the rear-facing position (that intended for children under 9-10 kg), there was a significant (between 65 and 89%) percentage of incorrect installations for all models. This finding is of particular interest and may be indicative of a more generalized problem with 'convertible' CRS designs when they are used in the rear-facing configuration. Furthermore, while certain design features were perceived by users as providing significantly better protection in the event of a collision, these also tended to be the features that were misused most often. The benefits and costs of various design features are discussed, and a method to test harness design usability is presented.

Dynamic Tunnel Usability Study: Format Recommendations for Synthetic Vision System Primary Flight Displays

NASA Technical Reports Server (NTRS)

Arthur, Jarvis J., III; Prinzel, Lawrence J., III; Kramer, Lynda J.; Bailey, Randall E.

2006-01-01

A usability study evaluating dynamic tunnel concepts has been completed under the Aviation Safety and Security Program, Synthetic Vision Systems Project. The usability study was conducted in the Visual Imaging Simulator for Transport Aircraft Systems (VISTAS) III simulator in the form of questionnaires and pilot-in-the-loop simulation sessions. Twelve commercial pilots participated in the study to determine their preferences via paired comparisons and subjective rankings regarding the color, line thickness and sensitivity of the dynamic tunnel. The results of the study showed that color was not significant in pilot preference paired comparisons or in pilot rankings. Line thickness was significant for both pilot preference paired comparisons and in pilot rankings. The preferred line/halo thickness combination was a line width of 3 pixels and a halo of 4 pixels. Finally, pilots were asked their preference for the current dynamic tunnel compared to a less sensitive dynamic tunnel. The current dynamic tunnel constantly gives feedback to the pilot with regard to path error while the less sensitive tunnel only changes as the path error approaches the edges of the tunnel. The tunnel sensitivity comparison results were not statistically significant.

Formal implementation of a performance evaluation model for the face recognition system.

PubMed

Shin, Yong-Nyuo; Kim, Jason; Lee, Yong-Jun; Shin, Woochang; Choi, Jin-Young

2008-01-01

Due to usability features, practical applications, and its lack of intrusiveness, face recognition technology, based on information, derived from individuals' facial features, has been attracting considerable attention recently. Reported recognition rates of commercialized face recognition systems cannot be admitted as official recognition rates, as they are based on assumptions that are beneficial to the specific system and face database. Therefore, performance evaluation methods and tools are necessary to objectively measure the accuracy and performance of any face recognition system. In this paper, we propose and formalize a performance evaluation model for the biometric recognition system, implementing an evaluation tool for face recognition systems based on the proposed model. Furthermore, we performed evaluations objectively by providing guidelines for the design and implementation of a performance evaluation system, formalizing the performance test process.

Evaluation of patient satisfaction with tailored online patient education information.

PubMed

Atack, Lynda; Luke, Robert; Chien, Elise

2008-01-01

Patients are using the Internet for access to standardized health information in ever-growing numbers. Although increased access to health information can be helpful, the quality of information varies widely. All too often, the information retrieved is incomplete, inaccurate, or inappropriate. An interdisciplinary team of clinicians, librarians, software engineers, and multimedia designers developed an online patient education system that enables clinicians to "prescribe" tailored, evidence-based health information. The system provides access to text and video that patients can adapt for language, vision, and hearing preferences. Usability testing was conducted with eight patients in a usability laboratory using the "think-aloud" method, surveys, and interviews. Results indicated that patients were highly satisfied and that the site has the potential to become a valuable resource in disease management. Patients made several recommendations regarding system appearance, function, and content that will have application for other groups developing online patient education systems.

The Skills, Competences, and Attitude toward Information and Communications Technology Recommender System: an online support program for teachers with personalized recommendations

NASA Astrophysics Data System (ADS)

Revilla Muñoz, Olga; Alpiste Penalba, Francisco; Fernández Sánchez, Joaquín

2016-01-01

Teachers deal with Information and Communications Technology (ICT) every day and they often have to solve problems by themselves. To help them in coping with this issue, an online support program has been created, where teachers can pose their problems on ICT and they can receive solutions from other teachers. A Recommender System has been defined and implemented into the support program to suggest to each teacher the most suitable solution based on her Skills, Competences, and Attitude toward ICT (SCAT-ICT). The support program has initially been populated with 70 problems from 86 teachers. 30 teachers grouped these problems into six categories with the card-sorting technique. Real solutions to these problems have been proposed by 25 trained teachers. Finally, 17 teachers evaluated the usability of the support program and the Recommender System, where results showed a high score on the standardized System Usability Scale.

Development and evaluation of RAMP I - a practitioner's tool for screening of musculoskeletal disorder risk factors in manual handling.

PubMed

Lind, Carl Mikael; Forsman, Mikael; Rose, Linda Maria

2017-10-16

RAMP I is a screening tool developed to support practitioners in screening for work-related musculoskeletal disorder risk factors related to manual handling. RAMP I, which is part of the RAMP tool, is based on research-based studies combined with expert group judgments. More than 80 practitioners participated in the development of RAMP I. The tool consists of dichotomous assessment items grouped into seven categories. Acceptable reliability was found for a majority of the assessment items for 15 practitioners who were given 1 h of training. The usability evaluation points to RAMP I being usable for screening for musculoskeletal disorder risk factors, i.e., usable for assessing risks, being usable as a decision base, having clear results and that the time needed for an assessment is acceptable. It is concluded that RAMP I is a usable tool for practitioners.

77 FR 25707 - Submission for OMB Review; Comment request

Federal Register 2010, 2011, 2012, 2013, 2014

2012-05-01

... and OMB Number: Electronic Health Record (EHR) Usability Survey; OMB Control Number 0720-TBD. Type of... information collection is to develop a longitudinal measure of how end-users perceive the usability of the... the focus has shifted towards understanding the usability of a system. This usability attribute...

A task-specific interactive game-based virtual reality rehabilitation system for patients with stroke: a usability test and two clinical experiments.

PubMed

Shin, Joon-Ho; Ryu, Hokyoung; Jang, Seong Ho

2014-03-06

Virtual reality (VR) is not commonly used in clinical rehabilitation, and commercial VR gaming systems may have mixed effects in patients with stroke. Therefore, we developed RehabMaster™, a task-specific interactive game-based VR system for post-stroke rehabilitation of the upper extremities, and assessed its usability and clinical efficacy. A participatory design and usability tests were carried out for development of RehabMaster with representative user groups. Two clinical trials were then performed. The first was an observational study in which seven patients with chronic stroke received 30 minutes of RehabMaster intervention per day for two weeks. The second was a randomised controlled trial of 16 patients with acute or subacute stroke who received 10 sessions of conventional occupational therapy only (OT-only group) or conventional occupational therapy plus 20 minutes of RehabMaster intervention (RehabMaster + OT group). The Fugl-Meyer Assessment score (FMA), modified Barthel Index (MBI), adverse effects, and drop-out rate were recorded. The requirements of a VR system for stroke rehabilitation were established and incorporated into RehabMaster. The reported advantages from the usability tests were improved attention, the immersive flow experience, and individualised intervention. The first clinical trial showed that the RehabMaster intervention improved the FMA (P = .03) and MBI (P = .04) across evaluation times. The second trial revealed that the addition of RehabMaster intervention tended to enhance the improvement in the FMA (P = .07) but did not affect the improvement in the MBI. One patient with chronic stroke left the trial, and no adverse effects were reported. The RehabMaster is a feasible and safe VR system for enhancing upper extremity function in patients with stroke.

A task-specific interactive game-based virtual reality rehabilitation system for patients with stroke: a usability test and two clinical experiments

PubMed Central

2014-01-01

Background Virtual reality (VR) is not commonly used in clinical rehabilitation, and commercial VR gaming systems may have mixed effects in patients with stroke. Therefore, we developed RehabMaster™, a task-specific interactive game-based VR system for post-stroke rehabilitation of the upper extremities, and assessed its usability and clinical efficacy. Methods A participatory design and usability tests were carried out for development of RehabMaster with representative user groups. Two clinical trials were then performed. The first was an observational study in which seven patients with chronic stroke received 30 minutes of RehabMaster intervention per day for two weeks. The second was a randomised controlled trial of 16 patients with acute or subacute stroke who received 10 sessions of conventional occupational therapy only (OT-only group) or conventional occupational therapy plus 20 minutes of RehabMaster intervention (RehabMaster + OT group). The Fugl-Meyer Assessment score (FMA), modified Barthel Index (MBI), adverse effects, and drop-out rate were recorded. Results The requirements of a VR system for stroke rehabilitation were established and incorporated into RehabMaster. The reported advantages from the usability tests were improved attention, the immersive flow experience, and individualised intervention. The first clinical trial showed that the RehabMaster intervention improved the FMA (P = .03) and MBI (P = .04) across evaluation times. The second trial revealed that the addition of RehabMaster intervention tended to enhance the improvement in the FMA (P = .07) but did not affect the improvement in the MBI. One patient with chronic stroke left the trial, and no adverse effects were reported. Conclusions The RehabMaster is a feasible and safe VR system for enhancing upper extremity function in patients with stroke. PMID:24597650

Quality and Usability of Arthritic Pain Self-Management Apps for Older Adults: A Systematic Review.

PubMed

Bhattarai, Priyanka; Newton-John, T R O; Phillips, Jane L

2018-03-01

To appraise the quality and usability of currently available pain applications that could be used by community-dwelling older adults to self-manage their arthritic pain. A systematic review. Searches were conducted in App Store and Google Play to identify pain self-management apps relevant to arthritic pain management. English language pain management apps providing pain assessment and documentation function and pain management education were considered for inclusion. A quality evaluation audit tool based on the Stanford Arthritis Self-Management Program was developed a priori to evaluate app content quality. The usability of included apps was assessed using an established usability evaluation tool. Out of the 373 apps that were identified, four met the inclusion criteria. The included apps all included a pain assessment and documentation function and instructions on medication use, communication with health professionals, cognitive behavioral therapy-based pain management, and physical exercise. Management of mood, depression, anxiety, and sleep were featured in most apps (N = 3). Three-quarters (N = 3) of the apps fell below the acceptable moderate usability score (≥3), while one app obtained a moderate score (3.2). Few of the currently available pain apps offer a comprehensive pain self-management approach incorporating evidence-based strategies in accordance with the Stanford Arthritis Self-Management Program. The moderate-level usability across the included apps indicates a need to consider the usability needs of the older population in future pain self-management app development endeavors.

A framework for evaluating electronic health record vendor user-centered design and usability testing processes.

PubMed

Ratwani, Raj M; Zachary Hettinger, A; Kosydar, Allison; Fairbanks, Rollin J; Hodgkins, Michael L

2017-04-01

Currently, there are few resources for electronic health record (EHR) purchasers and end users to understand the usability processes employed by EHR vendors during product design and development. We developed a framework, based on human factors literature and industry standards, to systematically evaluate the user-centered design processes and usability testing methods used by EHR vendors. We reviewed current usability certification requirements and the human factors literature to develop a 15-point framework for evaluating EHR products. The framework is based on 3 dimensions: user-centered design process, summative testing methodology, and summative testing results. Two vendor usability reports were retrieved from the Office of the National Coordinator's Certified Health IT Product List and were evaluated using the framework. One vendor scored low on the framework (5 pts) while the other vendor scored high on the framework (15 pts). The 2 scored vendor reports demonstrate the framework's ability to discriminate between the variabilities in vendor processes and to determine which vendors are meeting best practices. The framework provides a method to more easily comprehend EHR vendors' usability processes and serves to highlight where EHR vendors may be falling short in terms of best practices. The framework provides a greater level of transparency for both purchasers and end users of EHRs. The framework highlights the need for clearer certification requirements and suggests that the authorized certification bodies that examine vendor usability reports may need to be provided with clearer guidance. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Exploring the learnability and usability of a near field communication-based application for semantic enrichment in children with language disorders.

PubMed

Lorusso, Maria Luisa; Biffi, Emilia; Molteni, Massimo; Reni, Gianluigi

2018-01-01

Recently, a few software applications (apps) have been developed to enhance vocabulary and conceptual networks to address the needs of children with language impairments (LI), but there is no evidence about their impact and their usability in therapy contexts. Here, we try to fill this gap presenting a system aimed at improving the semantic competence and the structural knowledge of children with LI. The goal of the study is to evaluate learnability, usability, user satisfaction and quality of the interaction between the system and the children. The system consists of a tablet, hosting an app with educational and training purposes, equipped with a Near Field Communication (NFC) reader, used to interact with the user by means of objects. Fourteen preschool children with LI played with the device during one 45-minute speech therapy session. Reactions and feedbacks were recorded and rated. The system proved to be easy to understand and learn, as well as engaging and rewarding. The success of the device probably rests on the integration of smart technology and real, tangible objects. The device can be seen as a valuable aid to support and enhance communication abilities in children with LI as well as typically developing individuals.

Eye Tracking Based Control System for Natural Human-Computer Interaction

PubMed Central

Lin, Shu-Fan

2017-01-01

Eye movement can be regarded as a pivotal real-time input medium for human-computer communication, which is especially important for people with physical disability. In order to improve the reliability, mobility, and usability of eye tracking technique in user-computer dialogue, a novel eye control system with integrating both mouse and keyboard functions is proposed in this paper. The proposed system focuses on providing a simple and convenient interactive mode by only using user's eye. The usage flow of the proposed system is designed to perfectly follow human natural habits. Additionally, a magnifier module is proposed to allow the accurate operation. In the experiment, two interactive tasks with different difficulty (searching article and browsing multimedia web) were done to compare the proposed eye control tool with an existing system. The Technology Acceptance Model (TAM) measures are used to evaluate the perceived effectiveness of our system. It is demonstrated that the proposed system is very effective with regard to usability and interface design. PMID:29403528

Eye Tracking Based Control System for Natural Human-Computer Interaction.

PubMed

Zhang, Xuebai; Liu, Xiaolong; Yuan, Shyan-Ming; Lin, Shu-Fan

2017-01-01

Eye movement can be regarded as a pivotal real-time input medium for human-computer communication, which is especially important for people with physical disability. In order to improve the reliability, mobility, and usability of eye tracking technique in user-computer dialogue, a novel eye control system with integrating both mouse and keyboard functions is proposed in this paper. The proposed system focuses on providing a simple and convenient interactive mode by only using user's eye. The usage flow of the proposed system is designed to perfectly follow human natural habits. Additionally, a magnifier module is proposed to allow the accurate operation. In the experiment, two interactive tasks with different difficulty (searching article and browsing multimedia web) were done to compare the proposed eye control tool with an existing system. The Technology Acceptance Model (TAM) measures are used to evaluate the perceived effectiveness of our system. It is demonstrated that the proposed system is very effective with regard to usability and interface design.

Response, Emergency Staging, Communications, Uniform Management, and Evacuation (R.E.S.C.U.M.E.) : report on functional and performance requirements, and high-level data and communication needs.

DOT National Transportation Integrated Search

1995-06-01

INTELLIGENT VEHICLE INITIATIVE OR IVI ABSTRACT THE GOAL OF THE TRAVTEK CAMERA CAR STUDY WAS TO FURNISH A DETAILED EVALUATION OF DRIVING AND NAVIGATION PERFORMANCE, SYSTEM USABILITY, AND SAFETY FOR THE TRAVTEK SYSTEM. TO ACHIEVE THIS GOAL, AN INSTRUME...

Health system guidance appraisal--concept evaluation and usability testing.

PubMed

Ako-Arrey, Denis E; Brouwers, Melissa C; Lavis, John N; Giacomini, Mita K

2016-01-05

Health system guidance (HSG) provides recommendations aimed to address health system challenges. However, there is a paucity of methods to direct, appraise, and report HSG. Earlier research identified 30 candidate criteria (concepts) that can be used to evaluate the quality of HSG and guide development and reporting requirements. The objective of this paper was to describe two studies aimed at evaluating the importance of these 30 criteria, design a draft HSG appraisal tool, and test its usability. This study involved a two-step survey process. In step 1, respondents rated the 30 concepts for appropriateness to, relevance to, and priority for health system decisions and HSG. This led to a draft tool. In step 2, respondents reviewed HSG documents, appraised them using the tool, and answered a series of questions. Descriptive analyses were computed. Fifty participants were invited in step 1, and we had a response rate of 82 %. The mean response rates for each concept within each survey question were universally favorable. There was also an overall agreement about the need for a high-quality tool to systematically direct the development, appraisal, and reporting of HSG. Qualitative feedback and a consensus process by the team led to refinements to some of the concepts and the creation of a beta (draft) version of the HSG tool. In step 2, 35 participants were invited and we had a response rate of 74 %. Exploratory analyses showed that the quality of the HSGs reviewed varied as a function of the HSG item and the specific document assessed. A favorable consensus was reached with participants agreeing that the HSG items were easy to understand and easy to apply. Moreover, the overall agreement was high for the usability of the tool to systematically direct the development (85 %), appraisal (92 %), and reporting (81 %) of HSG. From this process, version 1.0 of the HSG appraisal tool was generated complete with 32 items (and their descriptions) and 4 domains. The final tool, named the Appraisal of Guidelines for Research and Evaluation for Health Systems (AGREE-HS) (version 1), defines expectations of HSG and facilitates informed decisions among policymakers on health system delivery, financial, and governance arrangements.

Usability of patient-centered health IT: mixed-methods usability study of ePill.

PubMed

Schmidt-Kraepelin, Manuel; Dehling, Tobias; Sunyaev, Ali

2014-01-01

To facilitate use of patient-centered health IT applications in everyday life, a high degree of usability is required. Based on the example of a patient-centered web application, we propose a usability study design enabling developers and researchers to assess usability of patient-centered health IT applications and derive implications for their improvement. Our study design integrates tasks that subjects have to process, an associated questionnaire based on Perceived Ease of Use, Perceived Usefulness, Attitude Toward Using, and Behavioral Intention to Use, a System Usability Scale questionnaire, and focus groups. Application of the usability study design demonstrates its feasibility and provides insights for assessment of usability in related projects in research and practice.

Tuberculosis-Diagnostic Expert System: an architecture for translating patients information from the web for use in tuberculosis diagnosis.

PubMed

Osamor, Victor C; Azeta, Ambrose A; Ajulo, Oluseyi O

2014-12-01

Over 1.5-2 million tuberculosis deaths occur annually. Medical professionals are faced with a lot of challenges in delivering good health-care with unassisted automation in hospitals where there are several patients who need the doctor's attention. To automate the pre-laboratory screening process against tuberculosis infection to aid diagnosis and make it fast and accessible to the public via the Internet. The expert system we have built is designed to also take care of people who do not have access to medical experts, but would want to check their medical status. A rule-based approach has been used, and unified modeling language and the client-server architecture technique were applied to model the system and to develop it as a web-based expert system for tuberculosis diagnosis. Algorithmic rules in the Tuberculosis-Diagnosis Expert System necessitate decision coverage where tuberculosis is either suspected or not suspected. The architecture consists of a rule base, knowledge base, and patient database. These units interact with the inference engine, which receives patient' data through the Internet via a user interface. We present the architecture of the Tuberculosis-Diagnosis Expert System and its implementation. We evaluated it for usability to determine the level of effectiveness, efficiency and user satisfaction. The result of the usability evaluation reveals that the system has a usability of 4.08 out of a scale of 5. This is an indication of a more-than-average system performance. Several existing expert systems have been developed for the purpose of supporting different medical diagnoses, but none is designed to translate tuberculosis patients' symptomatic data for online pre-laboratory screening. Our Tuberculosis-Diagnosis Expert System is an effective solution for the implementation of the needed web-based expert system diagnosis. © The Author(s) 2013.

U.S.A.B.I.L.I.T.Y. Framework for Older Adults.

PubMed

Caboral-Stevens, Meriam; Whetsell, Martha V; Evangelista, Lorraine S; Cypress, Brigitte; Nickitas, Donna

2015-01-01

The purpose of the current study was to present a framework to determine potential usability of health websites by older adults. Review of the literature showed paucity of nursing theory related to the use of technology and usability, particularly in older adults. The Roy Adaptation Model, a widely used nursing theory, was chosen to provide framework for the new model. Technology constructs from the Technology Acceptance Model and United Theory of Acceptance and Use of Technology and behavioral control construct from the Theory of Planned Behavior were integrated into the construction of the derived model. The Use of Technology for Adaptation by Older Adults and/or Those With Limited Literacy (U.S.A.B.I.L.I.T.Y.) Model was constructed from the integration of diverse theoretical/conceptual perspectives. The four determinants of usability in the conceptual model include (a) efficiency, (b) learnability, (c) perceived user experience, and (d) perceived control. Because of the lack of well-validated survey questionnaires to measure these determinants, a U.S.A.B.I.L.I.T.Y. Survey was developed. A panel of experts evaluated face and content validity of the new instrument. Internal consistency of the new instrument was 0.96. Usability is key to accepting technology. The derived U.S.A.B.I.L.I.T.Y. framework could serve as a guide for nurses in formative evaluation of technology. Copyright 2015, SLACK Incorporated.

Effect of Improving the Usability of an E-Learning Resource: A Randomized Trial

ERIC Educational Resources Information Center

Davids, Mogamat Razeen; Chikte, Usuf M. E.; Halperin, Mitchell L.

2014-01-01

Optimizing the usability of e-learning materials is necessary to reduce extraneous cognitive load and maximize their potential educational impact. However, this is often neglected, especially when time and other resources are limited. We conducted a randomized trial to investigate whether a usability evaluation of our multimedia e-learning…

Internationalization and localization: evaluating and testing a Website for Asian users.

PubMed

Rau, Pei-Luen Patrick; Liang, Sheau-Farn Max

2003-01-15

The objective of this study was to combine internationalization and localization of Websites and improvement of Website usability with user-centred design methods. This study designed for internationalization and localization of Websites for Asian users, and implemented usability engineering into every phase of Website usability testing, based on the internationalization and localization perspectives of the honeywell.com/your home Website. The first step was to develop the usage scenarios. Three Asian usability specialists carried out one heuristic evaluation session for the current honeywell.com/your home Website. The usability problems were analysed and possible solutions to these problems were discussed. In the next phase, cluster analysis was utilized to test current information architecture. The results provided options for future information architecture development for this Website. Finally, a performance measurement test was conducted to investigate the performance for Asian users. Based on the results, suggestions for improving the Website usability from the localization perspective were provided. The results demonstrate the user-centred design (UCD) approach and stress international and local issues in Website development to Website designers.

PROOF OF CONCEPT FOR A HUMAN RELIABILITY ANALYSIS METHOD FOR HEURISTIC USABILITY EVALUATION OF SOFTWARE

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ronald L. Boring; David I. Gertman; Jeffrey C. Joe

2005-09-01

An ongoing issue within human-computer interaction (HCI) is the need for simplified or “discount” methods. The current economic slowdown has necessitated innovative methods that are results driven and cost effective. The myriad methods of design and usability are currently being cost-justified, and new techniques are actively being explored that meet current budgets and needs. Recent efforts in human reliability analysis (HRA) are highlighted by the ten-year development of the Standardized Plant Analysis Risk HRA (SPAR-H) method. The SPAR-H method has been used primarily for determining humancentered risk at nuclear power plants. The SPAR-H method, however, shares task analysis underpinnings withmore » HCI. Despite this methodological overlap, there is currently no HRA approach deployed in heuristic usability evaluation. This paper presents an extension of the existing SPAR-H method to be used as part of heuristic usability evaluation in HCI.« less

Understanding Usability: Investigating an Integrated Design Environment and Management System

ERIC Educational Resources Information Center

Lee, Jason Chong; Wahid, Shahtab; McCrickard, D. Scott; Chewar, C. M.; Congleton, Ben

2007-01-01

Purpose: Decades of innovation in designing usable (and unusable) interfaces have resulted in a plethora of guidelines, usability methods, and other design tools. The purpose of this research is to develop ways for novice developers to effectively leverage and contribute to the large and growing body of usability knowledge and methods.…

Icon and user interface design for emergency medical information systems: a case study.

PubMed

Salman, Y Batu; Cheng, Hong-In; Patterson, Patrick E

2012-01-01

A usable medical information system should allow for reliable and accurate interaction between users and the system in emergencies. A participatory design approach was used to develop a medical information system in two Turkish hospitals. The process consisted of task and user analysis, an icon design survey, initial icon design, final icon design and evaluation, and installation of the iconic medical information system with the icons. We observed work sites to note working processes and tasks related to the information system and interviewed medical personnel. Emergency personnel then participated in the design process to develop a usable graphical user interface, by drawing icon sketches for 23 selected tasks. Similar sketches were requested for specific tasks such as family medical history, contact information, translation, addiction, required inspections, requests and applications, and nurse observations. The sketches were analyzed and redesigned into computer icons by professional designers and the research team. A second group of physicians and nurses then tested the understandability of the icons. The user interface layout was examined and evaluated by system users, followed by the system's installation. Medical personnel reported the participatory design process was interesting and believed the resulting designs would be more familiar and friendlier. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

A Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User Experience of Connected Health Systems: A Three-Phase Methodology

PubMed Central

Harte, Richard; Glynn, Liam; Rodríguez-Molinero, Alejandro; Baker, Paul MA; Scharf, Thomas; ÓLaighin, Gearóid

2017-01-01

Background Design processes such as human-centered design, which involve the end user throughout the product development and testing process, can be crucial in ensuring that the product meets the needs and capabilities of the user, particularly in terms of safety and user experience. The structured and iterative nature of human-centered design can often present a challenge when design teams are faced with the necessary, rapid, product development life cycles associated with the competitive connected health industry. Objective We wanted to derive a structured methodology that followed the principles of human-centered design that would allow designers and developers to ensure that the needs of the user are taken into account throughout the design process, while maintaining a rapid pace of development. In this paper, we present the methodology and its rationale before outlining how it was applied to assess and enhance the usability, human factors, and user experience of a connected health system known as the Wireless Insole for Independent and Safe Elderly Living (WIISEL) system, a system designed to continuously assess fall risk by measuring gait and balance parameters associated with fall risk. Methods We derived a three-phase methodology. In Phase 1 we emphasized the construction of a use case document. This document can be used to detail the context of use of the system by utilizing storyboarding, paper prototypes, and mock-ups in conjunction with user interviews to gather insightful user feedback on different proposed concepts. In Phase 2 we emphasized the use of expert usability inspections such as heuristic evaluations and cognitive walkthroughs with small multidisciplinary groups to review the prototypes born out of the Phase 1 feedback. Finally, in Phase 3 we emphasized classical user testing with target end users, using various metrics to measure the user experience and improve the final prototypes. Results We report a successful implementation of the methodology for the design and development of a system for detecting and predicting falls in older adults. We describe in detail what testing and evaluation activities we carried out to effectively test the system and overcome usability and human factors problems. Conclusions We feel this methodology can be applied to a wide variety of connected health devices and systems. We consider this a methodology that can be scaled to different-sized projects accordingly. PMID:28302594

Towards a virtual health record for mobile home care of elderly citizens.

PubMed

Koch, Sabine; Hägglund, Maria; Scandurra, Isabella; Moström, Dennis

2004-01-01

Mobile work situations within home care of the elderly require immediate and ubiquitous access to patient-oriented data. The ongoing Swedish research project "Technical support for Mobile CloseCare" focuses on the development and evaluation of work-scenario oriented ICT support for enhanced home care of elderly citizens. The aim of the project is to provide a seamless and consistent information flow between different health care providers and to give intuitive access to information services for the elderly and their relatives. For that purpose, different independent software components are connected through a mobile communication platform. Flexible access to prioritized information for different users in different work situations will be given through a virtual health record. In order to obtain both usable and clinically relevant results, a user centered system development approach is followed. Evaluation of the project results will be based on usability tests and quasi-experimental studies on how system implementation influences quality of care and job- and life satisfaction for care providers, patients and relatives.

Older Adults Co-Creating Meaningful Individualized Social Activities Online for Healthy Ageing.

PubMed

Blusi, Madeleine; Nilsson, Ingeborg; Lindgren, Helena

2018-01-01

Social isolation and loneliness among older people is a growing problem with negative effects on physical and mental health. In co-creation with older adults individualized social activities were designed where older adults through computer mediated communication were able to participate in social activities without leaving their homes. Four types of activities were designed; outdoor activity, music event, visiting a friend and leisure activity. A participatory action research design was applied, where end users together with scientists from two research fields developed, tested and evaluated online participation in the activities. Usability and safety of the systems were major concerns among older adults. The evaluation pointed out that level of simplicity, usability and audio-video quality determined the level of satisfaction with the human interaction during the activity, thereby affecting the meaningfulness of the activity. The research presented in this paper constitutes the first step in a long-term research process aiming at developing a digital coaching system that gives older adults personalized support for increasing participation in meaningful social activities.

The Relationship between National Culture and the Usability of an E-Learning System

ERIC Educational Resources Information Center

Adeoye, Blessing; Wentling, Rose Mary

2007-01-01

The purpose of this study was to investigate possible relationships between national culture and the usability of an e-learning system. The theoretical frameworks that were used to guide this study were Hofstede's (1980) cultural dimensions, and Nielson's (1993) usability attributes. The sample for this study was composed of 24 international…

[Benefits and usability of a pharmaceutical record in medical practice. A survey of hospital doctors and pharmacists (MATRIX study)].

PubMed

Schuers, M; Timsit, M; Gillibert, A; Fred, A; Griffon, N; Bénichou, J; Darmoni, S J; Staccini, P

2016-09-01

To evaluate the impact of the pharmaceutical patient record use in emergency, geriatric and anaesthesia and intensive care departments, an experimentation was launched in 2013 in 55 hospitals. The purpose of the study was to assess the opinions of physicians and pharmacists about the benefits and usability of the patient pharmaceutical record. An e-mailed self-administered questionnaire was sent to all the pharmacists, anaesthesiologists, geriatricians and emergency physicians of the 55 hospitals involved in the patient pharmaceutical record experimentation. The questionnaire assessed the usability of the patient pharmaceutical record using the "System Usability Scale", as well as its use, its benefits and limitations perceived in clinical practice, and overall user satisfaction. Questionnaires were collected from November 2014 to January 2015. Ninety-six questionnaires were collected, from 47 hospitals, representing 86% of the hospitals involved in the experimentation. The patient pharmaceutical record was effectively operational in 36 hospitals. Data from 73 questionnaires filled by physicians and pharmacists with potential experience with the patient pharmaceutical record were used for evaluation. Forty-two respondents were pharmacists (57%) and 31 were physicians (43%), including 13 geriatricians, 11 emergency physicians and 7 anaesthesiologists. Patient pharmaceutical record overall usability score was 62.5 out of 100. It did not vary with the profession or seniority of the respondent. It was positively correlated with the frequency of use. More than half of respondents reported that they never or uncommonly used the patient pharmaceutical record. The length of access to data period was considered as insufficient. Main obstacles to more utilization of the patient pharmaceutical record were the lack of information about the dosage of dispensed drugs, the low number of patients in possession of their health card and the low number of patients with an activated patient pharmaceutical record. Two years after the beginning of the experiment aiming to broaden the access to the patient pharmaceutical record to physicians, these first evaluation results are encouraging. The evaluation of the consequences of the access to the patient pharmaceutical record for physicians remains necessary. Copyright © 2016 Elsevier Masson SAS. All rights reserved.

Effective Usability Engineering in Healthcare: A Vision of Usable and Safer Healthcare IT.

PubMed

Kushniruk, Andre; Senathirajah, Yalini; Borycki, Elizabeth

2017-01-01

Persistent problems with healthcare IT that is unusable and unsafe have been reported worldwide. In this paper we present our vision for deploying usability engineering in healthcare in a more substantive way in order to improve the current situation. The argument will be made that stronger and more substantial efforts need to be made to bring multiple usability engineering methods to bear on points in both system design and deployment (and not just as a one-time effort restricted to software product development). In addition, improved processes for ensuring the usability of commercial vendor-based systems being implemented in healthcare organizations need to be addressed. A discussion will also be provided on challenges and barriers that will need to be overcome to ensure that the heatlhcare IT that is released is both usable and safe.

Computer-Based Tools for Evaluating Graphical User Interfaces

NASA Technical Reports Server (NTRS)

Moore, Loretta A.

1997-01-01

The user interface is the component of a software system that connects two very complex system: humans and computers. Each of these two systems impose certain requirements on the final product. The user is the judge of the usability and utility of the system; the computer software and hardware are the tools with which the interface is constructed. Mistakes are sometimes made in designing and developing user interfaces because the designers and developers have limited knowledge about human performance (e.g., problem solving, decision making, planning, and reasoning). Even those trained in user interface design make mistakes because they are unable to address all of the known requirements and constraints on design. Evaluation of the user inter-face is therefore a critical phase of the user interface development process. Evaluation should not be considered the final phase of design; but it should be part of an iterative design cycle with the output of evaluation being feed back into design. The goal of this research was to develop a set of computer-based tools for objectively evaluating graphical user interfaces. The research was organized into three phases. The first phase resulted in the development of an embedded evaluation tool which evaluates the usability of a graphical user interface based on a user's performance. An expert system to assist in the design and evaluation of user interfaces based upon rules and guidelines was developed during the second phase. During the final phase of the research an automatic layout tool to be used in the initial design of graphical inter- faces was developed. The research was coordinated with NASA Marshall Space Flight Center's Mission Operations Laboratory's efforts in developing onboard payload display specifications for the Space Station.

The use of fault reporting of medical equipment to identify latent design flaws.

PubMed

Flewwelling, C J; Easty, A C; Vicente, K J; Cafazzo, J A

2014-10-01

Poor device design that fails to adequately account for user needs, cognition, and behavior is often responsible for use errors resulting in adverse events. This poor device design is also often latent, and could be responsible for "No Fault Found" (NFF) reporting, in which medical devices sent for repair by clinical users are found to be operating as intended. Unresolved NFF reports may contribute to incident under reporting, clinical user frustration, and biomedical engineering technologist inefficacy. This study uses human factors engineering methods to investigate the relationship between NFF reporting frequency and device usability. An analysis of medical equipment maintenance data was conducted to identify devices with a high NFF reporting frequency. Subsequently, semi-structured interviews and heuristic evaluations were performed in order to identify potential usability issues. Finally, usability testing was conducted in order to validate that latent usability related design faults result in a higher frequency of NFF reporting. The analysis of medical equipment maintenance data identified six devices with a high NFF reporting frequency. Semi-structured interviews, heuristic evaluations and usability testing revealed that usability issues caused a significant portion of the NFF reports. Other factors suspected to contribute to increased NFF reporting include accessory issues, intermittent faults and environmental issues. Usability testing conducted on three of the devices revealed 23 latent usability related design faults. These findings demonstrate that latent usability related design faults manifest themselves as an increase in NFF reporting and that devices containing usability related design faults can be identified through an analysis of medical equipment maintenance data. Copyright © 2014 The Authors. Published by Elsevier Inc. All rights reserved.

Web usability evaluation with screen reader users: implementation of the partial concurrent thinking aloud technique.

PubMed

Federici, Stefano; Stefano, Federici; Borsci, Simone; Stamerra, Gianluca

2010-08-01

A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called partial concurrent thinking aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80%. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.

Multilingual medical dialog system developed as smartphone/tablet application.

PubMed

Hasegawa, Satoshi; Hasegawa, Akira; Takasu, Kento; Kojima, Takehito; Miyao, Masaru; Sugita, Namiho; Sakai, Yuki; Kato, Keisuke

2013-01-01

Along with the concomitant rise in foreign residents in Japan has come the need to improve understanding at several social levels. The need for clear communication is most immediate in the area of the emergency or health care fields. Several types of apps exist that can be used to assist with communication between Japanese medical staff and foreign patients. However, there are problems with ease of use. This study asked 34 subjects to evaluate three types of touch designs with the "ExLanguage Nurse" to improve the usability of such multilingual apps. Results indicate that touch designs on the apps are related to ease of usability.

Design, Implementation, and Wide Pilot Deployment of FitForAll: An Easy to use Exergaming Platform Improving Physical Fitness and Life Quality of Senior Citizens.

PubMed

Konstantinidis, Evdokimos I; Billis, Antonis S; Mouzakidis, Christos A; Zilidou, Vasiliki I; Antoniou, Panagiotis E; Bamidis, Panagiotis D

2016-01-01

Many platforms have emerged as response to the call for technology supporting active and healthy aging. Key requirements for any such e-health systems and any subsequent business exploitation are tailor-made design and proper evaluation. This paper presents the design, implementation, wide deployment, and evaluation of the low cost, physical exercise, and gaming (exergaming) FitForAll (FFA) platform system usability, user adherence to exercise, and efficacy are explored. The design of FFA is tailored to elderly populations, distilling literature guidelines and recommendations. The FFA architecture introduces standard physical exercise protocols in exergaming software engineering, as well as, standard physical assessment tests for augmented adaptability through adjustable exercise intensity. This opens up the way to next generation exergaming software, which may be more automatically/smartly adaptive. 116 elderly users piloted FFA five times/week, during an eight-week controlled intervention. Usability evaluation was formally conducted (SUS, SUMI questionnaires). Control group consisted of a size-matched elderly group following cognitive training. Efficacy was assessed objectively through the senior fitness (Fullerton) test, and subjectively, through WHOQoL-BREF comparisons of pre-postintervention between groups. Adherence to schedule was measured by attendance logs. The global SUMI score was 68.33±5.85%, while SUS was 77.7. Good usability perception is reflected in relatively high adherence of 82% for a daily two months pilot schedule. Compared to control group, elderly using FFA improved significantly strength, flexibility, endurance, and balance while presenting a significant trend in quality of life improvements. This is the first elderly focused exergaming platform intensively evaluated with more than 100 participants. The use of formal tools makes the findings comparable to other studies and forms an elderly exergaming corpus.

Hello World! - Experiencing Usability Methods without Usability Expertise

NASA Astrophysics Data System (ADS)

Eriksson, Elina; Cajander, Åsa; Gulliksen, Jan

How do you do usability work when no usability expertise is available? What happens in an organization when system developers, with no previous HCI knowledge, after a 3-day course, start applying usability methods, and particularly field studies? In order to answer these questions qualitative data were gathered through participatory observations, a feed back survey, field study documentation and interviews from 47 system developers from a public authority. Our results suggest that field studies enhance the developer’s understanding of the user perspective, and provide a more holistic overview of the use situation, but that some developers were unable to interpret their observations and see solutions to the users’ problems. The field study method was very much appreciated and has now become standard operating procedure within the organization. However, although field studies may be useful, it does not replace the need for usability pro fes sion als, as their knowledge is essential for more complex observations, analysis and for keeping the focus on usability.

Pilot study on the effects of a computer-based medical image system.

PubMed Central

Wu, S. C.; Smith, J. W.; Swan, J. E.

1996-01-01

Current medical imaging systems are developed for the purpose of data management. Evaluations of these systems are usually done by assessing users' subjective appreciation rather than objectively gauging performance influence. The present report discusses the evaluation of a medical image presentation system prototype utilizing a cognitive approach. Experimental results showed hypothesized performance improvement attributed to advanced presentation techniques. However, this improvement was almost inadvertently masked by users' previous strategies and interactions with new technology. Overall these data demonstrate the potential benefit of implementing such a system in actual practice as well as provide an example of applying the cognitive approach in evaluating the usability of medical systems. Images Figure 1 PMID:8947750

Usability testing of a prototype multi-user telehealth kiosk.

PubMed

Courtney, Karen L; Matthews, Judith T; McMillan, Julie M; Person Mecca, Laurel; Smailagic, Asim; Siewiorek, Daniel

2015-01-01

The overall purpose of this study was to learn how community-dwelling older adults would interact with our prototype multi-user telehealth kiosk and their views about its usability. Seven subjects participated in laboratory-based usability sessions to evaluate the physical design, appearance, functionality and perceived ease of use of a multi-user telehealth kiosk prototype. During usability testing participants recommended 18 new features (29% of comments), identified 15 software errors (23% of comments) and 29 user interface errors (47% of comments).

LATCH usability in vehicles.

DOT National Transportation Integrated Search

2012-04-01

This project investigated the usability of Lower Anchors and Tethers for CHildren (LATCH) hardware by measuring LATCH implementations in 98 2011 or 2010 model-year vehicles. ISO and SAE LATCH usability rating systems were used to assess all vehicles ...

Improving the Quality and Effectiveness of Computer-Mediated Instruction Through Usability Evaluations

ERIC Educational Resources Information Center

Crowther, Michael S.; Keller, Chris C.; Waddoups, Gregory L.

2004-01-01

Brigham Young University's Center for Instructional Design (CID) creates online courses and multimedia instructional applications for university faculty. This paper asserts that including usability testing as a part of evaluation improves the quality and effectiveness of computer-mediated instruction. The paper describes the fundamental purpose…

Evaluating the usability of 19 effluents for heterotrophic cultivation of microalgal consortia as biodiesel feedstock

USDA-ARS?s Scientific Manuscript database

A key reason inhibiting commercialization of algal oil as biodiesel feedstock is cultivation cost. For this reason, the usability of 19 readily available industrial effluents (autoclaved and non-autoclaved) to support heterotrophic growth and lipid accumulation was evaluated using six mixed algal cu...

User-Centered Design and Usability Testing of a Web Site: An Illustrative Case Study.

ERIC Educational Resources Information Center

Corry, Michael D.; Frick, Theodore W.; Hansen, Lisa

1997-01-01

Presents an overview of user-centered design and usability testing. Describes a Web site evaluation project at a university, the iterative process of rapid prototyping and usability testing, and how the findings helped to improve the design. Discusses recommendations for university Web site design and reflects on problems faced in usability…

Development and evaluation of a multimedia e-learning resource for electrolyte and acid-base disorders.

PubMed

Davids, Mogamat Razeen; Chikte, Usuf M E; Halperin, Mitchell L

2011-09-01

This article reports on the development and evaluation of a Web-based application that provides instruction and hands-on practice in managing electrolyte and acid-base disorders. Our teaching approach, which focuses on concepts rather than details, encourages quantitative analysis and a logical problem-solving approach. Identifying any dangers to the patient is a vital first step. Concepts such as an "appropriate response" to a given perturbation and the need for electroneutrality in body fluids are used repeatedly. Our Electrolyte Workshop was developed using Flash and followed an iterative design process. Two case-based tutorials were built in this first phase, with one tutorial including an interactive treatment simulation. Users select from a menu of therapies and see the impact of their choices on the patient. Appropriate text messages are displayed, and changes in body compartment sizes, brain size, and plasma sodium concentrations are illustrated via Flash animation. Challenges encountered included a shortage of skilled Flash developers, budgetary constraints, and challenges in communication between the authors and the developers. The application was evaluated via user testing by residents and specialists in internal medicine. Satisfaction was measured with a questionnaire based on the System Usability Scale. The mean System Usability Scale score was 78.4 ± 13.8, indicating a good level of usability. Participants rated the content as being scientifically sound; they liked the teaching approach and felt that concepts were conveyed clearly. They indicated that the application held their interest, that it increased their understanding of hyponatremia, and that they would recommend this learning resource to others.

Usability Testing of a Complex Clinical Decision Support Tool in the Emergency Department: Lessons Learned.

PubMed

Press, Anne; McCullagh, Lauren; Khan, Sundas; Schachter, Andy; Pardo, Salvatore; McGinn, Thomas

2015-09-10

As the electronic health record (EHR) becomes the preferred documentation tool across medical practices, health care organizations are pushing for clinical decision support systems (CDSS) to help bring clinical decision support (CDS) tools to the forefront of patient-physician interactions. A CDSS is integrated into the EHR and allows physicians to easily utilize CDS tools. However, often CDSS are integrated into the EHR without an initial phase of usability testing, resulting in poor adoption rates. Usability testing is important because it evaluates a CDSS by testing it on actual users. This paper outlines the usability phase of a study, which will test the impact of integration of the Wells CDSS for pulmonary embolism (PE) diagnosis into a large urban emergency department, where workflow is often chaotic and high stakes decisions are frequently made. We hypothesize that conducting usability testing prior to integration of the Wells score into an emergency room EHR will result in increased adoption rates by physicians. The objective of the study was to conduct usability testing for the integration of the Wells clinical prediction rule into a tertiary care center's emergency department EHR. We conducted usability testing of a CDS tool in the emergency department EHR. The CDS tool consisted of the Wells rule for PE in the form of a calculator and was triggered off computed tomography (CT) orders or patients' chief complaint. The study was conducted at a tertiary hospital in Queens, New York. There were seven residents that were recruited and participated in two phases of usability testing. The usability testing employed a "think aloud" method and "near-live" clinical simulation, where care providers interacted with standardized patients enacting a clinical scenario. Both phases were audiotaped, video-taped, and had screen-capture software activated for onscreen recordings. Phase I: Data from the "think-aloud" phase of the study showed an overall positive outlook on the Wells tool in assessing a patient for a PE diagnosis. Subjects described the tool as "well-organized" and "better than clinical judgment". Changes were made to improve tool placement into the EHR to make it optimal for decision-making, auto-populating boxes, and minimizing click fatigue. Phase II: After incorporating the changes noted in Phase 1, the participants noted tool improvements. There was less toggling between screens, they had all the clinical information required to complete the tool, and were able to complete the patient visit efficiently. However, an optimal location for triggering the tool remained controversial. This study successfully combined "think-aloud" protocol analysis with "near-live" clinical simulations in a usability evaluation of a CDS tool that will be implemented into the emergency room environment. Both methods proved useful in the assessment of the CDS tool and allowed us to refine tool usability and workflow.

Evaluating the Functionality and Usability of Two Novel Wheelchair Anti-Rollback Devices for Ramp Ascent in Manual Wheelchair Users With Spinal Cord Injury.

PubMed

Deems-Dluhy, Susan L; Jayaraman, Chandrasekaran; Green, Steve; Albert, Mark V; Jayaraman, Arun

2017-05-01

Difficulty ascending ramps and inclines with a manual wheelchair adversely affects the everyday mobility and overall quality of life of manual wheelchair users. Currently, various anti-rollback devices are available to assist manual wheelchair users to ascend ramps and inclines. However, these devices have 2 main shortcomings: restriction to backward motion limiting recovery from an overturning wheelchair, which is a safety concern; and difficulty in engaging/disengaging the device while on the ramp. To evaluate the functionality and usability of 2 novel wheelchair anti-rollback devices developed to address these shortcomings (prototypes "Wheel" and "Brake"). Cross-sectional. Rehabilitation research facility. Twelve adult participants with chronic spinal cord injury. Participants completed training and tested with both the wheelchair anti-rollback devices on a 7.3-m-long ramp. Number of stops, perceived physical exertion, pain, and ease of use of these devices as participants maneuvered their wheelchairs up a 7.3-m ramp were assessed. Participants also evaluated their satisfaction with the usability of both the devices using the Quebec User Evaluation of Satisfaction With Assistive Technology (QUEST 2.0). Both prototypes evaluated overcame the limitations of the existing anti-rollback devices. Nonparametric statistical tests showed that participants rated both prototypes similarly for the overall functional and usability aspects. However, the participants' satisfactory rating were higher for the prototype "Brake" than for the prototype "Wheel" based on a functional aspect (ie, engaging/disengaging easiness), and higher for Wheel than for Brake, based on a usability aspect (prototype size). The qualitative and quantitative outcomes of this investigation, based on the usability and functional evaluations, provided useful information for the improvement in the design of both anti-rollback devices, which may allow manual wheelchair users to manage ramp ascent more safely and easily. Further evaluations with a different SCI population is recommended. IV. Copyright © 2017 American Academy of Physical Medicine and Rehabilitation. Published by Elsevier Inc. All rights reserved.

Accessibility and usability of parks and playgrounds.

PubMed

Perry, Meredith A; Devan, Hemakumar; Fitzgerald, Harry; Han, Karen; Liu, Li-Ting; Rouse, Jack

2018-04-01

Public parks and playgrounds are an environment for leisure activity, which all generations can enjoy at low or no financial cost. Evaluating the accessibility and usability of parks and playgrounds is crucial because their design, environment (natural and built) and safety could restrict participation of persons with disabilities. To evaluate the accessibility and usability of 21 public parks and playgrounds in three metropolitan cities of New Zealand. Secondary aims were to compare the accessibility and usability by park type (destination or neighborhood) and deprivation level (high and low). Twenty-one parks were evaluated. A stratified random sampling was used to select 18 parks (six from each city). Three additional parks were purposely selected (one from each city) at the request of each respective city council. The parks and playgrounds were evaluated using a customized tool. Data were analyzed using descriptive statistics. None of the parks we evaluated met the national standards and/or international guidelines for park and playground design. We identified potential accessibility and usability issues with car parking spaces, path surfaces and play equipment as well as lack of lighting and fencing. The presence of amenities (e.g. toilets and drinking fountains) was more common in destination parks. Fewer parks in areas of higher deprivation had accessible car parking spaces and main paths wider than 1.5 m. Our evaluation identified potential design, environmental and safety barriers to park and playground based participation for persons with disabilities across the lifespan. A larger, more comprehensive evaluation of parks and playgrounds is required. Copyright © 2017 Elsevier Inc. All rights reserved.

Designing for Wide-Area Situation Awareness in Future Power Grid Operations

NASA Astrophysics Data System (ADS)

Tran, Fiona F.

Power grid operation uncertainty and complexity continue to increase with the rise of electricity market deregulation, renewable generation, and interconnectedness between multiple jurisdictions. Human operators need appropriate wide-area visualizations to help them monitor system status to ensure reliable operation of the interconnected power grid. We observed transmission operations at a control centre, conducted critical incident interviews, and led focus group sessions with operators. The results informed a Work Domain Analysis of power grid operations, which in turn informed an Ecological Interface Design concept for wide-area monitoring. I validated design concepts through tabletop discussions and a usability evaluation with operators, earning a mean System Usability Scale score of 77 out of 90. The design concepts aim to support an operator's complete and accurate understanding of the power grid state, which operators increasingly require due to the critical nature of power grid infrastructure and growing sources of system uncertainty.

A usability study of CPOE's medication administration functions: impact on physician-nurse cooperation.

PubMed

Beuscart-Zéphir, Marie Catherine; Pelayo, Sylvia; Degoulet, Patrice; Anceaux, Françoise; Guerlinger, Sandra; Meaux, Jean-Jacques

2004-01-01

Implementation of CPOE systems in Healthcare Institutions has proven efficient in reducing medication errors but it also induces hidden side-effects on Doctor-Nurse cooperation. We propose a usability engineering approach to this problem. An extensive activity analysis of the medication ordering and administration process was performed in several departments of 3 different hospitals. Two of these hospitals are still using paper-based orders, while the 3rd one is in the roll-out phase of medication functions of its CPOE system. We performed a usability assessment of this CPOE system. The usability assessment uncovered usability problems for the entry of medication administration time scheduling by the physician and revealed that the information can be ambiguous for the nurse. The comparison of cooperation models in both situation shows that users tend to adopt a distributed decision making paradigm in the paper-based situation, while the CPOE system supports a centralized decision making process. This analysis can support recommendation for the re-engineering of the Human-Computer Interface.

Usability and Acceptability of a Web-Based Program for Chemotherapy-Induced Peripheral Neuropathy.

PubMed

Tofthagen, Cindy; Kip, Kevin E; Passmore, Denise; Loy, Ian; Berry, Donna L

2016-07-01

Chemotherapy-induced neuropathy is a painful and debilitating adverse effect of certain chemotherapy drugs. There have not been any patient-centered, easily accessible Web-based interventions to assist with self-management of chemotherapy-induced neuropathy. The aims of this study were to evaluate usability and acceptability and to estimate an effect size of a Web-based intervention for assessing and managing chemotherapy-induced neuropathy. Participants (N = 14) were instructed to complete the Creativity, Optimism, Planning, and Expert Information for Chemotherapy-Induced Peripheral Neuropathy program and provide verbal responses to the program. Participants completed the Chemotherapy Induced Peripheral Neuropathy Assessment Tool and Post-Study System Usability Questionnaire. Iterative changes were made to the COPE-CIPN. Participants were asked to provide feedback on the revised COPE-CIPN, repeat the Chemotherapy Induced Peripheral Neuropathy Assessment Tool, and evaluate acceptability using the Acceptability e-Scale. The COPE-CIPN demonstrated high usability (mean, 1.98 [SD, 1.12]) and acceptability (mean, 4.40 [SD, 0.52]). Comments indicated that the interface was easy to use, and the information was helpful. While neuropathy symptoms continued to increase in this group of patients receiving neurotoxic chemotherapy, there was a decrease in mean level of interference with activities from 53.71 to 39.29 over 3 to 4 months, which indicated a moderate effect (d = 0.39) size. The COPE-CIPN may be a useful intervention to support self-management of chemotherapy-induced neuropathy.

Integration of MSFC Usability Lab with Usability Testing

NASA Technical Reports Server (NTRS)

Cheng, Yiwei; Richardson, Sally

2010-01-01

As part of the Stage Analysis Branch, human factors engineering plays an important role in relating humans to the systems of hardware and structure designs of the new launch vehicle. While many branches are involved in the technical aspects of creating a launch vehicle, human factors connects humans to the scientific systems with the goal of improving operational performance and safety while reducing operational error and damage to the hardware. Human factors engineers use physical and computerized models to visualize possible areas for improvements to ensure human accessibility to components requiring maintenance and that the necessary maintenance activities can be accomplished with minimal risks to human and hardware. Many methods of testing are used to fulfill this goal, such as physical mockups, computerized visualization, and usability testing. In this analysis, a usability test is conducted to test how usable a website is to users who are and are not familiar with it. The testing is performed using participants and Morae software to record and analyze the results. This analysis will be a preliminary test of the usability lab in preparation for use in new spacecraft programs, NASA Enterprise, or other NASA websites. The usability lab project is divided into two parts: integration of the usability lab and a preliminary test of the usability lab.

Usability of a mobile electronic medical record prototype: a verbal protocol analysis.

PubMed

Wu, Robert C; Orr, M Scott; Chignell, Mark; Straus, Sharon E

2008-06-01

Point of care access to electronic medical records may provide clinicians with the information they want when they need it and may in turn improve patient safety. Yet providing an electronic medical record on handheld devices presents many usability challenges, and it is unclear whether clinicians will use them. An iterative design process for the development and evaluation of a prototype of a mobile electronic medical record was performed. Usability sessions were conducted in which physicians were asked to 'think aloud' while working through clinical scenarios using the prototype. Verbal protocol analysis, which consists of coding utterances, was conducted on the transcripts from the sessions and common themes were extracted. Usability sessions were held with five family physicians and four internists with varying levels of computer expertise. Physicians were able to use the device to complete 52 of 54 required tasks. Users commented that it was intuitive (9/9), would increase accessibility (5/9) but for them to use it, it would need the system to be fast and time-saving (5/9). Users had difficulty entering information (5/9) and reading the screen (4/9). In terms of functionality, users had concerns about completeness of information (6/9), details of ordering (5/9) and desired billing functionality (5/9) and integration with other systems (4/9). While physicians can use mobile electronic medical records in realistic scenarios, certain requirements likely need to be met including a fast system with easy data selection, easy data entry and improved display before widespread adoption occurs.

Usability Laboratory Test of a Novel Mobile Homecare Application with Experienced Home Help Service Staff

PubMed Central

Scandurra, I; Hägglund, M; Koch, S; Lind, M

2008-01-01

Using participatory design, we developed and deployed a mobile Virtual Health Record (VHR) on a personal digital assistant (PDA) together with experienced homecare staff. To assess transferability to a second setting and usability when used by novice users with limited system education the application was tested in a usability lab. Eight participants from another homecare district performed tasks related to daily homecare work using the VHR. Test protocols were analyzed with regard to effectiveness, potential usability problems and user satisfaction. Usability problems having impact on system performance and contextual factors affecting system transferability were uncovered. Questionnaires revealed that the participants frequently used computers, but never PDAs. Surprisingly there were only minor differences in input efficiency between novice and experienced users. The participants were overall satisfied with the application. However, transfer to another district can not be performed, unless by means of careful field observations of contextual differences. PMID:19415140

Usability testing for the rest of us: the application of discount usability principles in the development of an online communications assessment application.

PubMed

Brock, Douglas; Kim, Sara; Palmer, Odawni; Gallagher, Thomas; Holmboe, Eric

2013-01-01

Usability evaluation provides developers and educators with the means to understand user needs, improve overall product utility, and increase user satisfaction. The application of "discount usability" principles developed to make usability testing more practical and useful may improve user experience at minimal cost and require little existing expertise to conduct. We describe an application of discount usability to a high-fidelity online communications assessment application developed by the University of Washington for the American Board of Internal Medicine. Eight internal medicine physicians completed a discount usability test. Sessions were recorded and the videos analyzed for significant usability concerns. Concerns were identified, summarized, discussed, and prioritized by the authors in collaboration with the software developers before implementing any changes to the interface. Thirty-eight significant usability issues were detected and four technical problems were identified. Each issue was responded to through modification of the software, by providing additional instruction, or delayed for a later version to be developed. Discount usability can be easily implemented in academic developmental activities. Our study resulted in the discovery and remediation of significant user problems, in addition to giving important insight into the novel methods built into the application.

36 CFR 1193.23 - Product design, development, and evaluation.

Code of Federal Regulations, 2014 CFR

2014-07-01

..., usability, and compatibility of telecommunications equipment and customer premises equipment and shall... as possible. Manufacturers shall identify barriers to accessibility and usability as part of such a...

Usability evaluation of mobile applications; where do we stand?

NASA Astrophysics Data System (ADS)

Zahra, Fatima; Hussain, Azham; Mohd, Haslina

2017-10-01

The range and availability of mobile applications is expanding rapidly. With the increased processing power available on portable devices, developers are increasing the range of services by embracing smartphones in their extensive and diverse practices. While usability testing and evaluations of mobile applications have not yet touched the accuracy level of other web based applications. The existing usability models do not adequately capture the complexities of interacting with applications on a mobile platform. Therefore, this study aims to presents review on existing usability models for mobile applications. These models are in their infancy but with time and more research they may eventually be adopted. Moreover, different categories of mobile apps (medical, entertainment, education) possess different functional and non-functional requirements thus customized models are required for diverse mobile applications.

Usability study of a novel, self-lighted, disposable speculum: military applications.

PubMed

Jones, Christina L; Gruber, Daniel D; Warner, William; Buller, Jerome L

2013-04-01

Data collected from a postutilization questionnaire were used to evaluate the usability of the OfficeSPEC disposable vaginal speculum, specifically the effectiveness, efficiency, and acceptability, in clinical, hospital, and austere environments. Usability data analysis showed the OfficeSPEC speculum had an effectiveness rating of 4.6/5, efficiency rating of 4.5/5, and acceptability rating of 4.6/5; overall usability in deployed environments was favorable. The overall rankings were 3.4 for plastic, 4.2 for metal (p < 0.001), and 4.5 for OfficeSPEC (p < 0.001). Cost analysis of the OfficeSPEC placed the disposable speculum as a reasonable alternative with yearly cost of $129,200, compared to traditional metal ($209,100) and plastic ($319,175). By evaluating the OfficeSPEC speculum within a usability framework, it proved to be practical, viable alternative in all environments, particularly in the forward deployed environment. Reprint & Copyright © 2013 Association of Military Surgeons of the U.S.

Adapting an in-person patient-caregiver communication intervention to a tailored web-based format.

PubMed

Zulman, Donna M; Schafenacker, Ann; Barr, Kathryn L C; Moore, Ian T; Fisher, Jake; McCurdy, Kathryn; Derry, Holly A; Saunders, Edward W; An, Lawrence C; Northouse, Laurel

2012-03-01

Interventions that target cancer patients and their caregivers have been shown to improve patient-caregiver communication, support, and emotional well-being. To adapt an in-person communication intervention for cancer patients and caregivers to a web-based format, and to examine the usability and acceptability of the web-based program among representative users. A tailored, interactive web-based communication program for cancer patients and their family caregivers was developed based on an existing in-person, nurse-delivered intervention. The development process involved: (1) building a multidisciplinary team of content and web design experts, (2) combining key components of the in-person intervention with the unique tailoring and interactive features of a web-based platform, and (3) conducting focus groups and usability testing to obtain feedback from representative program users at multiple time points. Four focus groups with 2-3 patient-caregiver pairs per group (n = 22 total participants) and two iterations of usability testing with four patient-caregiver pairs per session (n = 16 total participants) were conducted. Response to the program's structure, design, and content was favorable, even among users who were older or had limited computer and Internet experience. The program received high ratings for ease of use and overall usability (mean System Usability Score of 89.5 out of 100). Many elements of a nurse-delivered patient-caregiver intervention can be successfully adapted to a web-based format. A multidisciplinary design team and an iterative evaluation process with representative users were instrumental in the development of a usable and well-received web-based program. Copyright © 2011 John Wiley & Sons, Ltd.

Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation.

PubMed

Zahabi, Maryam; Kaber, David B; Swangnetr, Manida

2015-08-01

The objectives of this study were to (a) review electronic medical record (EMR) and related electronic health record (EHR) interface usability issues, (b) review how EMRs have been evaluated with safety analysis techniques along with any hazard recognition, and (c) formulate design guidelines and a concept for enhanced EMR interfaces with a focus on diagnosis and documentation processes. A major impact of information technology in health care has been the introduction of EMRs. Although numerous studies indicate use of EMRs to increase health care quality, there remain concerns with usability issues and safety. A literature search was conducted using Compendex, PubMed, CINAHL, and Web of Science databases to find EMR research published since 2000. Inclusion criteria included relevant English-language papers with subsets of keywords and any studies (manually) identified with a focus on EMR usability. Fifty studies met the inclusion criteria. Results revealed EMR and EHR usability problems to include violations of natural dialog, control consistency, effective use of language, effective information presentation, and customization principles as well as a lack of error prevention, minimization of cognitive load, and feedback. Studies focusing on EMR system safety made no objective assessments and applied only inductive reasoning methods for hazard recognition. On the basis of the identified usability problems and structure of safety analysis techniques, we provide EMR design guidelines and a design concept focused on the diagnosis process and documentation. The design guidelines and new interface concept can be used for prototyping and testing enhanced EMRs. © 2015, Human Factors and Ergonomics Society.

Brief Report: Adapting an In-Person Patient-Caregiver Communication Intervention to a Tailored Web-Based Format

PubMed Central

Zulman, Donna M.; Schafenacker, Ann; Barr, Kathryn L.C.; Moore, Ian T.; Fisher, Jake; McCurdy, Kathryn; Derry, Holly A.; Saunders, Edward W.; An, Lawrence C.; Northouse, Laurel

2011-01-01

Background Interventions that target cancer patients and their caregivers have been shown to improve communication, support, and emotional well-being. Objective To adapt an in-person communication intervention for cancer patients and caregivers to a web-based format, and to examine the usability and acceptability of the web-based program among representative users. Methods A tailored, interactive web-based communication program for cancer patients and their family caregivers was developed based on an existing in-person, nurse-delivered intervention. The development process involved: 1) building a multidisciplinary team of content and web design experts, 2) combining key components of the in-person intervention with the unique tailoring and interactive features of a web-based platform, and 3) conducting focus groups and usability testing to obtain feedback from representative program users at multiple time points. Results Four focus groups with 2 to 3 patient-caregiver pairs per group (n = 22 total participants) and two iterations of usability testing with 4 patient-caregiver pairs per session (n = 16 total participants) were conducted. Response to the program's structure, design, and content was favorable, even among users who were older or had limited computer and internet experience. The program received high ratings for ease of use and overall usability (mean System Usability Score of 89.5 out of 100). Conclusions Many elements of a nurse-delivered patient-caregiver intervention can be successfully adapted to a web-based format. A multidisciplinary design team and an iterative evaluation process with representative users were instrumental in the development of a usable and well-received web-based program. PMID:21830255

36 CFR § 1193.23 - Product design, development, and evaluation.

Code of Federal Regulations, 2013 CFR

2013-07-01

..., usability, and compatibility of telecommunications equipment and customer premises equipment and shall... as possible. Manufacturers shall identify barriers to accessibility and usability as part of such a...

Next Gen One Portal Usability Evaluation

NASA Technical Reports Server (NTRS)

Cross, E. V., III; Perera, J. S.; Hanson, A. M.; English, K.; Vu, L.; Amonette, W.

2018-01-01

Each exercise device on the International Space Station (ISS) has a unique, customized software system interface with unique layouts / hierarchy, and operational principles that require significant crew training. Furthermore, the software programs are not adaptable and provide no real-time feedback or motivation to enhance the exercise experience and/or prevent injuries. Additionally, the graphical user interfaces (GUI) of these systems present information through multiple layers resulting in difficulty navigating to the desired screens and functions. These limitations of current exercise device GUI's lead to increased crew time spent on initiating, loading, performing exercises, logging data and exiting the system. To address these limitations a Next Generation One Portal (NextGen One Portal) Crew Countermeasure System (CMS) was developed, which utilizes the latest industry guidelines in GUI designs to provide an intuitive ease of use approach (i.e., 80% of the functionality gained within 5-10 minutes of initial use without/limited formal training required). This is accomplished by providing a consistent interface using common software to reduce crew training, increase efficiency & user satisfaction while also reducing development & maintenance costs. Results from the usability evaluations showed the NextGen One Portal UI having greater efficiency, learnability, memorability, usability and overall user experience than the current Advanced Resistive Exercise Device (ARED) UI used by astronauts on ISS. Specifically, the design of the One-Portal UI as an app interface similar to those found on the Apple and Google's App Store, assisted many of the participants in grasping the concepts of the interface with minimum training. Although the NextGen One-Portal UI was shown to be an overall better interface, observations by the test facilitators noted specific exercise tasks appeared to have a significant impact on the NextGen One-Portal UI efficiency. Future updates to the NextGen One Portal UI will address these inefficiencies.

Product design enhancement using apparent usability and affective quality.

PubMed

Seva, Rosemary R; Gosiaco, Katherine Grace T; Santos, Ma Crea Eurice D; Pangilinan, Denise Mae L

2011-03-01

In this study, apparent usability and affective quality were integrated in a design framework called the Usability Perception and Emotion Enhancement Model (UPEEM). The UPEEM was validated using structural equation modeling (SEM). The methodology consists of four phases namely product selection, attribute identification, design alternative generation, and design alternative evaluation. The first stage involved the selection of a product that highly involves the consumer. In the attribute identification stage, design elements of the product were identified. The possible values of these elements were also determined for use in the experimentation process. Design of experiments was used to identify how the attributes will be varied in the design alternative stage and which of the attributes significantly contribute to affective quality, apparent usability, and desirability in the design evaluation stage. Results suggest that product attributes related to form are relevant in eliciting intense affect and perception of usability in mobile phones especially those directly related to functionality and aesthetics. This study considered only four product attributes among so many due to the constraints of the research design employed. Attributes related to aesthetic perception of a product enhance apparent usability such as those related to dimensional ratios. Copyright © 2010 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Is talking to an automated teller machine natural and fun?

PubMed

Chan, F Y; Khalid, H M

Usability and affective issues of using automatic speech recognition technology to interact with an automated teller machine (ATM) are investigated in two experiments. The first uncovered dialogue patterns of ATM users for the purpose of designing the user interface for a simulated speech ATM system. Applying the Wizard-of-Oz methodology, multiple mapping and word spotting techniques, the speech driven ATM accommodates bilingual users of Bahasa Melayu and English. The second experiment evaluates the usability of a hybrid speech ATM, comparing it with a simulated manual ATM. The aim is to investigate how natural and fun can talking to a speech ATM be for these first-time users. Subjects performed the withdrawal and balance enquiry tasks. The ANOVA was performed on the usability and affective data. The results showed significant differences between systems in the ability to complete the tasks as well as in transaction errors. Performance was measured on the time taken by subjects to complete the task and the number of speech recognition errors that occurred. On the basis of user emotions, it can be said that the hybrid speech system enabled pleasurable interaction. Despite the limitations of speech recognition technology, users are set to talk to the ATM when it becomes available for public use.

Toward a More Usable Home-Based Video Telemedicine System: A Heuristic Evaluation of the Clinician User Interfaces of Home-Based Video Telemedicine Systems

PubMed Central

Welch, Brandon; Brinda, FNU

2017-01-01

Background Telemedicine is the use of technology to provide and support health care when distance separates the clinical service and the patient. Home-based telemedicine systems involve the use of such technology for medical support and care connecting the patient from the comfort of their homes with the clinician. In order for such a system to be used extensively, it is necessary to understand not only the issues faced by the patients in using them but also the clinician. Objectives The aim of this study was to conduct a heuristic evaluation of 4 telemedicine software platforms—Doxy.me, Polycom, Vidyo, and VSee—to assess possible problems and limitations that could affect the usability of the system from the clinician’s perspective. Methods It was found that 5 experts individually evaluated all four systems using Nielsen’s list of heuristics, classifying the issues based on a severity rating scale. Results A total of 46 unique problems were identified by the experts. The heuristics most frequently violated were visibility of system status and Error prevention amounting to 24% (11/46 issues) each. Esthetic and minimalist design was second contributing to 13% (6/46 issues) of the total errors. Conclusions Heuristic evaluation coupled with a severity rating scale was found to be an effective method for identifying problems with the systems. Prioritization of these problems based on the rating provides a good starting point for resolving the issues affecting these platforms. There is a need for better transparency and a more streamlined approach for how physicians use telemedicine systems. Visibility of the system status and speaking the users’ language are keys for achieving this. PMID:28438724

Genomic imputation and evaluation using high density Holstein genotypes

USDA-ARS?s Scientific Manuscript database

Genomic evaluations for 161,341 Holsteins were computed using 311,725 of the 777,962 markers on the Illumina high-density (HD) chip. Initial edits with 1,741 HD genotypes from 5 breeds revealed that 636,967 markers were usable but that half were redundant. Usable Holstein genotypes included 1,510 an...

Usability Level of Distance Education Website (Sakarya University Sample)

ERIC Educational Resources Information Center

Isman, Aytekin; Isbulan, Onur

2010-01-01

The main goal of this research is to determine the usability of Sakarya University Adapazari Vocational Two-Year Community College's Web Site. A scale was developed to the evaluation and applied to 1512 students. From the applied questionnaire only appropriate ones were selected to be evaluated and 1229 questionnaires were analysed. Finally,…

Development of Usability Criteria for E-Learning Content Development Software

ERIC Educational Resources Information Center

Celik, Serkan

2012-01-01

Revolutionary advancements have been observed in e-learning technologies though an amalgamated evaluation methodology for new generation e-learning content development tools is not available. The evaluation of educational software for online use must consider its usability and as well as its pedagogic effectiveness. This study is a first step…

Establishing usability heuristics for heuristics evaluation in a specific domain: Is there a consensus?

PubMed

Hermawati, Setia; Lawson, Glyn

2016-09-01

Heuristics evaluation is frequently employed to evaluate usability. While general heuristics are suitable to evaluate most user interfaces, there is still a need to establish heuristics for specific domains to ensure that their specific usability issues are identified. This paper presents a comprehensive review of 70 studies related to usability heuristics for specific domains. The aim of this paper is to review the processes that were applied to establish heuristics in specific domains and identify gaps in order to provide recommendations for future research and area of improvements. The most urgent issue found is the deficiency of validation effort following heuristics proposition and the lack of robustness and rigour of validation method adopted. Whether domain specific heuristics perform better or worse than general ones is inconclusive due to lack of validation quality and clarity on how to assess the effectiveness of heuristics for specific domains. The lack of validation quality also affects effort in improving existing heuristics for specific domain as their weaknesses are not addressed. Copyright © 2016 Elsevier Ltd. All rights reserved.

75 FR 14575 - Voting Equipment Evaluations Phase III

Federal Register 2010, 2011, 2012, 2013, 2014

2010-03-26

..., Human Performance-Based Standards and Usability Testing. NIST Phase III research is designed to: (1... vendor equipment will not be released. Comparative information may be released in a blind manner... electronic poll book systems as well as software used for ballot design and creation. Dated: March 23, 2010...

Can Visualizing Document Space Improve Users' Information Foraging?

ERIC Educational Resources Information Center

Song, Min

1998-01-01

This study shows how users access relevant information in a visualized document space and determine whether BiblioMapper, a visualization tool, strengthens an information retrieval (IR) system and makes it more usable. BiblioMapper, developed for a CISI collection, was evaluated by accuracy, time, and user satisfaction. Users' navigation…

Usability/Sentiment for the Enterprise and ENTERPRISE

NASA Technical Reports Server (NTRS)

Meza, David; Berndt, Sarah

2014-01-01

The purpose of the Sentiment of Search Study for NASA Johnson Space Center (JSC) is to gain insight into the intranet search environment. With an initial usability survey, the authors were able to determine a usability score based on the Systems Usability Scale (SUS). Created in 1986, the freely available, well cited, SUS is commonly used to determine user perceptions of a system (in this case the intranet search environment). As with any improvement initiative, one must first examine and document the current reality of the situation. In this scenario, a method was needed to determine the usability of a search interface in addition to the user's perception on how well the search system was providing results. The use of the SUS provided a mechanism to quickly ascertain information in both areas, by adding one additional open-ended question at the end. The first ten questions allowed us to examine the usability of the system, while the last questions informed us on how the users rated the performance of the search results. The final analysis provides us with a better understanding of the current situation and areas to focus on for improvement. The power of search applications to enhance knowledge transfer is indisputable. The performance impact for any user unable to find needed information undermines project lifecycle, resource and scheduling requirements. Ever-increasing complexity of content and the user interface make usability considerations for the intranet, especially for search, a necessity instead of a 'nice-to-have'. Despite these arguments, intranet usability is largely disregarded due to lack of attention beyond the functionality of the infrastructure (White, 2013). The data collected from users of the JSC search system revealed their overall sentiment by means of the widely-known System Usability Scale. Results of the scores suggest 75%, +/-0.04, of the population rank the search system below average. In terms of a grading scaled, this equated to D or lower. It is obvious JSC users are not satisfied with the current situation, however they are eager to provide information and assistance in improving the search system. A majority of the respondents provided feedback on the issues most troubling them. This information will be used to enrich the next phase, root cause analysis and solution creation.

Evaluating the utility of two gestural discomfort evaluation methods

PubMed Central

Son, Minseok; Jung, Jaemoon; Park, Woojin

2017-01-01

Evaluating physical discomfort of designed gestures is important for creating safe and usable gesture-based interaction systems; yet, gestural discomfort evaluation has not been extensively studied in HCI, and few evaluation methods seem currently available whose utility has been experimentally confirmed. To address this, this study empirically demonstrated the utility of the subjective rating method after a small number of gesture repetitions (a maximum of four repetitions) in evaluating designed gestures in terms of physical discomfort resulting from prolonged, repetitive gesture use. The subjective rating method has been widely used in previous gesture studies but without empirical evidence on its utility. This study also proposed a gesture discomfort evaluation method based on an existing ergonomics posture evaluation tool (Rapid Upper Limb Assessment) and demonstrated its utility in evaluating designed gestures in terms of physical discomfort resulting from prolonged, repetitive gesture use. Rapid Upper Limb Assessment is an ergonomics postural analysis tool that quantifies the work-related musculoskeletal disorders risks for manual tasks, and has been hypothesized to be capable of correctly determining discomfort resulting from prolonged, repetitive gesture use. The two methods were evaluated through comparisons against a baseline method involving discomfort rating after actual prolonged, repetitive gesture use. Correlation analyses indicated that both methods were in good agreement with the baseline. The methods proposed in this study seem useful for predicting discomfort resulting from prolonged, repetitive gesture use, and are expected to help interaction designers create safe and usable gesture-based interaction systems. PMID:28423016

Development and usability of a computer-tailored pedometer-based physical activity advice for breast cancer survivors.

PubMed

De Cocker, K; Charlier, C; Van Hoof, E; Pauwels, E; Lechner, L; Bourgois, J; Spittaels, H; Vandelanotte, C; De Bourdeaudhuij, I

2015-09-01

This observational study aimed to adapt a computer-tailored step advice for the general population into a feasible advice for breast cancer survivors and to test its usability. First, several adaptations were made to the original design (adding cancer-related physical activity (PA) barriers and beliefs, and self-management strategies to improve survivors' personal control). Second, the adapted advice was evaluated in two phases: (1) a usability testing in healthy women (n = 3) and survivors (n = 6); and (2) a process evaluation during 3 weeks in breast cancer survivors (n = 8). Preliminary usability testing revealed no problems during logging-in; however, three survivors misinterpreted some questions. After refining the questionnaire and advice, survivors evaluated the advice as interesting, attractive to read, comprehensible and credible. Inactive survivors found the advice novel, but too long. The process evaluation indicated that the majority of the women (n = 5/8) reported increased steps. Monitoring step counts by using a pedometer was perceived as an important motivator to be more active. To conclude, this study provides initial support for the usability and acceptability of a computer-tailored pedometer-based PA advice for breast cancer survivors. After testing efficacy and effectiveness of this intervention, this tool can broaden the reach of PA promotion in breast cancer survivors. © 2014 John Wiley & Sons Ltd.

Proposing Electronic Health Record Usability Requirements Based on Enriched ISO 9241 Metric Usability Model

PubMed Central

Farzandipour, Mehrdad; Riazi, Hossein; Jabali, Monireh Sadeqi

2018-01-01

Introduction: System usability assessment is among the important aspects in assessing the quality of clinical information technology, especially when the end users of the system are concerned. This study aims at providing a comprehensive list of system usability. Methods: This research is a descriptive cross-sectional one conducted using Delphi technique in three phases in 2013. After experts’ ideas were concluded, the final version of the questionnaire including 163 items in three phases was presented to 40 users of information systems in hospitals. The grading ranged from 0-4. Data analysis was conducted using SPSS software. Those requirements with a mean point of three or higher were finally confirmed. Results: The list of system usability requirements for electronic health record was designed and confirmed in nine areas including suitability for the task (24 items), self-descriptiveness (22 items), controllability (19 questions), conformity with user expectations (25 items), error tolerance (21 items), suitability for individualization (7 items), suitability for learning (19 items), visual clarity (18 items) and auditory presentation (8 items). Conclusion: A relatively comprehensive model including useful requirements for using EHR was presented which can increase functionality, effectiveness and users’ satisfaction. Thus, it is suggested that the present model be adopted by system designers and healthcare system institutions to assess those systems. PMID:29719310

A hospital-wide transition from paper to digital problem-oriented clinical notes. A descriptive history and cross-sectional survey of use, usability, and satisfaction.

PubMed

Cillessen, Felix H J M; de Vries Robbé, Pieter F; Biermans, Marion C J

2017-05-17

To evaluate the use, usability, and physician satisfaction of a locally developed problem-oriented clinical notes application that replaced paper-based records in a large Dutch university medical center. Using a clinical notes database and an application event log file and a cross-sectional survey of usability, authors retrospectively analyzed system usage for medical specialties, users, and patients over 4 years. A standardized questionnaire measured usability. Authors analyzed the effects of sex, age, professional experience, training hours, and medical specialty on user satisfaction via univariate analysis of variance. Authors also examined the correlation between user satisfaction in relation to users' intensity of use of the application. In total 1,793 physicians used the application to record progress notes for 219,755 patients. The overall satisfaction score was 3.2 on a scale from 1 (highly dissatisfied) to 5 (highly satisfied). A statistically significant difference occurred in satisfaction by medical specialty, but no statistically significant differences in satisfaction took place by sex, age, professional experience, or training hours. Intensity of system use did not correlate with physician satisfaction. By two years after the start of the implementation, all medical specialties utilized the clinical notes application. User satisfaction was neutral (3.2 on a 1-5 scale). Authors believe that the significant factors facilitating this transition mirrored success factors reported by other groups: a generic, consistent, and transparent design of the application; intensive collaboration; continuous monitoring; and an incremental rollout.

A Pedagogical Approach toward Teaching an Information Systems Student How to Conduct a Web Usability Study for an Honors Project: A Case Study

ERIC Educational Resources Information Center

Jesse, Gayle

2013-01-01

The purpose of this paper is to provide educators with a course model and pedagogy to teach a computer information systems usability course. This paper offers a case study based on an honors student project titled "Web Usability: Phases of Developing an Interactive Event Database." Each individual phase--creating a prototype along with…

A knitted garment using intarsia technique for Heart Rate Variability biofeedback: Evaluation of initial prototype.

PubMed

Abtahi, F; Ji, G; Lu, K; Rödby, K; Seoane, F

2015-01-01

Heart rate variability (HRV) biofeedback is a method based on paced breathing at specific rate called resonance frequency by giving online feedbacks from user respiration and its effect on HRV. Since the HRV is also influence by different factors like stress and emotions, stress related to an unfamiliar measurement device, cables and skin electrodes may cover the underling effect of such kind of intervention. Wearable systems are usually considered as intuitive solutions which are more familiar to the end-user and can help to improve usability and hence reducing the stress. In this work, a prototype of a knitted garment using intarsia technique is developed and evaluated. Results show the satisfactory level of quality for Electrocardiogram and thoracic electrical bioimpedance i.e. for respiration monitoring as a part of HRV biofeedback system. Using intarsia technique and conductive yarn for making the connection instead of cables will reduce the complexity of fabrication in textile production and hence reduce the final costs in a final commercial product. Further development of garment and Android application is ongoing and usability and efficiency of final prototype will be evaluated in detail.

Assessing the Usability of University Websites: An Empirical Study on Namik Kemal University

ERIC Educational Resources Information Center

Mentes, S. Ahmet; Turan, Aykut H.

2012-01-01

Web sites are emerging as a key component of an organization's survival in our ever globalizing competitive world. Usability of a web site has assumed a great deal of importance in terms of satisfying web site users' needs and expectations. The aim of the study is to evaluate and to explore the usability level of Namik Kemal University (NKU)…

A review of privacy and usability issues in mobile health systems: Role of external factors.

PubMed

Katusiime, Jane; Pinkwart, Niels

2017-10-01

The increased penetration of mobile devices has created opportunities in the health sector and led to emerging of mobile health systems. As much as the mobile health systems have registered tremendous progress, they have been faced with privacy and usability issues. Due to the sensitivity of health information, there is an ethical need to equip mobile health systems with adequate privacy measures. However, these systems should also be useable by the intended users. Even though many researchers are working on solutions, the issues still persist. External factors such as cultural differences have also contributed to the issues, yet they have been under researched. In this article, we conduct a systematic literature review of 22 articles, categorize and present privacy and usability issues and possible solutions. We then discuss the relevance and implications of external factors to the findings on privacy and usability. We end with recommendations to address these external factors.

Usability Evaluation of a Web-Based Symptom Monitoring Application for Heart Failure.

PubMed

Wakefield, Bonnie; Pham, Kassie; Scherubel, Melody

2015-07-01

Symptom recognition and reporting by patients with heart failure are critical to avoid hospitalization. This project evaluated a patient symptom tracking application. Fourteen end users (nine patients, five clinicians) from a Midwestern Veterans Affairs Medical Center evaluated the website using a think aloud protocol. A structured observation protocol was used to assess success or failure for each task. Measures included task time, success, and satisfaction. Patients had a mean age of 70 years; clinicians averaged 42 years in age. Patients took 9.3 min and clinicians took less than 3 min per scenario. Most patients needed some assistance, but few patients were completely unable to complete some tasks. Clinicians demonstrated few problems navigating the site. Patient System Usability Scale item scores ranged from 2.0 to 3.6; clinician item scores ranged from 1.8 to 4.0. Further work is needed to determine whether using the web-based tool improves symptom recognition and reporting. © The Author(s) 2015.

Usability testing of ANSWER: a web-based methotrexate decision aid for patients with rheumatoid arthritis

PubMed Central

2013-01-01

Background Decision aids are evidence-based tools designed to inform people of the potential benefit and harm of treatment options, clarify their preferences and provide a shared decision-making structure for discussion at a clinic visit. For patients with rheumatoid arthritis (RA) who are considering methotrexate, we have developed a web-based patient decision aid called the ANSWER (Animated, Self-serve, Web-based Research Tool). This study aimed to: 1) assess the usability of the ANSWER prototype; 2) identify strengths and limitations of the ANSWER from the patient’s perspective. Methods The ANSWER prototype consisted of: 1) six animated patient stories and narrated information on the evidence of methotrexate for RA; 2) interactive questionnaires to clarify patients’ treatment preferences. Eligible participants for the usability test were patients with RA who had been prescribed methotrexate. They were asked to verbalize their thoughts (i.e., think aloud) while using the ANSWER, and to complete the System Usability Scale (SUS) to assess overall usability (range = 0-100; higher = more user friendly). Participants were audiotaped and observed, and field notes were taken. The testing continued until no new modifiable issues were found. We used descriptive statistics to summarize participant characteristics and the SUS scores. Content analysis was used to identified usability issues and navigation problems. Results 15 patients participated in the usability testing. The majority were aged 50 or over and were university/college graduates (n = 8, 53.4%). On average they took 56 minutes (SD = 34.8) to complete the tool. The mean SUS score was 81.2 (SD = 13.5). Content analysis of audiotapes and field notes revealed four categories of modifiable usability issues: 1) information delivery (i.e., clarity of the information and presentation style); 2) navigation control (i.e., difficulties in recognizing and using the navigation control buttons); 3) layout (i.e., position of the videos, text, diagrams and navigation buttons); 4) aesthetic (i.e., the colour, look and feel of the online tool). Conclusions Although the SUS score indicated high usability before and after major modification, findings from the think-aloud sessions illustrated areas that required further refinement. Our results highlight the importance of formative evaluation in usability testing. PMID:24289731

Usability testing of ANSWER: a web-based methotrexate decision aid for patients with rheumatoid arthritis.

PubMed

Li, Linda C; Adam, Paul M; Townsend, Anne F; Lacaille, Diane; Yousefi, Charlene; Stacey, Dawn; Gromala, Diane; Shaw, Chris D; Tugwell, Peter; Backman, Catherine L

2013-12-01

Decision aids are evidence-based tools designed to inform people of the potential benefit and harm of treatment options, clarify their preferences and provide a shared decision-making structure for discussion at a clinic visit. For patients with rheumatoid arthritis (RA) who are considering methotrexate, we have developed a web-based patient decision aid called the ANSWER (Animated, Self-serve, Web-based Research Tool). This study aimed to: 1) assess the usability of the ANSWER prototype; 2) identify strengths and limitations of the ANSWER from the patient's perspective. The ANSWER prototype consisted of: 1) six animated patient stories and narrated information on the evidence of methotrexate for RA; 2) interactive questionnaires to clarify patients' treatment preferences. Eligible participants for the usability test were patients with RA who had been prescribed methotrexate. They were asked to verbalize their thoughts (i.e., think aloud) while using the ANSWER, and to complete the System Usability Scale (SUS) to assess overall usability (range = 0-100; higher = more user friendly). Participants were audiotaped and observed, and field notes were taken. The testing continued until no new modifiable issues were found. We used descriptive statistics to summarize participant characteristics and the SUS scores. Content analysis was used to identified usability issues and navigation problems. 15 patients participated in the usability testing. The majority were aged 50 or over and were university/college graduates (n = 8, 53.4%). On average they took 56 minutes (SD = 34.8) to complete the tool. The mean SUS score was 81.2 (SD = 13.5). Content analysis of audiotapes and field notes revealed four categories of modifiable usability issues: 1) information delivery (i.e., clarity of the information and presentation style); 2) navigation control (i.e., difficulties in recognizing and using the navigation control buttons); 3) layout (i.e., position of the videos, text, diagrams and navigation buttons); 4) aesthetic (i.e., the colour, look and feel of the online tool). Although the SUS score indicated high usability before and after major modification, findings from the think-aloud sessions illustrated areas that required further refinement. Our results highlight the importance of formative evaluation in usability testing.

NASA Johnson Space Center Usability Testing and Analysis Facility (UTAF) Overview

NASA Technical Reports Server (NTRS)

Whitmore, M.

2004-01-01

The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility provides support to the Office of Biological and Physical Research, the Space Shuttle Program, the International Space Station Program, and other NASA organizations. In addition, there are ongoing collaborative research efforts with external businesses and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes. This presentation will provide an overview of ongoing activities, and will address how the projects will evolve to meet new space initiatives.

NASA Johnson Space Center Usability Testing and Analysis Facility (WAF) Overview

NASA Technical Reports Server (NTRS)

Whitmore, M.

2004-01-01

The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility provides support to the Office of Biological and Physical Research, the Space Shuttle Program, the International Space Station Program, and other NASA organizations. In addition, there are ongoing collaborative research efforts with external businesses and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes. This presentation will provide an overview of ongoing activities, and will address how the projects will evolve to meet new space initiatives.

Web-based Education for Low-literate Parents in Neonatal Intensive Care Unit: Development of a Website and Heuristic Evaluation and Usability Testing

PubMed Central

Choi, Jeungok; Bakken, Suzanne

2010-01-01

Purpose Low health literacy has been associated with poor health-related outcomes. The purposes are to report the development of a website for low-literate parents in the Neonatal Intensive Care Unit (NICU), and the findings of heuristic evaluation and a usability testing of this website. Methods To address low literacy of NICU parents, multimedia educational Website using visual aids (e.g., pictographs, photographs), voice-recorded text message in addition to a simplified text was developed. The text was created at the 5th grade readability level. The heuristic evaluation was conducted by three usability experts using 10 heuristics. End-users’ performance was measured by counting the time spent completing tasks and number of errors, as well as recording users’ perception of ease of use and usefulness (PEUU) in a sample of 10 NICU parents. Results Three evaluators identified 82 violations across the 10 heuristics. All violations, however, received scores <2, indicating minor usability problems. Participants’ time to complete task varies from 81.2 seconds (SD=30.9) to 2.2 seconds (SD=1.3). Participants rated the Website as easy to use and useful (PEUU Mean= 4.52, SD=0.53). Based on the participants’ comments, appropriate modifications were made. Discussion and Conclusions Different types of visuals on the Website were well accepted by low-literate users and agreement of visuals with text improved understanding of the educational materials over that with text alone. The findings suggest that using concrete and realistic pictures and pictographs with clear captions would maximize the benefit of visuals. One emerging theme was “simplicity” in design (e.g., limited use of colors, one font type and size), content (e.g., avoid lengthy text), and technical features (e.g., limited use of pop-ups). The heuristic evaluation by usability experts and the usability test with actual users provided complementary expertise, which can give a richer assessment of a design for low literacy Website. These results facilitated design modification and implementation of solutions by categorizing and prioritizing the usability problems. PMID:20617546

Web-based education for low-literate parents in Neonatal Intensive Care Unit: development of a website and heuristic evaluation and usability testing.

PubMed

Choi, Jeungok; Bakken, Suzanne

2010-08-01

Low health literacy has been associated with poor health-related outcomes. The purposes are to report the development of a website for low-literate parents in the Neonatal Intensive Care Unit (NICU), and the findings of heuristic evaluation and a usability testing of this website. To address low literacy of NICU parents, multimedia educational Website using visual aids (e.g., pictographs, photographs), voice-recorded text message in addition to a simplified text was developed. The text was created at the 5th grade readability level. The heuristic evaluation was conducted by three usability experts using 10 heuristics. End-users' performance was measured by counting the time spent completing tasks and number of errors, as well as recording users' perception of ease of use and usefulness (PEUU) in a sample of 10 NICU parents. Three evaluators identified 82 violations across the 10 heuristics. All violations, however, received scores <2, indicating minor usability problems. Participants' time to complete task varies from 81.2 s (SD = 30.9) to 2.2 s (SD = 1.3). Participants rated the Website as easy to use and useful (PEUU mean = 4.52, SD = 0.53). Based on the participants' comments, appropriate modifications were made. Different types of visuals on the Website were well accepted by low-literate users and agreement of visuals with text improved understanding of the educational materials over that with text alone. The findings suggest that using concrete and realistic pictures and pictographs with clear captions would maximize the benefit of visuals. One emerging theme was "simplicity" in design (e.g., limited use of colors, one font type and size), content (e.g., avoid lengthy text), and technical features (e.g., limited use of pop-ups). The heuristic evaluation by usability experts and the usability test with actual users provided complementary expertise, which can give a richer assessment of a design for low literacy Website. These results facilitated design modification and implementation of solutions by categorizing and prioritizing the usability problems.

Multi-Touch Tabletop System Using Infrared Image Recognition for User Position Identification.

PubMed

Suto, Shota; Watanabe, Toshiya; Shibusawa, Susumu; Kamada, Masaru

2018-05-14

A tabletop system can facilitate multi-user collaboration in a variety of settings, including small meetings, group work, and education and training exercises. The ability to identify the users touching the table and their positions can promote collaborative work among participants, so methods have been studied that involve attaching sensors to the table, chairs, or to the users themselves. An effective method of recognizing user actions without placing a burden on the user would be some type of visual process, so the development of a method that processes multi-touch gestures by visual means is desired. This paper describes the development of a multi-touch tabletop system using infrared image recognition for user position identification and presents the results of touch-gesture recognition experiments and a system-usability evaluation. Using an inexpensive FTIR touch panel and infrared light, this system picks up the touch areas and the shadow area of the user's hand by an infrared camera to establish an association between the hand and table touch points and estimate the position of the user touching the table. The multi-touch gestures prepared for this system include an operation to change the direction of an object to face the user and a copy operation in which two users generate duplicates of an object. The system-usability evaluation revealed that prior learning was easy and that system operations could be easily performed.

Multi-Touch Tabletop System Using Infrared Image Recognition for User Position Identification

PubMed Central

Suto, Shota; Watanabe, Toshiya; Shibusawa, Susumu; Kamada, Masaru

2018-01-01

A tabletop system can facilitate multi-user collaboration in a variety of settings, including small meetings, group work, and education and training exercises. The ability to identify the users touching the table and their positions can promote collaborative work among participants, so methods have been studied that involve attaching sensors to the table, chairs, or to the users themselves. An effective method of recognizing user actions without placing a burden on the user would be some type of visual process, so the development of a method that processes multi-touch gestures by visual means is desired. This paper describes the development of a multi-touch tabletop system using infrared image recognition for user position identification and presents the results of touch-gesture recognition experiments and a system-usability evaluation. Using an inexpensive FTIR touch panel and infrared light, this system picks up the touch areas and the shadow area of the user’s hand by an infrared camera to establish an association between the hand and table touch points and estimate the position of the user touching the table. The multi-touch gestures prepared for this system include an operation to change the direction of an object to face the user and a copy operation in which two users generate duplicates of an object. The system-usability evaluation revealed that prior learning was easy and that system operations could be easily performed. PMID:29758006

An Evaluation of Learning Objects in Singapore Primary Education: A Case Study Approach

ERIC Educational Resources Information Center

Grace, Tay Pei Lyn; Suan, Ng Peck; Wanzhen, Liaw

2008-01-01

Purpose: The purpose of this paper is to evaluate the usability and interface design of e-learning portal developed for primary schools in Singapore. Design/methodology/approach: Using Singapore-based learning EDvantage (LEAD) portal as a case study, this paper reviews and analyses the usability and usefulness of embedded learning objects (LOs)…

Web Accessibility and Usability of the Homepages from Academy of Human Resource Development Members' Institutions

ERIC Educational Resources Information Center

Zeng, Xiaoming; Sligar, Steven R.

2008-01-01

Human resource development programs in various institutions communicate with their constituencies including persons with disabilities through websites. Web sites need to be accessible for legal, economic and ethical reasons. We used an automated web usability evaluation tool, aDesigner, to evaluate 205 home pages from the organizations of AHRD…

A Concise and Practical Framework for the Development and Usability Evaluation of Patient Information Websites.

PubMed

Peute, L W; Knijnenburg, S L; Kremer, L C; Jaspers, M W M

2015-01-01

The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS). To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population. The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA. The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content. Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations.

Usability evaluation with mental health professionals and young people to develop an Internet-based cognitive-behaviour therapy program for adolescents with anxiety disorders.

PubMed

Wozney, Lori; Baxter, Pamela; Newton, Amanda S

2015-12-16

Use of the Internet to deliver cognitive behavioural therapy, a frontline treatment for anxiety disorders, is emerging as an option to increase access to treatment among adolescents with anxiety disorders. This study examined the usability of the Internet-based component of Breathe, a CBT program designed for adolescents with mild to moderate anxiety and impairments. A mixed-method usability testing design with semi-structured interviews, task completion, and survey by trained usability moderators was undertaken with two interactive cycles to determine the usability (ease of use, efficiency, errors, and user satisfaction) of the user interface and content areas of the program. Purposeful sampling was used to recruit mental health clinicians with expertise in treating adolescent anxiety disorders and young people aged 15 to 24 years involved. Testing involved using Web-conferencing software that allowed remote participation through personal computers. Two testing cycles involved participants completing structured 'think aloud' and 'cognitive walkthrough' tasks within the program. At the end of each cycle participants completed a 15-item global usability evaluation survey and were asked a series of open-ended questions. Descriptive and simple content analyses were used to identify and score usability issues for frequency and severity. Five clinicians and four young people (all < 20 years of age) participated. Most participants described their computer skills as 'good' (60% clinicians, 50% young people), and attitudes toward Internet-based health care ranged from negative (75% young people) to positive (60% clinicians, 25% young people). Scores from the global usability evaluation after both testing cycles ranged from 3.5 to 5 out of 5 in strong agreement/support of the program in terms of user performance indicators (i.e., learnability, efficiency and number of errors) and user satisfaction. Participants were able to complete all critical tasks with minimal errors. Errors and issues identified during testing were predominantly around enhancements to the visual design and navigational support. Opinions across usability elements did not differ between young people and clinician participants. A multi-method remote usability approach provided the opportunity to improve the technical interface, therapeutic messaging and user experience of an Internet-based treatment program for adolescent anxiety disorders.

A stable wireless energy transmission system for gastrointestinal microsystems.

PubMed

Xin, W H; Yan, G Z; Wang, W X

2010-01-01

A wireless energy transmission system using a Helmholtz primary coil outside and a 3-dimensional secondary coil inside the body is introduced. It is designed to transmit stable power to a gastrointestinal microsystem regardless of its position and orientation when working in the gastric tract. Up to 310 mW of usable DC power can be delivered under worst-case geometrical conditions. Measured data of the system performance are presented and evaluated.

Passive sensor technology interface to assess elder activity in independent living.

PubMed

Alexander, Gregory L; Wakefield, Bonnie J; Rantz, Marilyn; Skubic, Marjorie; Aud, Myra A; Erdelez, Sanda; Ghenaimi, Said Al

2011-01-01

The effectiveness of clinical information systems to improve nursing and patient outcomes depends on human factors, including system usability, organizational workflow, and user satisfaction. The aim of this study was to examine to what extent residents, family members, and clinicians find a sensor data interface used to monitor elder activity levels usable and useful in an independent living setting. Three independent expert reviewers conducted an initial heuristic evaluation. Subsequently, 20 end users (5 residents, 5 family members, 5 registered nurses, and 5 physicians) participated in the evaluation. During the evaluation, each participant was asked to complete three scenarios taken from three residents. Morae recorder software was used to capture data during the user interactions. The heuristic evaluation resulted in 26 recommendations for interface improvement; these were classified under the headings content, aesthetic appeal, navigation, and architecture, which were derived from heuristic results. Total time for elderly residents to complete scenarios was much greater than for other users. Family members spent more time than clinicians but less time than residents did to complete scenarios. Elder residents and family members had difficulty interpreting clinical data and graphs, experienced information overload, and did not understand terminology. All users found the sensor data interface useful for identifying changing resident activities. Older adult users have special needs that should be addressed when designing clinical interfaces for them, especially information as important as health information. Evaluating human factors during user interactions with clinical information systems should be a requirement before implementation.

A Usability Case Study Using TREC and ZPRISE.

ERIC Educational Resources Information Center

Downey, Laura L.; Tice, Dawn M.

1999-01-01

Examines the challenges involved in conducting an informal usability case study based on the introduction of a new information-retrieval system to experienced users. Identifies problems users were having with TREC (Text Retrieval Conference) and examines the usability of the new ZPRISE interface. (Author/LRW)

A Method for Automated Detection of Usability Problems from Client User Interface Events

PubMed Central

Saadawi, Gilan M.; Legowski, Elizabeth; Medvedeva, Olga; Chavan, Girish; Crowley, Rebecca S.

2005-01-01

Think-aloud usability analysis provides extremely useful data but is very time-consuming and expensive to perform because of the extensive manual video analysis that is required. We describe a simple method for automated detection of usability problems from client user interface events for a developing medical intelligent tutoring system. The method incorporates (1) an agent-based method for communication that funnels all interface events and system responses to a centralized database, (2) a simple schema for representing interface events and higher order subgoals, and (3) an algorithm that reproduces the criteria used for manual coding of usability problems. A correction factor was empirically determining to account for the slower task performance of users when thinking aloud. We tested the validity of the method by simultaneously identifying usability problems using TAU and manually computing them from stored interface event data using the proposed algorithm. All usability problems that did not rely on verbal utterances were detectable with the proposed method. PMID:16779121

Methods for semi-automated indexing for high precision information retrieval.

PubMed

Berrios, Daniel C; Cucina, Russell J; Fagan, Lawrence M

2002-01-01

To evaluate a new system, ISAID (Internet-based Semi-automated Indexing of Documents), and to generate textbook indexes that are more detailed and more useful to readers. Pilot evaluation: simple, nonrandomized trial comparing ISAID with manual indexing methods. Methods evaluation: randomized, cross-over trial comparing three versions of ISAID and usability survey. Pilot evaluation: two physicians. Methods evaluation: twelve physicians, each of whom used three different versions of the system for a total of 36 indexing sessions. Total index term tuples generated per document per minute (TPM), with and without adjustment for concordance with other subjects; inter-indexer consistency; ratings of the usability of the ISAID indexing system. Compared with manual methods, ISAID decreased indexing times greatly. Using three versions of ISAID, inter-indexer consistency ranged from 15% to 65% with a mean of 41%, 31%, and 40% for each of three documents. Subjects using the full version of ISAID were faster (average TPM: 5.6) and had higher rates of concordant index generation. There were substantial learning effects, despite our use of a training/run-in phase. Subjects using the full version of ISAID were much faster by the third indexing session (average TPM: 9.1). There was a statistically significant increase in three-subject concordant indexing rate using the full version of ISAID during the second indexing session (p < 0.05). Users of the ISAID indexing system create complex, precise, and accurate indexing for full-text documents much faster than users of manual methods. Furthermore, the natural language processing methods that ISAID uses to suggest indexes contributes substantially to increased indexing speed and accuracy.

Evaluating the usability of a commercial cooling vest in the Hong Kong industries.

PubMed

Chan, Albert P; Yang, Yang; Song, Wen-Fang

2018-03-01

The provision of appropriate personal cooling vests is recognized as an effective measure to combat heat stress. However, personal cooling vests are not widely implemented in the Hong Kong industries. The current study aims to evaluate the usability of a hybrid cooling vest that is associated with the success of its application in industrial settings. A self-administrated questionnaire focusing on 10 subjective attributes of cooling effect, ergonomic design and usability of a hybrid cooling vest was administered with 232 occupational workers in the construction, horticultural and cleaning, airport apron services and kitchen and catering industries. A structural equation model estimated by analysis of moment structures was constructed to evaluate the usability of the cooling vest, as influenced by cooling effect and ergonomic design. Results showed that cooling effect (path coefficient = 0.69, p < 0.001) and ergonomic design (path coefficient = 0.55, p < 0.001) significantly affect the usability of the cooling vest. The structural equation model is feasible to examine the complex nature of the structural relationships among the subjective perceptions of personal cooling vests. The empirical findings furnish sound evidence for further optimization of the hybrid cooling vest in terms of cooling effect and ergonomic design for occupational workers.

Current State of Agile User-Centered Design: A Survey

NASA Astrophysics Data System (ADS)

Hussain, Zahid; Slany, Wolfgang; Holzinger, Andreas

Agile software development methods are quite popular nowadays and are being adopted at an increasing rate in the industry every year. However, these methods are still lacking usability awareness in their development lifecycle, and the integration of usability/User-Centered Design (UCD) into agile methods is not adequately addressed. This paper presents the preliminary results of a recently conducted online survey regarding the current state of the integration of agile methods and usability/UCD. A world wide response of 92 practitioners was received. The results show that the majority of practitioners perceive that the integration of agile methods with usability/UCD has added value to their adopted processes and to their teams; has resulted in the improvement of usability and quality of the product developed; and has increased the satisfaction of the end-users of the product developed. The top most used HCI techniques are low-fidelity prototyping, conceptual designs, observational studies of users, usability expert evaluations, field studies, personas, rapid iterative testing, and laboratory usability testing.

A Comparison of What Is Part of Usability Testing in Three Countries

NASA Astrophysics Data System (ADS)

Clemmensen, Torkil

The cultural diversity of users of technology challenges our methods for usability evaluation. In this paper we report and compare three ethnographic interview studies of what is a part of a standard (typical) usability test in a company in Mumbai, Beijing and Copenhagen. At each of these three locations, we use structural and contrast questions do a taxonomic and paradigm analysis of a how a company performs a usability test. We find similar parts across the three locations. We also find different results for each location. In Mumbai, most parts of the usability test are not related to the interactive application that is tested, but to differences in user characteristics, test preparation, method, and location. In Copenhagen, considerations about the client's needs are part of a usability test. In Beijing, the only varying factor is the communication pattern and relation to the user. These results are then contrasted in a cross cultural matrix to identify cultural themes that can help interpret results from existing laboratory research in usability test methods.

A user experience evaluation of Amazon Kindle mobile application

NASA Astrophysics Data System (ADS)

Hussain, Azham; Mkpojiogu, Emmanuel O. C.; Musa, Ja'afaru; Mortada, Salah

2017-10-01

There is a dramatic increase in the development of mobile applications in recent years. This makes the usability evaluation of these mobile applications an important aspect in the advancement and application of technology. In this paper, a laboratory-based usability evaluation was carried out on the Amazon Kindle app using 15 users who performed 5 tasks on the Kindle e-book mobile app. A post-test questionnaire was administered to elicit users' perception on the usability of the application. The results demonstrate that almost all the participants were satisfied with services provided by the Amazon Kindle e-book mobile app. On all the four user experience factors examined, namely, perceived ease-of-use, perceived visibility, perceived enjoyabilty, and perceived efficiency, the evaluation outcome shows that the participants had a good and rich mobile experience with the application.

Electronic physiologic and subjective data acquisition in home-dwelling heart failure patients: An assessment of patient use and perception of usability.

PubMed

Gardner, Cubby L; Flanagan, Michael C; Franklin, Cathy; John-Swayers, Cherly; Walsh-Pouch, Stacy; Bryant, F Joyce; Romano, Carol A; Gibbons, Susanne; De Jong, Marla; Hoang, Albert; Becher, Dorothy; Burke, Harry B

2016-09-01

The current approach to the outpatient management of heart failure involves patients recollecting what has happened to them since their last clinic visit. But patients' recollection of their symptoms may not be sufficiently accurate to optimally manage their disease. Most of what is known about heart failure is related to patients' diurnal symptoms and activities. Some mobile electronic technologies can operate continuously to collect data from the time patients go to bed until they get up in the morning. We were therefore interested to evaluate if patients would use a system of selected patient-facing devices to collect physiologic and subjective state data in and around the patients' period of sleep, and if there were differences in device use and perceptions of usability at the device level This descriptive observational study of home-dwelling patients with heart failure, between 21 and 90 years of age, enrolled in an outpatient heart failure clinic was conducted between December 2014 and June 2015. Patients received five devices, namely, body weight scale, blood pressure device, an iPad-based subjective states assessment, pulse oximeter, and actigraph, to collect their physiologic (body weight, blood pressure, heart rate, blood oxygen saturation, and physical activity) and subjective state data (symptoms and subjective states) at home for the next six consecutive nights. Use was defined as the ratio of observed use over expected use, where 1.0 is observed equals expected. Usability was determined by the overall System Usability Scale score. Participants were 39 clinical heart failure patients, mean age 68.1 (SD, 12.3), 72% male, 62% African American. The ratio of observed over expected use for the body weight scale, blood pressure device, iPad application, pulse oximeter and actigraph was 0.8, 1.0, 1.1, 0.9, and 1.9, respectively. The mean overall System Usability Scale score for each device were 84.5, 89.7, 85.7, 87.6, and 85.2, respectively. Patients were able to use all of the devices and they rated the usability of all the devices higher than expected. Our study provides support for at-home patient-collected physiologic and subjective state data. To our knowledge, this is the first study to assess the use and usability of electronic objective and subjective data collection devices in heart failure patients' homes overnight. Published by Elsevier Ireland Ltd.

Developing and Evaluating JIApp: Acceptability and Usability of a Smartphone App System to Improve Self-Management in Young People With Juvenile Idiopathic Arthritis

PubMed Central

Beste, Dominik; Chaplin, Hema; Varakliotis, Socrates; Suffield, Linda; Josephs, Francesca; Sen, Debajit; Wedderburn, Lucy R; Ioannou, Yiannakis; Hailes, Stephen; Eleftheriou, Despina

2017-01-01

Background Flare-ups in juvenile idiopathic arthritis (JIA) are characterized by joint pain and swelling and often accompanied with fatigue, negative emotions, and reduced participation in activities. To minimize the impact of JIA on the physical and psychosocial development and well-being of young people (YP), it is essential to regularly monitor disease activity and side effects, as well as to support self-management such as adherence to treatment plans and engagement in general health-promoting behaviors. Smartphone technology has the potential to engage YP with their health care through convenient self-monitoring and easy access to information. In addition, having a more accurate summary of self-reported fluctuations in symptoms, behaviors, and psychosocial problems can help both YP and health care professionals (HCPs) better understand the patient’s condition, identify barriers to self-management, and assess treatment effectiveness and additional health care needs. No comprehensive smartphone app has yet been developed in collaboration with YP with JIA, their parents, and HCPs involved in their care. Objectives The objective of this study was to design, develop, and evaluate the acceptability and usability of JIApp, a self-management smartphone app system for YP with JIA and HCPs. Methods We used a qualitative, user-centered design approach involving YP, parents, and HCPs from the rheumatology team. The study was conducted in three phases: (1) phase I focused on developing consensus on the features, content, and design of the app; (2) phase II was used for further refining and evaluating the app prototype; and (3) phase III focused on usability testing of the app. The interview transcripts were analyzed using qualitative content analysis. Results A total of 29 YP (aged 10-23, median age 17) with JIA, 7 parents, and 21 HCPs were interviewed. Major themes identified as the ones that helped inform app development in phase I were: (1) remote monitoring of symptoms, well-being, and activities; (2) treatment adherence; and (3) education and support. During phase II, three more themes emerged that informed further refinement of the app prototype. These included (4) adapting a reward system to motivate end users for using the app; (5) design of the app interface; and (6) clinical practice integration. The usability testing during phase III demonstrated high rates of overall satisfaction and further affirmed the content validity of the app. Conclusions We present the development and evaluation of a smartphone app to encourage self-management and engagement with health care for YP with JIA. The app was found to have high levels of acceptability and usability among YP and HCPs and has the potential to improve health care and outcomes for this age group. Future feasibility testing in a prospective study will firmly establish the reliability, efficacy, and cost-effectiveness of such an app intervention for patients with arthritis. PMID:28811270

An Evaluation of Understandability of Patient Journey Models in Mental Health.

PubMed

Percival, Jennifer; McGregor, Carolyn

2016-07-28

There is a significant trend toward implementing health information technology to reduce administrative costs and improve patient care. Unfortunately, little awareness exists of the challenges of integrating information systems with existing clinical practice. The systematic integration of clinical processes with information system and health information technology can benefit the patients, staff, and the delivery of care. This paper presents a comparison of the degree of understandability of patient journey models. In particular, the authors demonstrate the value of a relatively new patient journey modeling technique called the Patient Journey Modeling Architecture (PaJMa) when compared with traditional manufacturing based process modeling tools. The paper also presents results from a small pilot case study that compared the usability of 5 modeling approaches in a mental health care environment. Five business process modeling techniques were used to represent a selected patient journey. A mix of both qualitative and quantitative methods was used to evaluate these models. Techniques included a focus group and survey to measure usability of the various models. The preliminary evaluation of the usability of the 5 modeling techniques has shown increased staff understanding of the representation of their processes and activities when presented with the models. Improved individual role identification throughout the models was also observed. The extended version of the PaJMa methodology provided the most clarity of information flows for clinicians. The extended version of PaJMa provided a significant improvement in the ease of interpretation for clinicians and increased the engagement with the modeling process. The use of color and its effectiveness in distinguishing the representation of roles was a key feature of the framework not present in other modeling approaches. Future research should focus on extending the pilot case study to a more diversified group of clinicians and health care support workers.

Data Processing and Quality Evaluation of a Boat-Based Mobile Laser Scanning System

PubMed Central

Vaaja, Matti; Kukko, Antero; Kaartinen, Harri; Kurkela, Matti; Kasvi, Elina; Flener, Claude; Hyyppä, Hannu; Hyyppä, Juha; Järvelä, Juha; Alho, Petteri

2013-01-01

Mobile mapping systems (MMSs) are used for mapping topographic and urban features which are difficult and time consuming to measure with other instruments. The benefits of MMSs include efficient data collection and versatile usability. This paper investigates the data processing steps and quality of a boat-based mobile mapping system (BoMMS) data for generating terrain and vegetation points in a river environment. Our aim in data processing was to filter noise points, detect shorelines as well as points below water surface and conduct ground point classification. Previous studies of BoMMS have investigated elevation accuracies and usability in detection of fluvial erosion and deposition areas. The new findings concerning BoMMS data are that the improved data processing approach allows for identification of multipath reflections and shoreline delineation. We demonstrate the possibility to measure bathymetry data in shallow (0–1 m) and clear water. Furthermore, we evaluate for the first time the accuracy of the BoMMS ground points classification compared to manually classified data. We also demonstrate the spatial variations of the ground point density and assess elevation and vertical accuracies of the BoMMS data. PMID:24048340

Data processing and quality evaluation of a boat-based mobile laser scanning system.

PubMed

Vaaja, Matti; Kukko, Antero; Kaartinen, Harri; Kurkela, Matti; Kasvi, Elina; Flener, Claude; Hyyppä, Hannu; Hyyppä, Juha; Järvelä, Juha; Alho, Petteri

2013-09-17

Mobile mapping systems (MMSs) are used for mapping topographic and urban features which are difficult and time consuming to measure with other instruments. The benefits of MMSs include efficient data collection and versatile usability. This paper investigates the data processing steps and quality of a boat-based mobile mapping system (BoMMS) data for generating terrain and vegetation points in a river environment. Our aim in data processing was to filter noise points, detect shorelines as well as points below water surface and conduct ground point classification. Previous studies of BoMMS have investigated elevation accuracies and usability in detection of fluvial erosion and deposition areas. The new findings concerning BoMMS data are that the improved data processing approach allows for identification of multipath reflections and shoreline delineation. We demonstrate the possibility to measure bathymetry data in shallow (0-1 m) and clear water. Furthermore, we evaluate for the first time the accuracy of the BoMMS ground points classification compared to manually classified data. We also demonstrate the spatial variations of the ground point density and assess elevation and vertical accuracies of the BoMMS data.

Content and Usability Evaluation of Patient Oriented Drug-Drug Interaction Websites.

PubMed

Adam, Terrence J; Vang, Joseph

Drug-Drug Interactions (DDI) are an important source of preventable adverse drug events and a common reason for hospitalization among patients on multiple drug therapy regimens. DDI information systems are important patient safety tools with the capacity to identify and warn health professionals of clinically significant DDI risk. While substantial research has been completed on DDI information systems in professional settings such as community, hospital, and independent pharmacies; there has been limited research on DDI systems offered through online websites directly for use by ambulatory patients. The focus of this project is to test patient oriented website capacity to correctly identify drug interactions among well established and clinically significant medication combinations and convey clinical risk data to patients. The patient education capability was assessed by evaluating website Information Capacity, Patient Usability and Readability. The study results indicate that the majority of websites identified which met the inclusion and exclusion criteria operated similarly, but vary in risk severity assessment and are not optimally patient oriented to effectively deliver risk information. The limited quality of information and complex medical term content complicate DDI risk data conveyance and the sites may not provide optimal information delivery to allow medication consumers to understand and manage their medication regimens.

VoIP Accessibility: A Usability Study of Voice over Internet Protocol (VoIP) Systems and A Survey of VoIP Users with Vision Loss

ERIC Educational Resources Information Center

Packer, Jaclyn; Reuschel, William

2018-01-01

Introduction: Accessibility of Voice over Internet Protocol (VoIP) systems was tested with a hands-on usability study and an online survey of VoIP users who are visually impaired. The survey examined the importance of common VoIP features, and both methods assessed difficulty in using those features. Methods: The usability test included four paid…

Mobile phone-based interactive voice response as a tool for improving access to healthcare in remote areas in Ghana - an evaluation of user experiences.

PubMed

Brinkel, J; May, J; Krumkamp, R; Lamshöft, M; Kreuels, B; Owusu-Dabo, E; Mohammed, A; Bonacic Marinovic, A; Dako-Gyeke, P; Krämer, A; Fobil, J N

2017-05-01

To investigate and determine the factors that enhanced or constituted barriers to the acceptance of an mHealth system which was piloted in Asante-Akim North District of Ghana to support healthcare of children. Four semi-structured focus group discussions were conducted with a total of 37 mothers. Participants were selected from a study population of mothers who subscribed to a pilot mHealth system which used an interactive voice response (IVR) for its operations. Data were evaluated using qualitative content analysis methods. In addition, a short quantitative questionnaire assessed system's usability (SUS). Results revealed 10 categories of factors that facilitated user acceptance of the IVR system including quality-of-care experience, health education and empowerment of women. The eight categories of factors identified as barriers to user acceptance included the lack of human interaction, lack of update and training on the electronic advices provided and lack of social integration of the system into the community. The usability (SUS median: 79.3; range: 65-97.5) of the system was rated acceptable. The principles of the tested mHealth system could be of interest during infectious disease outbreaks, such as Ebola or Lassa fever, when there might be a special need for disease-specific health information within populations. © 2017 John Wiley & Sons Ltd.

A Human-Centered Design Methodology to Enhance the Usability, Human Factors, and User Experience of Connected Health Systems: A Three-Phase Methodology.

PubMed

Harte, Richard; Glynn, Liam; Rodríguez-Molinero, Alejandro; Baker, Paul Ma; Scharf, Thomas; Quinlan, Leo R; ÓLaighin, Gearóid

2017-03-16

Design processes such as human-centered design, which involve the end user throughout the product development and testing process, can be crucial in ensuring that the product meets the needs and capabilities of the user, particularly in terms of safety and user experience. The structured and iterative nature of human-centered design can often present a challenge when design teams are faced with the necessary, rapid, product development life cycles associated with the competitive connected health industry. We wanted to derive a structured methodology that followed the principles of human-centered design that would allow designers and developers to ensure that the needs of the user are taken into account throughout the design process, while maintaining a rapid pace of development. In this paper, we present the methodology and its rationale before outlining how it was applied to assess and enhance the usability, human factors, and user experience of a connected health system known as the Wireless Insole for Independent and Safe Elderly Living (WIISEL) system, a system designed to continuously assess fall risk by measuring gait and balance parameters associated with fall risk. We derived a three-phase methodology. In Phase 1 we emphasized the construction of a use case document. This document can be used to detail the context of use of the system by utilizing storyboarding, paper prototypes, and mock-ups in conjunction with user interviews to gather insightful user feedback on different proposed concepts. In Phase 2 we emphasized the use of expert usability inspections such as heuristic evaluations and cognitive walkthroughs with small multidisciplinary groups to review the prototypes born out of the Phase 1 feedback. Finally, in Phase 3 we emphasized classical user testing with target end users, using various metrics to measure the user experience and improve the final prototypes. We report a successful implementation of the methodology for the design and development of a system for detecting and predicting falls in older adults. We describe in detail what testing and evaluation activities we carried out to effectively test the system and overcome usability and human factors problems. We feel this methodology can be applied to a wide variety of connected health devices and systems. We consider this a methodology that can be scaled to different-sized projects accordingly. ©Richard Harte, Liam Glynn, Alejandro Rodríguez-Molinero, Paul MA Baker, Thomas Scharf, Leo R Quinlan, Gearóid ÓLaighin. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 16.03.2017.

Usability Evaluation of NLP-PIER: A Clinical Document Search Engine for Researchers.

PubMed

Hultman, Gretchen; McEwan, Reed; Pakhomov, Serguei; Lindemann, Elizabeth; Skube, Steven; Melton, Genevieve B

2017-01-01

NLP-PIER (Natural Language Processing - Patient Information Extraction for Research) is a self-service platform with a search engine for clinical researchers to perform natural language processing (NLP) queries using clinical notes. We conducted user-centered testing of NLP-PIER's usability to inform future design decisions. Quantitative and qualitative data were analyzed. Our findings will be used to improve the usability of NLP-PIER.

User ratings of prosthetic usability and satisfaction in VA study to optimize DEKA arm.

PubMed

Resnik, Linda; Borgia, Matthew

2014-01-01

The Department of Veterans Affairs study to optimize the DEKA Arm provided feedback to inform optimization of the gen 2 (second-generation) prototype and evaluate the gen 3 (third-generation) prototype. This article summarizes recommendations to improve gen 2 and reports satisfaction and usability ratings of gen 2 and gen 3. Data were collected from 39 subjects; 37 subjects were included in this analysis. Of the subjects, 24 were fit with gen 2 (8 radial configuration [RC], 6 humeral configuration [HC], and 10 shoulder configuration [SC]), 13 were fit with gen 3 (4 RC, 5 HC, and 4 SC), and 5 were fit with both. Usability and satisfaction were evaluated using the Trinity Amputation and Prosthesis Experience Scale (TAPES) and study-specific usability and satisfaction scales. Descriptive statistics were examined and prototypes compared using Wilcoxon rank-sum. Results were stratified by configuration level and outcomes compared by prototype. Satisfaction and usability were greater for gen 3 than gen 2. Overall TAPES scores were similar; however, scores of the TAPES aesthetic satisfaction subscale were higher for gen 3. Compared with gen 2 users, gen 3 users were more satisfied with appearance, grips, and doffing and rated overall usability higher. Features of gen 3, including weight, external cables and wires, hand covering, and fingernails, would benefit from further optimization.

Usability study of clinical exome analysis software: top lessons learned and recommendations.

PubMed

Shyr, Casper; Kushniruk, Andre; Wasserman, Wyeth W

2014-10-01

New DNA sequencing technologies have revolutionized the search for genetic disruptions. Targeted sequencing of all protein coding regions of the genome, called exome analysis, is actively used in research-oriented genetics clinics, with the transition to exomes as a standard procedure underway. This transition is challenging; identification of potentially causal mutation(s) amongst ∼10(6) variants requires specialized computation in combination with expert assessment. This study analyzes the usability of user interfaces for clinical exome analysis software. There are two study objectives: (1) To ascertain the key features of successful user interfaces for clinical exome analysis software based on the perspective of expert clinical geneticists, (2) To assess user-system interactions in order to reveal strengths and weaknesses of existing software, inform future design, and accelerate the clinical uptake of exome analysis. Surveys, interviews, and cognitive task analysis were performed for the assessment of two next-generation exome sequence analysis software packages. The subjects included ten clinical geneticists who interacted with the software packages using the "think aloud" method. Subjects' interactions with the software were recorded in their clinical office within an urban research and teaching hospital. All major user interface events (from the user interactions with the packages) were time-stamped and annotated with coding categories to identify usability issues in order to characterize desired features and deficiencies in the user experience. We detected 193 usability issues, the majority of which concern interface layout and navigation, and the resolution of reports. Our study highlights gaps in specific software features typical within exome analysis. The clinicians perform best when the flow of the system is structured into well-defined yet customizable layers for incorporation within the clinical workflow. The results highlight opportunities to dramatically accelerate clinician analysis and interpretation of patient genomic data. We present the first application of usability methods to evaluate software interfaces in the context of exome analysis. Our results highlight how the study of user responses can lead to identification of usability issues and challenges and reveal software reengineering opportunities for improving clinical next-generation sequencing analysis. While the evaluation focused on two distinctive software tools, the results are general and should inform active and future software development for genome analysis software. As large-scale genome analysis becomes increasingly common in healthcare, it is critical that efficient and effective software interfaces are provided to accelerate clinical adoption of the technology. Implications for improved design of such applications are discussed. Copyright © 2014 The Authors. Published by Elsevier Inc. All rights reserved.

A four stage approach for ontology-based health information system design.

PubMed

Kuziemsky, Craig E; Lau, Francis

2010-11-01

To describe and illustrate a four stage methodological approach to capture user knowledge in a biomedical domain area, use that knowledge to design an ontology, and then implement and evaluate the ontology as a health information system (HIS). A hybrid participatory design-grounded theory (GT-PD) method was used to obtain data and code them for ontology development. Prototyping was used to implement the ontology as a computer-based tool. Usability testing evaluated the computer-based tool. An empirically derived domain ontology and set of three problem-solving approaches were developed as a formalized model of the concepts and categories from the GT coding. The ontology and problem-solving approaches were used to design and implement a HIS that tested favorably in usability testing. The four stage approach illustrated in this paper is useful for designing and implementing an ontology as the basis for a HIS. The approach extends existing ontology development methodologies by providing an empirical basis for theory incorporated into ontology design. Copyright © 2010 Elsevier B.V. All rights reserved.

Usability evaluation of a web-based support system for people with a schizophrenia diagnosis.

PubMed

van der Krieke, Lian; Emerencia, Ando C; Aiello, Marco; Sytema, Sjoerd

2012-02-06

Routine Outcome Monitoring (ROM) is a systematic way of assessing service users' health conditions for the purpose of better aiding their care. ROM consists of various measures used to assess a service user's physical, psychological, and social condition. While ROM is becoming increasingly important in the mental health care sector, one of its weaknesses is that ROM is not always sufficiently service user-oriented. First, clinicians tend to concentrate on those ROM results that provide information about clinical symptoms and functioning, whereas it has been suggested that a service user-oriented approach needs to focus on personal recovery. Second, service users have limited access to ROM results and they are often not equipped to interpret them. These problems need to be addressed, as access to resources and the opportunity to share decision making has been indicated as a prerequisite for service users to become a more equal partner in communication with their clinicians. Furthermore, shared decision making has been shown to improve the therapeutic alliance and to lead to better care. Our aim is to build a web-based support system which makes ROM results more accessible to service users and to provide them with more concrete and personalized information about their functioning (ie, symptoms, housing, social contacts) that they can use to discuss treatment options with their clinician. In this study, we will report on the usability of the web-based support system for service users with schizophrenia. First, we developed a prototype of a web-based support system in a multidisciplinary project team, including end-users. We then conducted a usability study of the support system consisting of (1) a heuristic evaluation, (2) a qualitative evaluation and (3) a quantitative evaluation. Fifteen service users with a schizophrenia diagnosis and four information and communication technology (ICT) experts participated in the study. The results show that people with a schizophrenia diagnosis were able to use the support system easily. Furthermore, the content of the advice generated by the support system was considered meaningful and supportive. This study shows that the support system prototype has valuable potential to improve the ROM practice and it is worthwhile to further develop it into a more mature system. Furthermore, the results add to prior research into web applications for people with psychotic disorders, in that it shows that this group of end users can work with web-based and computer-based systems, despite the cognitive problems they experience.

Do Perceptions of Insulin Pump Usability Impact Attitudes Toward Insulin Pump Therapy? A Pilot Study of Individuals With Type 1 and Insulin-Treated Type 2 Diabetes

PubMed Central

Gilgen, Emily

2014-01-01

Background: We assessed the impact of perceived insulin pump usability on attitudes toward insulin pump therapy in diabetic individuals currently treated with multiple daily insulin injections (MDI). Method: This comparative, single-arm study recruited 28 adults with type 1 (n = 16) and insulin-treated type 2 diabetes (n = 12) to evaluate 2 current insulin pumps: Medtronic Revel 723 (Pump 1), Asante Snap Insulin Pump (Pump 2). Participants were randomized 1:1 to 1 of 2 assessment sequences: Pump 1 followed by Pump 2; and Pump 2 followed by Pump 1. Structured observational protocols were utilized to assess participants’ ability and time required to learn/perform common tasks associated with pump setup/use. Participants used a modified version of the System Usability Scale (SUS) and investigator-developed questionnaires to rate pump usability and task difficulty; pre-post questionnaires assessed changes in attitudes toward insulin pump therapy. Results: All participants completed the study. SUS scores showed Pump 2 to be more usable than Pump 1 on all usability attributes. Participants rated Pump 2 more positively than Pump 1, overall mean SUS scores of 5.7 versus 4.1 respectively, F(1, 52) = 32.7, P < .001, and SUS scores were higher if participants used the Pump 2 last, 5.3 versus 4.4 for Pump 1 last, F(1, 52) = 10.8, P < .01. Pump 2 was preferred for all tasks: manual bolus (86%), bolus calculation (71%), managing basal rates (93%), interpreting alarms (96%), transferring settings (100%), changing insulin and infusion sets (93%), all P < .05. Conclusions: Perceptions of pump usability can directly impact acceptance and use of features that may benefit those who wear them. Simpler pump devices that decrease perceptions of complexity may encourage broader use of this technology. PMID:25269659

Design and evaluation of a software prototype for participatory planning of environmental adaptations.

PubMed

Eriksson, J; Ek, A; Johansson, G

2000-03-01

A software prototype to support the planning process for adapting home and work environments for people with physical disabilities was designed and later evaluated. The prototype exploits low-cost three-dimensional (3-D) graphics products in the home computer market. The essential features of the prototype are: interactive rendering with optional hardware acceleration, interactive walk-throughs, direct manipulation tools for moving objects and measuring distances, and import of 3-D-objects from a library. A usability study was conducted, consisting of two test sessions (three weeks apart) and a final interview. The prototype was then tested and evaluated by representatives of future users: five occupational therapist students, and four persons with physical disability, with no previous experience of the prototype. Emphasis in the usability study was placed on the prototype's efficiency and learnability. We found that it is possible to realise a planning tool for environmental adaptations, both regarding usability and technical efficiency. The usability evaluation confirms our findings from previous case studies, regarding the relevance and positive attitude towards this kind of planning tool. Although the prototype was found to be satisfactorily efficient for the basic tasks, the paper presents several suggestions for improvement of future prototype versions.

Evaluation and implementation of QR Code Identity Tag system for Healthcare in Turkey.

PubMed

Uzun, Vassilya; Bilgin, Sami

2016-01-01

For this study, we designed a QR Code Identity Tag system to integrate into the Turkish healthcare system. This system provides QR code-based medical identification alerts and an in-hospital patient identification system. Every member of the medical system is assigned a unique QR Code Tag; to facilitate medical identification alerts, the QR Code Identity Tag can be worn as a bracelet or necklace or carried as an ID card. Patients must always possess the QR Code Identity bracelets within hospital grounds. These QR code bracelets link to the QR Code Identity website, where detailed information is stored; a smartphone or standalone QR code scanner can be used to scan the code. The design of this system allows authorized personnel (e.g., paramedics, firefighters, or police) to access more detailed patient information than the average smartphone user: emergency service professionals are authorized to access patient medical histories to improve the accuracy of medical treatment. In Istanbul, we tested the self-designed system with 174 participants. To analyze the QR Code Identity Tag system's usability, the participants completed the System Usability Scale questionnaire after using the system.

Videoconferencing for site initiations in clinical studies: Mixed methods evaluation of usability, acceptability, and impact on recruitment.

PubMed

Randell, Rebecca; Backhouse, Michael R; Nelson, E Andrea

2016-12-01

A critical issue for multicentre clinical studies is conducting site initiations, ensuring sites are trained in study procedures and comply with relevant governance requirements before they begin recruiting patients. How technology can support site initiations has not previously been explored. This study sought to evaluate use of off-the-shelf web-based videoconferencing to deliver site initiations for a large national multicentre study. Participants in the initiations, including podiatrists, diabetologists, trial coordinators, and research nurses, completed an online questionnaire based on the System Usability Scale (SUS) (N = 15). This was followed by semi-structured interviews, with a consultant diabetologist, a trial coordinator, and three research nurses, exploring perceived benefits and limitations of videoconferencing. The mean SUS score for the videoconferencing platform was 87.2 (SD = 13.7), suggesting a good level of usability. Interview participants perceived initiations delivered by videoconferencing as being more interactive and easier to follow than those delivered by teleconference. In comparison to face-to-face initiations, videoconferencing takes less time, easily fitting in with the work of staff at the local sites. Perceptions of impact on communication varied according to the hardware used. Off-the-shelf videoconferencing is a viable alternative to face-to-face site initiations and confers advantages over teleconferencing.

User-friendly technology to help family carers cope.

PubMed

Chambers, Mary; Connor, Samantha L

2002-12-01

Increases in the older adult population are occurring simultaneously with a growth in new technology. Modern technology presents an opportunity to enhance the quality of life and independence of older people and their family carers through communication and access to health care information. To evaluate the usability of a multimedia software application designed to provide family carers of the elderly or disabled with information, advice and psychological support to increase their coping capacity. The interactive application consisted of an information-based package that provided carers with advice on the promotion of psychological health, including relaxation and other coping strategies. The software application also included a carer self-assessment instrument, designed to provide both family and professional carers with information to assess how family carers were coping with their care-giving role. Usability evaluation was carried out in two stages. In the first stage (verification), user trials and an evaluation questionnaire were used to refine and develop the content and usability of the multimedia software application. In the second (demonstration), stage evaluation questionnaires were used to appraise the usability of the modified software application. The findings evidenced that the majority of users found the software to be usable and informative. Some areas were highlighted for improvement in the navigation of the software. The authors conclude that with further refinement, the software application has the potential to offer information and support to those who are caring for the elderly and disabled at home.

Heuristic Evaluation and Usability Testing of a Computerized Patient-Reported Outcomes Survey for Headache Sufferers

PubMed Central

Saris-Baglama, Renee N.; Smith, Kevin J.; DeRosa, Michael A.; Paulsen, Christine A.; Hogue, Sarah J.

2011-01-01

Abstract Objective The aim of this study was to evaluate usability of a prototype tablet PC-administered computerized adaptive test (CAT) of headache impact and patient feedback report, referred to as HEADACHE-CAT. Materials and Methods Heuristic evaluation specialists (n = 2) formed a consensus opinion on the application's strengths and areas for improvement based on general usability principles and human factors research. Usability testing involved structured interviews with headache sufferers (n = 9) to assess how they interacted with and navigated through the application, and to gather input on the survey and report interface, content, visual design, navigation, instructions, and user preferences. Results Specialists identified the need for improved instructions and text formatting, increased font size, page setup that avoids scrolling, and simplified presentation of feedback reports. Participants found the tool useful, and indicated a willingness to complete it again and recommend it to their healthcare provider. However, some had difficulty using the onscreen keyboard and autoadvance option; understanding the difference between generic and headache-specific questions; and interpreting score reports. Conclusions Heuristic evaluation and user testing can help identify usability problems in the early stages of application development, and improve the construct validity of electronic assessments such as the HEADACHE-CAT. An improved computerized HEADACHE-CAT measure can offer headache sufferers an efficient tool to increase patient self-awareness, monitor headaches over time, aid patient–provider communications, and improve quality of life. PMID:21214341

Pervasive assistive technology for people with dementia: a UCD case

PubMed Central

Rønn-Andersen, Kristoffer V.H.; Bień, Paulina; Özkil, Ali Gürcan; Forchhammer, Birgitte Hysse; Maier, Anja M.

2016-01-01

Smart mobile and wearable technology offers exciting opportunities to support people with dementia (PwD). Its ubiquity and popularity could even benefit user adoption – a great challenge for assistive technology (AT) for PwD that calls for user-centred design (UCD) methods. This study describes a user-centred approach to developing and testing AT based on off-the-shelf pervasive technologies. A prototype is created by combining a smartphone, smartwatch and various applications to offer six support features. This is tested among five end-users (PwD) and their caregivers. Controlled usability testing was followed by field testing in a real-world context. Data is gathered from video recordings, interaction logs, system usability scale questionnaires, logbooks, application usage logs and interviews structured on the unified theory of acceptance and use of technology model. The data is analysed to evaluate usability, usefulness and user acceptance. Results show some promise for user adoption, but highlight challenges to be overcome, emphasising personalisation and familiarity as key considerations. The complete findings regarding usability issues, usefulness of support features and four identified adoption profiles are used to provide a set of recommendations for practitioners and further research. These contribute toward UCD practices for improved smart, pervasive AT for dementia. PMID:28008366

From Paper to PDA: Design and Evaluation of a Clinical Ward Instruction on a Mobile Device

NASA Astrophysics Data System (ADS)

Kanstrup, Anne Marie; Stage, Jan

Mobile devices with small screens and minimal facilities for interaction are increasingly being used in complex human activities for accessing and processing information, while the user is moving. This paper presents a case study of the design and evaluation of a mobile system, which involved transformation of complex text and tables to digital format on a PDA. The application domain was an emergency medical ward, and the user group was junior registrars. We designed a PDA-based system for accessing information, focusing on the ward instruction, implemented a prototype and evaluated it for usability and utility. The evaluation results indicate significant problems in the interaction with the system as well as the extent to which the system is useful for junior registrars in their daily work.

Surface operations usability study utilizing Capstone phase I avionics : quick look report

DOT National Transportation Integrated Search

2000-10-07

Evaluate usability, suitability and acceptability of of the surface moving map implemented within Capstone Phase 1 Avionics for surface operations : Task 1: Airport Surface Situational Awareness (ASSA) : Task 2: Surface-Final Approach Runway Occupanc...

Telemonitoring of patients with Parkinson's disease using inertia sensors.

PubMed

Piro, N E; Baumann, L; Tengler, M; Piro, L; Blechschmidt-Trapp, R

2014-01-01

Medical treatment in patients suffering from Parkinson's disease is very difficult as dose-finding is mainly based on selective and subjective impressions by the physician. To allow for the objective evaluation of patients' symptoms required for optimal dosefinding, a telemonitoring system tracks the motion of patients in their surroundings. The system focuses on providing interoperability and usability in order to ensure high acceptance. Patients wear inertia sensors and perform standardized motor tasks. Data are recorded, processed and then presented to the physician in a 3D animated form. In addition, the same data is rated based on the UPDRS score. Interoperability is realized by developing the system in compliance with the recommendations of the Continua Health Alliance. Detailed requirements analysis and continuous collaboration with respective user groups help to achieve high usability. A sensor platform was developed that is capable of measuring acceleration and angular rate of motions as well as the absolute orientation of the device itself through an included compass sensor. The system architecture was designed and required infrastructure, and essential parts of the communication between the system components were implemented following Continua guidelines. Moreover, preliminary data analysis based on three-dimensional acceleration and angular rate data could be established. A prototype system for the telemonitoring of Parkinson's disease patients was successfully developed. The developed sensor platform fully satisfies the needs of monitoring patients of Parkinson's disease and is comparable to other sensor platforms, although these sensor platforms have yet to be tested rigorously against each other. Suitable approaches to provide interoperability and usability were identified and realized and remain to be tested in the field.

Improving brain injury cognitive rehabilitation by personalized telerehabilitation services: Guttmann neuropersonal trainer.

PubMed

Solana, Javier; Cáceres, César; García-Molina, Alberto; Opisso, Eloy; Roig, Teresa; Tormos, José M; Gómez, Enrique J

2015-01-01

Cognitive rehabilitation aims to remediate or alleviate the cognitive deficits appearing after an episode of acquired brain injury (ABI). The purpose of this work is to describe the telerehabilitation platform called Guttmann Neuropersonal Trainer (GNPT) which provides new strategies for cognitive rehabilitation, improving efficiency and access to treatments, and to increase knowledge generation from the process. A cognitive rehabilitation process has been modeled to design and develop the system, which allows neuropsychologists to configure and schedule rehabilitation sessions, consisting of set of personalized computerized cognitive exercises grounded on neuroscience and plasticity principles. It provides remote continuous monitoring of patient's performance, by an asynchronous communication strategy. An automatic knowledge extraction method has been used to implement a decision support system, improving treatment customization. GNPT has been implemented in 27 rehabilitation centers and in 83 patients' homes, facilitating the access to the treatment. In total, 1660 patients have been treated. Usability and cost analysis methodologies have been applied to measure the efficiency in real clinical environments. The usability evaluation reveals a system usability score higher than 70 for all target users. The cost efficiency study results show a relation of 1-20 compared to face-to-face rehabilitation. GNPT enables brain-damaged patients to continue and further extend rehabilitation beyond the hospital, improving the efficiency of the rehabilitation process. It allows customized therapeutic plans, providing information to further development of clinical practice guidelines.

Evidence-based development and first usability testing of a social serious game based multi-modal system for early screening for atypical socio-cognitive development.

PubMed

Gyori, Miklos; Borsos, Zsófia; Stefanik, Krisztina

2015-01-01

At current, screening for, and diagnosis of, autism spectrum disorders (ASD) are based on purely behavioral data; established screening tools rely on human observation and ratings of relevant behaviors. The research and development project in the focus of this paper is aimed at designing, creating and evaluating a social serious game based multi-modal, interactive software system for screening for high functioning cases of ASD at kindergarten age. The aims of this paper are (1) to summarize the evidence-based design process and (2) to present results from the first usability test of the system. Game topic, candidate responses, and candidate game contents were identified via an iterative literature review. On this basis, the 1^st partial prototype of the fully playable game has been created, with complete data recording functionality but without the decision making component. A first usability test was carried out on this prototype (n=13). Overall results were unambiguously promising. Although sporadic difficulties in, and slightly negative attitudes towards, using the game occasionally arose, these were confined to non-target-group children only. The next steps of development include (1) completing the game design; (2) carrying out first large-n field test; (3) creating the first prototype of the decision making component.

Better bioinformatics through usability analysis.

PubMed

Bolchini, Davide; Finkelstein, Anthony; Perrone, Vito; Nagl, Sylvia

2009-02-01

Improving the usability of bioinformatics resources enables researchers to find, interact with, share, compare and manipulate important information more effectively and efficiently. It thus enables researchers to gain improved insights into biological processes with the potential, ultimately, of yielding new scientific results. Usability 'barriers' can pose significant obstacles to a satisfactory user experience and force researchers to spend unnecessary time and effort to complete their tasks. The number of online biological databases available is growing and there is an expanding community of diverse users. In this context there is an increasing need to ensure the highest standards of usability. Using 'state-of-the-art' usability evaluation methods, we have identified and characterized a sample of usability issues potentially relevant to web bioinformatics resources, in general. These specifically concern the design of the navigation and search mechanisms available to the user. The usability issues we have discovered in our substantial case studies are undermining the ability of users to find the information they need in their daily research activities. In addition to characterizing these issues, specific recommendations for improvements are proposed leveraging proven practices from web and usability engineering. The methods and approach we exemplify can be readily adopted by the developers of bioinformatics resources.

Ergonomic Evaluation of the Foot Restraint Equipment Device (FRED)

NASA Technical Reports Server (NTRS)

Whitmore, Mihriban; Chmielewski, Cindy; Qazi, A. S.; Mount, Francis

1999-01-01

Within the scope of the Microgravity Workstation and Restraint Evaluation project, funded by the NASA Headquarters Life Sciences Division, evaluations were proposed to be conducted in ground, KC-135, and/or Shuttle environments to investigate the human factors engineering (HFE) issues concerning confined/unique workstations, including crew restraint requirements. As part of these evaluations, KC-135 flights were conducted to investigate user/ workstation/ restraint integration for microgravity use of the FRED with the RMS workstation. This evaluation was a pre-cursor to Detailed Supplementary Objective (DSO) - 904 on STS-88. On that mission, a small-statured astronaut will be using the FRED restraint while working at the Aft RMS workstation. The DSO will collect video for later posture analyses, as well as subjective data in the form of an electronic questionnaire. This report describes the current FRED KC-135 evaluations. The primary objectives were to evaluate the usability of the FRED and to verify the DSO in-flight setup. The restraint interface evaluation consisted of four basic areas of restraint use: 1) adjustability; 2) general usability and comfort; 3) usability at the RMS workstation; and 4) assembly and disassembly.

Engaging Community Stakeholders to Evaluate the Design, Usability, and Acceptability of a Chronic Obstructive Pulmonary Disease Social Media Resource Center

PubMed Central

Chaney, Beth; Chaney, Don; Paige, Samantha; Payne-Purvis, Caroline; Tennant, Bethany; Walsh-Childers, Kim; Sriram, PS; Alber, Julia

2015-01-01

Background Patients with chronic obstructive pulmonary disease (COPD) often report inadequate access to comprehensive patient education resources. Objective The purpose of this study was to incorporate community-engagement principles within a mixed-method research design to evaluate the usability and acceptability of a self-tailored social media resource center for medically underserved patients with COPD. Methods A multiphase sequential design (qual → QUANT → quant + QUAL) was incorporated into the current study, whereby a small-scale qualitative (qual) study informed the design of a social media website prototype that was tested with patients during a computer-based usability study (QUANT). To identify usability violations and determine whether or not patients found the website prototype acceptable for use, each patient was asked to complete an 18-item website usability and acceptability questionnaire, as well as a retrospective, in-depth, semistructured interview (quant + QUAL). Results The majority of medically underserved patients with COPD (n=8, mean 56 years, SD 7) found the social media website prototype to be easy to navigate and relevant to their self-management information needs. Mean responses on the 18-item website usability and acceptability questionnaire were very high on a scale of 1 (strongly disagree) to 5 (strongly agree) (mean 4.72, SD 0.33). However, the majority of patients identified several usability violations related to the prototype’s information design, interactive capabilities, and navigational structure. Specifically, 6 out of 8 (75%) patients struggled to create a log-in account to access the prototype, and 7 out of 8 patients (88%) experienced difficulty posting and replying to comments on an interactive discussion forum. Conclusions Patient perceptions of most social media website prototype features (eg, clickable picture-based screenshots of videos, comment tools) were largely positive. Mixed-method stakeholder feedback was used to make design recommendations, categorize usability violations, and prioritize potential solutions for improving the usability of a social media resource center for COPD patient education. PMID:25630449

"Think aloud" and "Near live" usability testing of two complex clinical decision support tools.

PubMed

Richardson, Safiya; Mishuris, Rebecca; O'Connell, Alexander; Feldstein, David; Hess, Rachel; Smith, Paul; McCullagh, Lauren; McGinn, Thomas; Mann, Devin

2017-10-01

Low provider adoption continues to be a significant barrier to realizing the potential of clinical decision support. "Think Aloud" and "Near Live" usability testing were conducted on two clinical decision support tools. Each was composed of an alert, a clinical prediction rule which estimated risk of either group A Streptococcus pharyngitis or pneumonia and an automatic order set based on risk. The objective of this study was to further understanding of the facilitators of usability and to evaluate the types of additional information gained from proceeding to "Near Live" testing after completing "Think Aloud". This was a qualitative observational study conducted at a large academic health care system with 12 primary care providers. During "Think Aloud" testing, participants were provided with written clinical scenarios and asked to verbalize their thought process while interacting with the tool. During "Near Live" testing participants interacted with a mock patient. Morae usability software was used to record full screen capture and audio during every session. Participant comments were placed into coding categories and analyzed for generalizable themes. Themes were compared across usability methods. "Think Aloud" and "Near Live" usability testing generated similar themes under the coding categories visibility, workflow, content, understand-ability and navigation. However, they generated significantly different themes under the coding categories usability, practical usefulness and medical usefulness. During both types of testing participants found the tool easier to use when important text was distinct in its appearance, alerts were passive and appropriately timed, content was up to date, language was clear and simple, and each component of the tool included obvious indicators of next steps. Participant comments reflected higher expectations for usability and usefulness during "Near Live" testing. For example, visit aids, such as automatically generated order sets, were felt to be less useful during "Near-Live" testing because they would not be all inclusive for the visit. These complementary types of usability testing generated unique and generalizable insights. Feedback during "Think Aloud" testing primarily helped to improve the tools' ease of use. The additional feedback from "Near Live" testing, which mimics a real clinical encounter, was helpful for eliciting key barriers and facilitators to provider workflow and adoption. Copyright © 2017 Elsevier B.V. All rights reserved.

Grappling with Distributed Usability: A Cultural-Historical Examination of Documentation Genres Over Four Decades.

ERIC Educational Resources Information Center

Spinuzzi, Clay

2001-01-01

Describes and illustrates a distributed approach to usability (envisioning usability across the genres, practices, uses, and goals of a given activity) using a four-decade examination of a traffic accident location and analysis system. Uses the theoretical framework of "genre ecologies" to show how communities of users interact with…

Improving access to clinical practice guidelines with an interactive graphical interface using an iconic language

PubMed Central

2014-01-01

Background Clinical practice guidelines are useful for physicians, and guidelines are available on the Internet from various websites such as Vidal Recos. However, these guidelines are long and difficult to read, especially during consultation. Similar difficulties have been encountered with drug summaries of product characteristics. In a previous work, we have proposed an iconic language (called VCM, for Visualization of Concepts in Medicine) for representing patient conditions, treatments and laboratory tests, and we have used these icons to design a user interface that graphically indexes summaries of product characteristics. In the current study, our objective was to design and evaluate an iconic user interface for the consultation of clinical practice guidelines by physicians. Methods Focus groups of physicians were set up to identify the difficulties encountered when reading guidelines. Icons were integrated into Vidal Recos, taking human factors into account. The resulting interface includes a graphical summary and an iconic indexation of the guideline. The new interface was evaluated. We compared the response times and the number of errors recorded when physicians answered questions about two clinical scenarios using the interactive iconic interface or a textual interface. Users’ perceived usability was evaluated with the System Usability Scale. Results The main difficulties encountered by physicians when reading guidelines were obtaining an overview and finding recommendations for patients corresponding to “particular cases”. We designed a graphical interface for guideline consultation, using icons to identify particular cases and providing a graphical summary of the icons organized by anatomy and etiology. The evaluation showed that physicians gave clinical responses more rapidly with the iconic interface than the textual interface (25.2 seconds versus 45.6, p < 0.05). The physicians appreciated the new interface, and the System Usability Scale score value was 75 (between good and excellent). Conclusion An interactive iconic interface can provide physicians with an overview of clinical practice guidelines, and can decrease the time required to access the content of such guidelines. PMID:25158762

Improving access to clinical practice guidelines with an interactive graphical interface using an iconic language.

PubMed

Pereira, Suzanne; Hassler, Sylvain; Hamek, Saliha; Boog, César; Leroy, Nicolas; Beuscart-Zéphir, Marie-Catherine; Favre, Madeleine; Venot, Alain; Duclos, Catherine; Lamy, Jean-Baptiste

2014-08-26

Clinical practice guidelines are useful for physicians, and guidelines are available on the Internet from various websites such as Vidal Recos. However, these guidelines are long and difficult to read, especially during consultation. Similar difficulties have been encountered with drug summaries of product characteristics. In a previous work, we have proposed an iconic language (called VCM, for Visualization of Concepts in Medicine) for representing patient conditions, treatments and laboratory tests, and we have used these icons to design a user interface that graphically indexes summaries of product characteristics. In the current study, our objective was to design and evaluate an iconic user interface for the consultation of clinical practice guidelines by physicians. Focus groups of physicians were set up to identify the difficulties encountered when reading guidelines. Icons were integrated into Vidal Recos, taking human factors into account. The resulting interface includes a graphical summary and an iconic indexation of the guideline. The new interface was evaluated. We compared the response times and the number of errors recorded when physicians answered questions about two clinical scenarios using the interactive iconic interface or a textual interface. Users' perceived usability was evaluated with the System Usability Scale. The main difficulties encountered by physicians when reading guidelines were obtaining an overview and finding recommendations for patients corresponding to "particular cases". We designed a graphical interface for guideline consultation, using icons to identify particular cases and providing a graphical summary of the icons organized by anatomy and etiology. The evaluation showed that physicians gave clinical responses more rapidly with the iconic interface than the textual interface (25.2 seconds versus 45.6, p < 0.05). The physicians appreciated the new interface, and the System Usability Scale score value was 75 (between good and excellent). An interactive iconic interface can provide physicians with an overview of clinical practice guidelines, and can decrease the time required to access the content of such guidelines.

The Use of an Adapted Health IT Usability Evaluation Model (Health-ITUEM) for Evaluating Consumer Reported Ratings of Diabetes mHealth Applications: Implications for Diabetes Care and Management

PubMed Central

Househ, Mowafa S.; Shubair, Mamdouh M.; Yunus, Faisel; Jamal, Amr; Aldossari, Bakheet

2015-01-01

Background: The aim of this paper is to present a usability analysis of the consumer ratings of key diabetes mHealth applications using an adapted Health IT Usability Evaluation Model (Health-ITUEM). Methods: A qualitative content analysis method was used to analyze publicly available consumer reported data posted on the Android Market and Google Play for four leading diabetes mHealth applications. Health-ITUEM concepts including information needs, flexibility/customizability, learnability, performance speed, and competency guided the categorization and analysis of the data. Health impact was an additional category that was included in the study. A total of 405 consumers’ ratings collected from January 9, 2014 to February 17, 2014 were included in the study. Results: Overall, the consumers’ ratings of the leading diabetes mHealth applications for both usability and health impacts were positive. The performance speed of the mHealth application and the information needs of the consumers were the primary usability factors impacting the use of the diabetes mHealth applications. There was also evidence on the positive health impacts of such applications. Conclusions: Consumers are more likely to use diabetes related mHealth applications that perform well and meet their information needs. Furthermore, there is preliminary evidence that diabetes mHealth applications can have positive impact on the health of patients. PMID:26635437

The Use of an Adapted Health IT Usability Evaluation Model (Health-ITUEM) for Evaluating Consumer Reported Ratings of Diabetes mHealth Applications: Implications for Diabetes Care and Management.

PubMed

Househ, Mowafa S; Shubair, Mamdouh M; Yunus, Faisel; Jamal, Amr; Aldossari, Bakheet

2015-10-01

The aim of this paper is to present a usability analysis of the consumer ratings of key diabetes mHealth applications using an adapted Health IT Usability Evaluation Model (Health-ITUEM). A qualitative content analysis method was used to analyze publicly available consumer reported data posted on the Android Market and Google Play for four leading diabetes mHealth applications. Health-ITUEM concepts including information needs, flexibility/customizability, learnability, performance speed, and competency guided the categorization and analysis of the data. Health impact was an additional category that was included in the study. A total of 405 consumers' ratings collected from January 9, 2014 to February 17, 2014 were included in the study. Overall, the consumers' ratings of the leading diabetes mHealth applications for both usability and health impacts were positive. The performance speed of the mHealth application and the information needs of the consumers were the primary usability factors impacting the use of the diabetes mHealth applications. There was also evidence on the positive health impacts of such applications. Consumers are more likely to use diabetes related mHealth applications that perform well and meet their information needs. Furthermore, there is preliminary evidence that diabetes mHealth applications can have positive impact on the health of patients.

A usability evaluation of medical software at an expert conference setting.

PubMed

Bond, Raymond Robert; Finlay, Dewar D; Nugent, Chris D; Moore, George; Guldenring, Daniel

2014-01-01

A usability test was employed to evaluate two medical software applications at an expert conference setting. One software application is a medical diagnostic tool (electrocardiogram [ECG] viewer) and the other is a medical research tool (electrode misplacement simulator [EMS]). These novel applications have yet to be adopted by the healthcare domain, thus, (1) we wanted to determine the potential user acceptance of these applications and (2) we wanted to determine the feasibility of evaluating medical diagnostic and medical research software at a conference setting as opposed to the conventional laboratory setting. The medical diagnostic tool (ECG viewer) was evaluated using seven delegates and the medical research tool (EMS) was evaluated using 17 delegates that were recruited at the 2010 International Conference on Computing in Cardiology. Each delegate/participant was required to use the software and undertake a set of predefined tasks during the session breaks at the conference. User interactions with the software were recorded using screen-recording software. The 'think-aloud' protocol was also used to elicit verbal feedback from the participants whilst they attempted the pre-defined tasks. Before and after each session, participants completed a pre-test and a post-test questionnaire respectively. The average duration of a usability session at the conference was 34.69 min (SD=10.28). However, taking into account that 10 min was dedicated to the pre-test and post-test questionnaires, the average time dedication to user interaction of the medical software was 24.69 min (SD=10.28). Given we have shown that usability data can be collected at conferences, this paper details the advantages of conference-based usability studies over the laboratory-based approach. For example, given delegates gather at one geographical location, a conference-based usability evaluation facilitates recruitment of a convenient sample of international subject experts. This would otherwise be very expensive to arrange. A conference-based approach also allows for data to be collected over a few days as opposed to months by avoiding administration duties normally involved in laboratory based approach, e.g. mailing invitation letters as part of a recruitment campaign. Following analysis of the user video recordings, 41 (previously unknown) use errors were identified in the advanced ECG viewer and 29 were identified in the EMS application. All use errors were given a consensus severity rating from two independent usability experts. Out of a rating scale of 4 (where 1=cosmetic and 4=critical), the average severity rating for the ECG viewer was 2.24 (SD=1.09) and the average severity rating for the EMS application was 2.34 (SD=0.97). We were also able to extract task completion rates and times from the video recordings to determine the effectiveness of the software applications. For example, six out of seven tasks were completed by all participants when using both applications. This statistic alone suggests both applications already have a high degree of usability. As well as extracting data from the video recordings, we were also able to extract data from the questionnaires. Using a semantic differential scale (where 1=poor and 5=excellent), delegates highly rated the 'responsiveness', 'usefulness', 'learnability' and the 'look and feel' of both applications. This study has shown the potential user acceptance and user-friendliness of the novel EMS and the ECG viewer applications within the healthcare domain. It has also shown that both medical diagnostic software and medical research software can be evaluated for their usability at an expert conference setting. The primary advantage of a conference-based usability evaluation over a laboratory-based evaluation is the high concentration of experts at one location, which is convenient, less time consuming and less expensive. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Applying usability testing techniques to improve a health promotion website.

PubMed

Hinchliffe, Anetta; Mummery, W Kerry

2008-04-01

Use of the Internet for health promotion is increasing; however, the lack of published research regarding website usability suggests that health promotion websites are being developed without consultation with their users or formal evaluation. This study conducted usability testing of an existing health promotion website to inform modifications and to identify common usability themes that should be addressed by organisations developing or maintaining a health promotion website. A combination of qualitative and quantitative techniques were implemented during the usability testing sessions to gather data from users while completing tasks on the website. Techniques included performance measures (time taken), direct observation (participant observation) and subjective user preferences (questionnaire and interview). Improvements to the website were measured in terms of reduced problems reported, reduced time taken to complete tasks and increased subjective reports. Seven usability themes emerged from the data: design, feedback, format, instructions, navigation, terminology and learnability. This study demonstrates the application of usability testing to the design and modification of a health promotion website and illustrates the areas or themes that can be used as a framework for testing and modification.

A web-based program for informal caregivers of persons with Alzheimer's disease: an iterative user-centered design.

PubMed

Cristancho-Lacroix, Victoria; Moulin, Florence; Wrobel, Jérémy; Batrancourt, Bénédicte; Plichart, Matthieu; De Rotrou, Jocelyne; Cantegreil-Kallen, Inge; Rigaud, Anne-Sophie

2014-09-15

Web-based programs have been developed for informal caregivers of people with Alzheimer's disease (PWAD). However, these programs can prove difficult to adopt, especially for older people, who are less familiar with the Internet than other populations. Despite the fundamental role of usability testing in promoting caregivers' correct use and adoption of these programs, to our knowledge, this is the first study describing this process before evaluating a program for caregivers of PWAD in a randomized clinical trial. The objective of the study was to describe the development process of a fully automated Web-based program for caregivers of PWAD, aiming to reduce caregivers' stress, and based on the user-centered design approach. There were 49 participants (12 health care professionals, 6 caregivers, and 31 healthy older adults) that were involved in a double iterative design allowing for the adaptation of program content and for the enhancement of website usability. This process included three component parts: (1) project team workshops, (2) a proof of concept, and (3) two usability tests. The usability tests were based on a mixed methodology using behavioral analysis, semistructured interviews, and a usability questionnaire. The user-centered design approach provided valuable guidelines to adapt the content and design of the program, and to improve website usability. The professionals, caregivers (mainly spouses), and older adults considered that our project met the needs of isolated caregivers. Participants underlined that contact between caregivers would be desirable. During usability observations, the mistakes of users were also due to ergonomics issues from Internet browsers and computer interfaces. Moreover, negative self-stereotyping was evidenced, when comparing interviews and results of behavioral analysis. Face-to-face psycho-educational programs may be used as a basis for Web-based programs. Nevertheless, a user-centered design approach involving targeted users (or their representatives) remains crucial for their correct use and adoption. For future user-centered design studies, we recommend to involve end-users from preconception stages, using a mixed research method in usability evaluations, and implementing pilot studies to evaluate acceptability and feasibility of programs.

The effect of user's perceived presence and promotion focus on usability for interacting in virtual environments.

PubMed

Sun, Huey-Min; Li, Shang-Phone; Zhu, Yu-Qian; Hsiao, Bo

2015-09-01

Technological advance in human-computer interaction has attracted increasing research attention, especially in the field of virtual reality (VR). Prior research has focused on examining the effects of VR on various outcomes, for example, learning and health. However, which factors affect the final outcomes? That is, what kind of VR system design will achieve higher usability? This question remains largely. Furthermore, when we look at VR system deployment from a human-computer interaction (HCI) lens, does user's attitude play a role in achieving the final outcome? This study aims to understand the effect of immersion and involvement, as well as users' regulatory focus on usability for a somatosensory VR learning system. This study hypothesized that regulatory focus and presence can effectively enhance user's perceived usability. Survey data from 78 students in Taiwan indicated that promotion focus is positively related to user's perceived efficiency, whereas involvement and promotion focus are positively related to user's perceived effectiveness. Promotion focus also predicts user satisfaction and overall usability perception. Copyright © 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.

An Assessment-for-Learning System in Mathematics for Individuals with Visual Impairments

ERIC Educational Resources Information Center

Hansen, Eric G.; Shute, Valerie J.; Landau, Steven

2010-01-01

This study examined the usability of an assessment-for-learning (AfL) system that provides audio-tactile graphics for algebra content (geometric sequences) for individuals with visual impairments--two who are blind and two with low vision. It found that the system is generally usable as a mathematics AfL system. (Contains 4 tables.)

The EU-project United4Health: User-centred design of an information system for a Norwegian telemedicine service.

PubMed

Smaradottir, Berglind; Gerdes, Martin; Martinez, Santiago; Fensli, Rune

2016-10-01

Organizational changes of health care services in Norway brought to light a need for new clinical pathways. This study presents the design and evaluation of an information system for a new telemedicine service for chronic obstructive pulmonary disease patients after hospital discharge. A user-centred design approach was employed composed of a workshop with end-users, two user tests and a field trial. For data collection, qualitative methods such as observations, semi-structured interviews and a questionnaire were used. User workshop's outcome informed the implementation of the system initial prototype, evaluated by end-users in a usability laboratory. Several usability and functionality issues were identified and solved, such as the interface between the initial colour scheme and the triage colours. Iterative refinements were made and a second user evaluation showed that the main issues were solved. The responses to a questionnaire presented a high score of user satisfaction. In the final phase, a field trial showed satisfactory use of the system. This study showed how the target end-users groups were actively involved in identifying the needs, suggestions and preferences. These aspects were addressed in the development of an information system through a user-centred design process. The process efficiently enabled users to give feedback about design and functionality. Continuous refinement of the system was the key to full development and suitability for the telemedicine service. This research was a result of the international cooperation between partners within the project United4Health, a part of the Seventh Framework Programme for Research of the European Union. © The Author(s) 2015.