Physical activity and healthy eating environmental audit tools in youth care settings: A systematic review.

PubMed

Ajja, Rahma; Beets, Michael W; Chandler, Jessica; Kaczynski, Andrew T; Ward, Dianne S

2015-08-01

There is a growing interest in evaluating the physical activity (PA) and healthy eating (HE) policy and practice environment characteristics in settings frequented by youth (≤18years). This review evaluates the measurement properties of audit tools designed to assess PA and HE policy and practice environment characteristics in settings that care for youth (e.g., childcare, school, afterschool, summer camp). Three electronic databases, reference lists, educational department and national health organizations' web pages were searched between January 1980 and February 2014 to identify tools assessing PA and/or HE policy and practice environments in settings that care for youth (≤18years). Sixty-five audit tools were identified of which 53 individual tools met the inclusion criteria. Thirty-three tools assessed both the PA and HE domains, 6 assessed the PA domain and 14 assessed the HE domain solely. The majority of the tools were self-assessment tools (n=40), and were developed to assess the PA and/or HE environment in school settings (n=33), childcare (n=12), and after school programs (n=4). Four tools assessed the community at-large and had sections for assessing preschool, school and/or afterschool settings within the tool. The majority of audit tools lacked validity and/or reliability data (n=42). Inter-rater reliability and construct validity were the most frequently reported reliability (n=7) and validity types (n=5). Limited attention has been given to establishing the reliability and validity of audit tools for settings that care for youth. Future efforts should be directed towards establishing a strong measurement foundation for these important environmental audit tools. Published by Elsevier Inc.

Content Validation and Evaluation of an Endovascular Teamwork Assessment Tool.

PubMed

Hull, L; Bicknell, C; Patel, K; Vyas, R; Van Herzeele, I; Sevdalis, N; Rudarakanchana, N

2016-07-01

To modify, content validate, and evaluate a teamwork assessment tool for use in endovascular surgery. A multistage, multimethod study was conducted. Stage 1 included expert review and modification of the existing Observational Teamwork Assessment for Surgery (OTAS) tool. Stage 2 included identification of additional exemplar behaviours contributing to effective teamwork and enhanced patient safety in endovascular surgery (using real-time observation, focus groups, and semistructured interviews of multidisciplinary teams). Stage 3 included content validation of exemplar behaviours using expert consensus according to established psychometric recommendations and evaluation of structure, content, feasibility, and usability of the Endovascular Observational Teamwork Assessment Tool (Endo-OTAS) by an expert multidisciplinary panel. Stage 4 included final team expert review of exemplars. OTAS core team behaviours were maintained (communication, coordination, cooperation, leadership team monitoring). Of the 114 OTAS behavioural exemplars, 19 were modified, four removed, and 39 additional endovascular-specific behaviours identified. Content validation of these 153 exemplar behaviours showed that 113/153 (73.9%) reached the predetermined Item-Content Validity Index rating for teamwork and/or patient safety. After expert team review, 140/153 (91.5%) exemplars were deemed to warrant inclusion in the tool. More than 90% of the expert panel agreed that Endo-OTAS is an appropriate teamwork assessment tool with observable behaviours. Some concerns were noted about the time required to conduct observations and provide performance feedback. Endo-OTAS is a novel teamwork assessment tool, with evidence for content validity and relevance to endovascular teams. Endo-OTAS enables systematic objective assessment of the quality of team performance during endovascular procedures. Copyright © 2016. Published by Elsevier Ltd.

Competency-Based Training and Simulation: Making a "Valid" Argument.

PubMed

Noureldin, Yasser A; Lee, Jason Y; McDougall, Elspeth M; Sweet, Robert M

2018-02-01

The use of simulation as an assessment tool is much more controversial than is its utility as an educational tool. However, without valid simulation-based assessment tools, the ability to objectively assess technical skill competencies in a competency-based medical education framework will remain challenging. The current literature in urologic simulation-based training and assessment uses a definition and framework of validity that is now outdated. This is probably due to the absence of awareness rather than an absence of comprehension. The following review article provides the urologic community an updated taxonomy on validity theory as it relates to simulation-based training and assessments and translates our simulation literature to date into this framework. While the old taxonomy considered validity as distinct subcategories and focused on the simulator itself, the modern taxonomy, for which we translate the literature evidence, considers validity as a unitary construct with a focus on interpretation of simulator data/scores.

Validity of the Consensual Assessment Technique--Evidence with Three Groups of Judges and an Elementary School Student Sample

ERIC Educational Resources Information Center

Long, Haiying

2012-01-01

As one of the most widely used creativity assessment tools, the Consensual Assessment Technique (CAT) has been praised as a valid tool to assess creativity. In Amabile's (1982) seminal work, the inter-rater reliability was defined as construct validity of the CAT. During the past three decades, researchers followed this definition and…

Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts.

PubMed

Löfmark, Anna; Mårtensson, Gunilla

2017-03-01

The aim of the present study was to establish the validity of the tool Assessment of Clinical Education (AssCE). The tool is widely used in Sweden and some Nordic countries for assessing nursing students' performance in clinical education. It is important that the tools in use be subjected to regular audit and critical reviews. The validation process, performed in two stages, was concluded with a high level of congruence. In the first stage, Delphi technique was used to elaborate the AssCE tool using a group of 35 clinical nurse lecturers. After three rounds, we reached consensus. In the second stage, a group of 46 clinical nurse lecturers representing 12 universities in Sweden and Norway audited the revised version of the AssCE in relation to learning outcomes from the last clinical course at their respective institutions. Validation of the revised AssCE was established with high congruence between the factors in the AssCE and examined learning outcomes. The revised AssCE tool seems to meet its objective to be a validated assessment tool for use in clinical nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

The validity of a professional competence tool for physiotherapy students in simulation-based clinical education: a Rasch analysis.

PubMed

Judd, Belinda K; Scanlan, Justin N; Alison, Jennifer A; Waters, Donna; Gordon, Christopher J

2016-08-05

Despite the recent widespread adoption of simulation in clinical education in physiotherapy, there is a lack of validated tools for assessment in this setting. The Assessment of Physiotherapy Practice (APP) is a comprehensive tool used in clinical placement settings in Australia to measure professional competence of physiotherapy students. The aim of the study was to evaluate the validity of the APP for student assessment in simulation settings. A total of 1260 APPs were collected, 971 from students in simulation and 289 from students in clinical placements. Rasch analysis was used to examine the construct validity of the APP tool in three different simulation assessment formats: longitudinal assessment over 1 week of simulation; longitudinal assessment over 2 weeks; and a short-form (25 min) assessment of a single simulation scenario. Comparison with APPs from 5 week clinical placements in hospital and clinic-based settings were also conducted. The APP demonstrated acceptable fit to the expectations of the Rasch model for the 1 and 2 week clinical simulations, exhibiting unidimensional properties that were able to distinguish different levels of student performance. For the short-form simulation, nine of the 20 items recorded greater than 25 % of scores as 'not-assessed' by clinical educators which impacted on the suitability of the APP tool in this simulation format. The APP was a valid assessment tool when used in longitudinal simulation formats. A revised APP may be required for assessment in short-form simulation scenarios.

Validation of the self-assessment teamwork tool (SATT) in a cohort of nursing and medical students.

PubMed

Roper, Lucinda; Shulruf, Boaz; Jorm, Christine; Currie, Jane; Gordon, Christopher J

2018-02-09

Poor teamwork has been implicated in medical error and teamwork training has been shown to improve patient care. Simulation is an effective educational method for teamwork training. Post-simulation reflection aims to promote learning and we have previously developed a self-assessment teamwork tool (SATT) for health students to measure teamwork performance. This study aimed to evaluate the psychometric properties of a revised self-assessment teamwork tool. The tool was tested in 257 medical and nursing students after their participation in one of several mass casualty simulations. Using exploratory and confirmatory factor analysis, the revised self-assessment teamwork tool was shown to have strong construct validity, high reliability, and the construct demonstrated invariance across groups (Medicine & Nursing). The modified SATT was shown to be a reliable and valid student self-assessment tool. The SATT is a quick and practical method of guiding students' reflection on important teamwork skills.

Development and psychometric testing of Holistic Clinical Assessment Tool (HCAT) for undergraduate nursing students.

PubMed

Wu, Xi Vivien; Enskär, Karin; Pua, Lay Hoon; Heng, Doreen Gek Noi; Wang, Wenru

2016-09-22

A major focus in nursing education is on the judgement of clinical performance, and it is a complex process due to the diverse nature of nursing practice. A holistic approach in assessment of competency is advocated. Difficulties in the development of valid and reliable assessment measures in nursing competency have resulted in the development of assessment instruments with an increase in face and content validity, but few studies have tested these instruments psychometrically. It is essential to develop a holistic assessment tool to meet the needs of the clinical education. The study aims to develop a Holistic Clinical Assessment Tool (HCAT) and test its psychometric properties. The HCAT was developed based on the systematic literature review and the findings of qualitative studies. An expert panel was invited to evaluate the content validity of the tool. A total of 130 final-year nursing undergraduate students were recruited to evaluate the psychometric properties (i.e. factor structure, internal consistency and test-retest reliability) of the tool. The HCAT has good content validity with content validity index of .979. The exploratory factor analysis reveals a four-factor structure of the tool. The internal consistency and test-retest reliability of the HCAT are satisfactory with Cronbach alpha ranging from .789 to .965 and Intraclass Correlation Coefficient ranging from .881 to .979 for the four subscales and total scale. HCAT has the potential to be used as a valid measure to evaluate clinical competence in nursing students, and provide specific and ongoing feedback to enhance the holistic clinical learning experience. In addition, HCAT functions as a tool for self-reflection, peer-assessment and guides preceptors in clinical teaching and assessment.

Development and Validity Testing of an Arthritis Self-Management Assessment Tool.

PubMed

Oh, HyunSoo; Han, SunYoung; Kim, SooHyun; Seo, WhaSook

Because of the chronic, progressive nature of arthritis and the substantial effects it has on quality of life, patients may benefit from self-management. However, no valid, reliable self-management assessment tool has been devised for patients with arthritis. This study was conducted to develop a comprehensive self-management assessment tool for patients with arthritis, that is, the Arthritis Self-Management Assessment Tool (ASMAT). To develop a list of qualified items corresponding to the conceptual definitions and attributes of arthritis self-management, a measurement model was established on the basis of theoretical and empirical foundations. Content validity testing was conducted to evaluate whether listed items were suitable for assessing arthritis self-management. Construct validity and reliability of the ASMAT were tested. Construct validity was examined using confirmatory factor analysis and nomological validity. The 32-item ASMAT was developed with a sample composed of patients in a clinic in South Korea. Content validity testing validated the 32 items, which comprised medical (10 items), behavioral (13 items), and psychoemotional (9 items) management subscales. Construct validity testing of the ASMAT showed that the 32 items properly corresponded with conceptual constructs of arthritis self-management, and were suitable for assessing self-management ability in patients with arthritis. Reliability was also well supported. The ASMAT devised in the present study may aid the evaluation of patient self-management ability and the effectiveness of self-management interventions. The authors believe the developed tool may also aid the identification of problems associated with the adoption of self-management practice, and thus improve symptom management, independence, and quality of life of patients with arthritis.

Design of capability measurement instruments pedagogic content knowledge (PCK) for prospective mathematics teachers

NASA Astrophysics Data System (ADS)

Aminah, N.; Wahyuni, I.

2018-05-01

The purpose of this study is to find out how the process of designing a tool of measurement Pedagogical Content Knowledge (PCK) capabilities, especially for prospective mathematics teachers are valid and practical. The design study of this measurement appliance uses modified Plomp development step, which consists of (1) initial assessment stage, (2) design stage at this stage, the researcher designs the measuring grille of PCK capability, (3) realization stage that is making measurement tool ability of PCK, (4) test phase, evaluation, and revision that is testing validation of measurement tools conducted by experts. Based on the results obtained that the design of PCK capability measurement tool is valid as indicated by the assessment of expert validator, and the design of PCK capability measurement tool, shown based on the assessment of teachers and lecturers as users of states strongly agree the design of PCK measurement tools can be used.

Clinical instruments: reliability and validity critical appraisal.

PubMed

Brink, Yolandi; Louw, Quinette A

2012-12-01

RATIONALE, AIM AND OBJECTIVES: There is a lack of health care practitioners using objective clinical tools with sound psychometric properties. There is also a need for researchers to improve their reporting of the validity and reliability results of these clinical tools. Therefore, to promote the use of valid and reliable tools or tests for clinical evaluation, this paper reports on the development of a critical appraisal tool to assess the psychometric properties of objective clinical tools. A five-step process was followed to develop the new critical appraisal tool: (1) preliminary conceptual decisions; (2) defining key concepts; (3) item generation; (4) assessment of face validity; and (5) formulation of the final tool. The new critical appraisal tool consists of 13 items, of which five items relate to both validity and reliability studies, four items to validity studies only and four items to reliability studies. The 13 items could be scored as 'yes', 'no' or 'not applicable'. This critical appraisal tool will aid both the health care practitioner to critically appraise the relevant literature and researchers to improve the quality of reporting of the validity and reliability of objective clinical tools. © 2011 Blackwell Publishing Ltd.

Development and validation of a risk assessment tool for gastric cancer in a general Japanese population.

PubMed

Iida, Masahiro; Ikeda, Fumie; Hata, Jun; Hirakawa, Yoichiro; Ohara, Tomoyuki; Mukai, Naoko; Yoshida, Daigo; Yonemoto, Koji; Esaki, Motohiro; Kitazono, Takanari; Kiyohara, Yutaka; Ninomiya, Toshiharu

2018-05-01

There have been very few reports of risk score models for the development of gastric cancer. The aim of this study was to develop and validate a risk assessment tool for discerning future gastric cancer risk in Japanese. A total of 2444 subjects aged 40 years or over were followed up for 14 years from 1988 (derivation cohort), and 3204 subjects of the same age group were followed up for 5 years from 2002 (validation cohort). The weighting (risk score) of each risk factor for predicting future gastric cancer in the risk assessment tool was determined based on the coefficients of a Cox proportional hazards model in the derivation cohort. The goodness of fit of the established risk assessment tool was assessed using the c-statistic and the Hosmer-Lemeshow test in the validation cohort. During the follow-up, gastric cancer developed in 90 subjects in the derivation cohort and 35 subjects in the validation cohort. In the derivation cohort, the risk prediction model for gastric cancer was established using significant risk factors: age, sex, the combination of Helicobacter pylori antibody and pepsinogen status, hemoglobin A1c level, and smoking status. The incidence of gastric cancer increased significantly as the sum of risk scores increased (P trend < 0.001). The risk assessment tool was validated internally and showed good discrimination (c-statistic = 0.76) and calibration (Hosmer-Lemeshow test P = 0.43) in the validation cohort. We developed a risk assessment tool for gastric cancer that provides a useful guide for stratifying an individual's risk of future gastric cancer.

Developing and Validating a New Classroom Climate Observation Assessment Tool

PubMed Central

Leff, Stephen S.; Thomas, Duane E.; Shapiro, Edward S.; Paskewich, Brooke; Wilson, Kim; Necowitz-Hoffman, Beth; Jawad, Abbas F.

2011-01-01

The climate of school classrooms, shaped by a combination of teacher practices and peer processes, is an important determinant for children’s psychosocial functioning and is a primary factor affecting bullying and victimization. Given that there are relatively few theoretically-grounded and validated assessment tools designed to measure the social climate of classrooms, our research team developed an observation tool through participatory action research (PAR). This article details how the assessment tool was designed and preliminarily validated in 18 third-, fourth-, and fifth-grade classrooms in a large urban public school district. The goals of this study are to illustrate the feasibility of a PAR paradigm in measurement development, ascertain the psychometric properties of the assessment tool, and determine associations with different indices of classroom levels of relational and physical aggression. PMID:21643447

Developing and Validating a New Classroom Climate Observation Assessment Tool.

PubMed

Leff, Stephen S; Thomas, Duane E; Shapiro, Edward S; Paskewich, Brooke; Wilson, Kim; Necowitz-Hoffman, Beth; Jawad, Abbas F

2011-01-01

The climate of school classrooms, shaped by a combination of teacher practices and peer processes, is an important determinant for children's psychosocial functioning and is a primary factor affecting bullying and victimization. Given that there are relatively few theoretically-grounded and validated assessment tools designed to measure the social climate of classrooms, our research team developed an observation tool through participatory action research (PAR). This article details how the assessment tool was designed and preliminarily validated in 18 third-, fourth-, and fifth-grade classrooms in a large urban public school district. The goals of this study are to illustrate the feasibility of a PAR paradigm in measurement development, ascertain the psychometric properties of the assessment tool, and determine associations with different indices of classroom levels of relational and physical aggression.

Different type 2 diabetes risk assessments predict dissimilar numbers at ‘high risk’: a retrospective analysis of diabetes risk-assessment tools

PubMed Central

Gray, Benjamin J; Bracken, Richard M; Turner, Daniel; Morgan, Kerry; Thomas, Michael; Williams, Sally P; Williams, Meurig; Rice, Sam; Stephens, Jeffrey W

2015-01-01

Background Use of a validated risk-assessment tool to identify individuals at high risk of developing type 2 diabetes is currently recommended. It is under-reported, however, whether a different risk tool alters the predicted risk of an individual. Aim This study explored any differences between commonly used validated risk-assessment tools for type 2 diabetes. Design and setting Cross-sectional analysis of individuals who participated in a workplace-based risk assessment in Carmarthenshire, South Wales. Method Retrospective analysis of 676 individuals (389 females and 287 males) who participated in a workplace-based diabetes risk-assessment initiative. Ten-year risk of type 2 diabetes was predicted using the validated QDiabetes®, Leicester Risk Assessment (LRA), FINDRISC, and Cambridge Risk Score (CRS) algorithms. Results Differences between the risk-assessment tools were apparent following retrospective analysis of individuals. CRS categorised the highest proportion (13.6%) of individuals at ‘high risk’ followed by FINDRISC (6.6%), QDiabetes (6.1%), and, finally, the LRA was the most conservative risk tool (3.1%). Following further analysis by sex, over one-quarter of males were categorised at high risk using CRS (25.4%), whereas a greater percentage of females were categorised as high risk using FINDRISC (7.8%). Conclusion The adoption of a different valid risk-assessment tool can alter the predicted risk of an individual and caution should be used to identify those individuals who really are at high risk of type 2 diabetes. PMID:26541180

Different type 2 diabetes risk assessments predict dissimilar numbers at 'high risk': a retrospective analysis of diabetes risk-assessment tools.

PubMed

Gray, Benjamin J; Bracken, Richard M; Turner, Daniel; Morgan, Kerry; Thomas, Michael; Williams, Sally P; Williams, Meurig; Rice, Sam; Stephens, Jeffrey W

2015-12-01

Use of a validated risk-assessment tool to identify individuals at high risk of developing type 2 diabetes is currently recommended. It is under-reported, however, whether a different risk tool alters the predicted risk of an individual. This study explored any differences between commonly used validated risk-assessment tools for type 2 diabetes. Cross-sectional analysis of individuals who participated in a workplace-based risk assessment in Carmarthenshire, South Wales. Retrospective analysis of 676 individuals (389 females and 287 males) who participated in a workplace-based diabetes risk-assessment initiative. Ten-year risk of type 2 diabetes was predicted using the validated QDiabetes(®), Leicester Risk Assessment (LRA), FINDRISC, and Cambridge Risk Score (CRS) algorithms. Differences between the risk-assessment tools were apparent following retrospective analysis of individuals. CRS categorised the highest proportion (13.6%) of individuals at 'high risk' followed by FINDRISC (6.6%), QDiabetes (6.1%), and, finally, the LRA was the most conservative risk tool (3.1%). Following further analysis by sex, over one-quarter of males were categorised at high risk using CRS (25.4%), whereas a greater percentage of females were categorised as high risk using FINDRISC (7.8%). The adoption of a different valid risk-assessment tool can alter the predicted risk of an individual and caution should be used to identify those individuals who really are at high risk of type 2 diabetes. © British Journal of General Practice 2015.

Psychometric and cognitive validation of a social capital measurement tool in Peru and Vietnam.

PubMed

De Silva, Mary J; Harpham, Trudy; Tuan, Tran; Bartolini, Rosario; Penny, Mary E; Huttly, Sharon R

2006-02-01

Social capital is a relatively new concept which has attracted significant attention in recent years. No consensus has yet been reached on how to measure social capital, resulting in a large number of different tools available. While psychometric validation methods such as factor analysis have been used by a few studies to assess the internal validity of some tools, these techniques rely on data already collected by the tool and are therefore not capable of eliciting what the questions are actually measuring. The Young Lives (YL) study includes quantitative measures of caregiver's social capital in four countries (Vietnam, Peru, Ethiopia, and India) using a short version of the Adapted Social Capital Assessment Tool (SASCAT). A range of different psychometric methods including factor analysis were used to evaluate the construct validity of SASCAT in Peru and Vietnam. In addition, qualitative cognitive interviews with 20 respondents from Peru and 24 respondents from Vietnam were conducted to explore what each question is actually measuring. We argue that psychometric validation techniques alone are not sufficient to adequately validate multi-faceted social capital tools for use in different cultural settings. Psychometric techniques show SASCAT to be a valid tool reflecting known constructs and displaying postulated links with other variables. However, results from the cognitive interviews present a more mixed picture with some questions being appropriately interpreted by respondents, and others displaying significant differences between what the researchers intended them to measure and what they actually do. Using evidence from a range of methods of assessing validity has enabled the modification of an existing instrument into a valid and low cost tool designed to measure social capital within larger surveys in Peru and Vietnam, with the potential for use in other developing countries following local piloting and cultural adaptation of the tool.

Older adult mistreatment risk screening: contribution to the validation of a screening tool in a domestic setting.

PubMed

Lindenbach, Jeannette M; Larocque, Sylvie; Lavoie, Anne-Marise; Garceau, Marie-Luce

2012-06-01

ABSTRACTThe hidden nature of older adult mistreatment renders its detection in the domestic setting particularly challenging. A validated screening instrument that can provide a systematic assessment of risk factors can facilitate this detection. One such instrument, the "expanded Indicators of Abuse" tool, has been previously validated in the Hebrew language in a hospital setting. The present study has contributed to the validation of the "e-IOA" in an English-speaking community setting in Ontario, Canada. It consisted of two phases: (a) a content validity review and adaptation of the instrument by experts throughout Ontario, and (b) an inter-rater reliability assessment by home visiting nurses. The adaptation, the "Mistreatment of Older Adult Risk Factors" tool, offers a comprehensive tool for screening in the home setting. This instrument is significant to professional practice as practitioners working with older adults will be better equipped to assess for risk of mistreatment.

Pain assessment for people with dementia: a systematic review of systematic reviews of pain assessment tools.

PubMed

Lichtner, Valentina; Dowding, Dawn; Esterhuizen, Philip; Closs, S José; Long, Andrew F; Corbett, Anne; Briggs, Michelle

2014-12-17

There is evidence of under-detection and poor management of pain in patients with dementia, in both long-term and acute care. Accurate assessment of pain in people with dementia is challenging and pain assessment tools have received considerable attention over the years, with an increasing number of tools made available. Systematic reviews on the evidence of their validity and utility mostly compare different sets of tools. This review of systematic reviews analyses and summarises evidence concerning the psychometric properties and clinical utility of pain assessment tools in adults with dementia or cognitive impairment. We searched for systematic reviews of pain assessment tools providing evidence of reliability, validity and clinical utility. Two reviewers independently assessed each review and extracted data from them, with a third reviewer mediating when consensus was not reached. Analysis of the data was carried out collaboratively. The reviews were synthesised using a narrative synthesis approach. We retrieved 441 potentially eligible reviews, 23 met the criteria for inclusion and 8 provided data for extraction. Each review evaluated between 8 and 13 tools, in aggregate providing evidence on a total of 28 tools. The quality of the reviews varied and the reporting often lacked sufficient methodological detail for quality assessment. The 28 tools appear to have been studied in a variety of settings and with varied types of patients. The reviews identified several methodological limitations across the original studies. The lack of a 'gold standard' significantly hinders the evaluation of tools' validity. Most importantly, the samples were small providing limited evidence for use of any of the tools across settings or populations. There are a considerable number of pain assessment tools available for use with the elderly cognitive impaired population. However there is limited evidence about their reliability, validity and clinical utility. On the basis of this review no one tool can be recommended given the existing evidence.

A Systematic Review of Tools Used to Assess Team Leadership in Health Care Action Teams.

PubMed

Rosenman, Elizabeth D; Ilgen, Jonathan S; Shandro, Jamie R; Harper, Amy L; Fernandez, Rosemarie

2015-10-01

To summarize the characteristics of tools used to assess leadership in health care action (HCA) teams. HCA teams are interdisciplinary teams performing complex, critical tasks under high-pressure conditions. The authors conducted a systematic review of the PubMed/MEDLINE, CINAHL, ERIC, EMBASE, PsycINFO, and Web of Science databases, key journals, and review articles published through March 2012 for English-language articles that applied leadership assessment tools to HCA teams in all specialties. Pairs of reviewers assessed identified articles for inclusion and exclusion criteria and abstracted data on study characteristics, tool characteristics, and validity evidence. Of the 9,913 abstracts screened, 83 studies were included. They described 61 team leadership assessment tools. Forty-nine tools (80%) provided behaviors, skills, or characteristics to define leadership. Forty-four tools (72%) assessed leadership as one component of a larger assessment, 13 tools (21%) identified leadership as the primary focus of the assessment, and 4 (7%) assessed leadership style. Fifty-three studies (64%) assessed leadership at the team level; 29 (35%) did so at the individual level. Assessments of simulated (n = 55) and live (n = 30) patient care events were performed. Validity evidence included content validity (n = 75), internal structure (n = 61), relationship to other variables (n = 44), and response process (n = 15). Leadership assessment tools applied to HCA teams are heterogeneous in content and application. Comparisons between tools are limited by study variability. A systematic approach to team leadership tool development, evaluation, and implementation will strengthen understanding of this important competency.

Assessing physiotherapists' communication skills for promoting patient autonomy for self-management: reliability and validity of the communication evaluation in rehabilitation tool.

PubMed

Murray, Aileen; Hall, Amanda; Williams, Geoffrey C; McDonough, Suzanne M; Ntoumanis, Nikos; Taylor, Ian; Jackson, Ben; Copsey, Bethan; Hurley, Deirdre A; Matthews, James

2018-02-27

To assess the inter-rater reliability and concurrent validity of the Communication Evaluation in Rehabilitation Tool, which aims to externally assess physiotherapists competency in using Self-Determination Theory-based communication strategies in practice. Audio recordings of initial consultations between 24 physiotherapists and 24 patients with chronic low back pain in four hospitals in Ireland were obtained as part of a larger randomised controlled trial. Three raters, all of whom had Ph.Ds in psychology and expertise in motivation and physical activity, independently listened to the 24 audio recordings and completed the 18-item Communication Evaluation in Rehabilitation Tool. Inter-rater reliability between all three raters was assessed using intraclass correlation coefficients. Concurrent validity was assessed using Pearson's r correlations with a reference standard, the Health Care Climate Questionnaire. The total score for the Communication Evaluation in Rehabilitation Tool is an average of all 18 items. Total scores demonstrated good inter-rater reliability (Intraclass Correlation Coefficient (ICC) = 0.8) and concurrent validity with the Health Care Climate Questionnaire total score (range: r = 0.7-0.88). Item-level scores of the Communication Evaluation in Rehabilitation Tool identified five items that need improvement. Results provide preliminary evidence to support future use and testing of the Communication Evaluation in Rehabilitation Tool. Implications for Rehabilitation Promoting patient autonomy is a learned skill and while interventions exist to train clinicians in these skills there are no tools to assess how well clinicians use these skills when interacting with a patient. The lack of robust assessment has severe implications regarding both the fidelity of clinician training packages and resulting outcomes for promoting patient autonomy. This study has developed a novel measurement tool Communication Evaluation in Rehabilitation Tool and a comprehensive user manual to assess how well health care providers use autonomy-supportive communication strategies in real world-clinical settings. This tool has demonstrated good inter-rater reliability and concurrent validity in its initial testing phase. The Communication Evaluation in Rehabilitation Tool can be used in future studies to assess autonomy-supportive communication and undergo further measurement property testing as per our recommendations.

The Development and Validation of the Contextualized Assessment Tool for Risk and Protection Management (CAT-RPM)

ERIC Educational Resources Information Center

Bower, Julie M.; Carroll, Annemaree; Ashman, Adrian

2014-01-01

Traditional resources for ascertaining risk and protection for disengaged youth are often unsuitable, due to the stamina and skill required to complete them. Many of these tools assess risk without considering participants' potential for personal growth. The present study outlines the development and initial validation of a tool titled the…

Teamwork Assessment Tools in Obstetric Emergencies: A Systematic Review.

PubMed

Onwochei, Desire N; Halpern, Stephen; Balki, Mrinalini

2017-06-01

Team-based training and simulation can improve patient safety, by improving communication, decision making, and performance of team members. Currently, there is no general consensus on whether or not a specific assessment tool is better adapted to evaluate teamwork in obstetric emergencies. The purpose of this qualitative systematic review was to find the tools available to assess team effectiveness in obstetric emergencies. We searched Embase, Medline, PubMed, Web of Science, PsycINFO, CINAHL, and Google Scholar for prospective studies that evaluated nontechnical skills in multidisciplinary teams involving obstetric emergencies. The search included studies from 1944 until January 11, 2016. Data on reliability and validity measures were collected and used for interpretation. A descriptive analysis was performed on the data. Thirteen studies were included in the final qualitative synthesis. All the studies assessed teams in the context of obstetric simulation scenarios, but only six included anesthetists in the simulations. One study evaluated their teamwork tool using just validity measures, five using just reliability measures, and one used both. The most reliable tools identified were the Clinical Teamwork Scale, the Global Assessment of Obstetric Team Performance, and the Global Rating Scale of performance. However, they were still lacking in terms of quality and validity. More work needs to be conducted to establish the validity of teamwork tools for nontechnical skills, and the development of an ideal tool is warranted. Further studies are required to assess how outcomes, such as performance and patient safety, are influenced when using these tools.

Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany.

PubMed

Dreier, Maren; Borutta, Birgit; Stahmeyer, Jona; Krauth, Christian; Walter, Ulla

2010-06-14

HEALTH CARE POLICY BACKGROUND: Findings from scientific studies form the basis for evidence-based health policy decisions. Quality assessments to evaluate the credibility of study results are an essential part of health technology assessment reports and systematic reviews. Quality assessment tools (QAT) for assessing the study quality examine to what extent study results are systematically distorted by confounding or bias (internal validity). The tools can be divided into checklists, scales and component ratings. What QAT are available to assess the quality of interventional studies or studies in the field of health economics, how do they differ from each other and what conclusions can be drawn from these results for quality assessments? A systematic search of relevant databases from 1988 onwards is done, supplemented by screening of the references, of the HTA reports of the German Agency for Health Technology Assessment (DAHTA) and an internet search. The selection of relevant literature, the data extraction and the quality assessment are carried out by two independent reviewers. The substantive elements of the QAT are extracted using a modified criteria list consisting of items and domains specific to randomized trials, observational studies, diagnostic studies, systematic reviews and health economic studies. Based on the number of covered items and domains, more and less comprehensive QAT are distinguished. In order to exchange experiences regarding problems in the practical application of tools, a workshop is hosted. A total of eight systematic methodological reviews is identified as well as 147 QAT: 15 for systematic reviews, 80 for randomized trials, 30 for observational studies, 17 for diagnostic studies and 22 for health economic studies. The tools vary considerably with regard to the content, the performance and quality of operationalisation. Some tools do not only include the items of internal validity but also the items of quality of reporting and external validity. No tool covers all elements or domains. Design-specific generic tools are presented, which cover most of the content criteria. The evaluation of QAT by using content criteria is difficult, because there is no scientific consensus on the necessary elements of internal validity, and not all of the generally accepted elements are based on empirical evidence. Comparing QAT with regard to contents neglects the operationalisation of the respective parameters, for which the quality and precision are important for transparency, replicability, the correct assessment and interrater reliability. QAT, which mix items on the quality of reporting and internal validity, should be avoided. There are different, design-specific tools available which can be preferred for quality assessment, because of its wider coverage of substantive elements of internal validity. To minimise the subjectivity of the assessment, tools with a detailed and precise operationalisation of the individual elements should be applied. For health economic studies, tools should be developed and complemented with instructions, which define the appropriateness of the criteria. Further research is needed to identify study characteristics that influence the internal validity of studies.

Above, Beyond, and Over the Side rails: Evaluating the New Memorial Emergency Department Fall-Risk-Assessment Tool.

PubMed

Scott, Robin A; Oman, Kathleen S; Flarity, Kathleen; Comer, Jennifer L

2018-03-06

Patient falls are a significant issue in hospitalized patients and financially costly to hospitals. The Joint Commission requires that patients be assessed for fall risk and interventions in place to mitigate the risk of falls. It is imperative to have a patient population/setting specific fall risk assessment tool to identify patients at risk for falling. The purpose of this study was to evaluate the reliability and validity of the 2013 Memorial ED Fall Risk Assessment tool (MEDFRAT) specifically designed for the ED population. A two-phase prospective design was used for this study. Phase one determined the interrater reliability of the MEDFRAT. Phase two assessed the validity of the MEDFRAT in an emergency department (ED) within a 600-bed academic/teaching institution; Level II Trauma Center with >100,000 annual patient visits. The Memorial ED Fall Risk Assessment Tool was validated in this ED setting. The tool demonstrated positive interrater reliability (k=0.701) and when implemented with a falls prevention strategy and staff education demonstrated a 48% decrease in ED fall rate (0.57 falls/1000 patient visits) post implementation during the study period. The MEDFRAT, an evidenced based ED-specific fall risk tool was implemented on the basis of the risk factors consistently identified in the literature: prior fall history, impaired mobility, altered mental status, altered elimination, and the use of sedative medication. The Memorial ED Fall Risk Assessment Tool demonstrated to be a valid tool for this hospital system. Copyright © 2018 Emergency Nurses Association. Published by Elsevier Inc. All rights reserved.

Validity and applicability of a video-based animated tool to assess mobility in elderly Latin American populations

PubMed Central

Guerra, Ricardo Oliveira; Oliveira, Bruna Silva; Alvarado, Beatriz Eugenia; Curcio, Carmen Lucia; Rejeski, W Jack; Marsh, Anthony P; Ip, Edward H; Barnard, Ryan T; Guralnik, Jack M; Zunzunegui, Maria Victoria

2016-01-01

Aim To assess the reliability and the validity of Portuguese- and Spanish-translated versions of the video-based short-form Mobility Assessment Tool in assessing self-reported mobility, and to provide evidence for the applicability of these videos in elderly Latin American populations as a complement to physical performance measures. Methods The sample consisted of 300 elderly participants (150 from Brazil, 150 from Colombia) recruited at neighborhood social centers. Mobility was assessed with the Mobility Assessment Tool, and compared with the Short Physical Performance Battery score and self-reported functional limitations. Reliability was calculated using intraclass correlation coefficients. Multiple linear regression analyses were used to assess associations among mobility assessment tools and health, and sociodemographic variables. Results A significant gradient of increasing Mobility Assessment Tool score with better physical function was observed for both self-reported and objective measures, and in each city. Associations between self-reported mobility and health were strong, and significant. Mobility Assessment Tool scores were lower in women at both sites. Intraclass correlation coefficients of the Mobility Assessment Tool were 0.94 (95% confidence interval 0.90–0.97) in Brazil and 0.81 (95% confidence interval 0.66–0.91) in Colombia. Mobility Assessment Tool scores were lower in Manizales than in Natal after adjustment by Short Physical Performance Battery, self-rated health and sex. Conclusions These results provide evidence for high reliability and good validity of the Mobility Assessment Tool in its Spanish and Portuguese versions used in Latin American populations. In addition, the Mobility Assessment Tool can detect mobility differences related to environmental features that cannot be captured by objective perfor mance measures. PMID:24666718

Validity and applicability of a video-based animated tool to assess mobility in elderly Latin American populations.

PubMed

Guerra, Ricardo Oliveira; Oliveira, Bruna Silva; Alvarado, Beatriz Eugenia; Curcio, Carmen Lucia; Rejeski, W Jack; Marsh, Anthony P; Ip, Edward H; Barnard, Ryan T; Guralnik, Jack M; Zunzunegui, Maria Victoria

2014-10-01

To assess the reliability and the validity of Portuguese- and Spanish-translated versions of the video-based short-form Mobility Assessment Tool in assessing self-reported mobility, and to provide evidence for the applicability of these videos in elderly Latin American populations as a complement to physical performance measures. The sample consisted of 300 elderly participants (150 from Brazil, 150 from Colombia) recruited at neighborhood social centers. Mobility was assessed with the Mobility Assessment Tool, and compared with the Short Physical Performance Battery score and self-reported functional limitations. Reliability was calculated using intraclass correlation coefficients. Multiple linear regression analyses were used to assess associations among mobility assessment tools and health, and sociodemographic variables. A significant gradient of increasing Mobility Assessment Tool score with better physical function was observed for both self-reported and objective measures, and in each city. Associations between self-reported mobility and health were strong, and significant. Mobility Assessment Tool scores were lower in women at both sites. Intraclass correlation coefficients of the Mobility Assessment Tool were 0.94 (95% confidence interval 0.90-0.97) in Brazil and 0.81 (95% confidence interval 0.66-0.91) in Colombia. Mobility Assessment Tool scores were lower in Manizales than in Natal after adjustment by Short Physical Performance Battery, self-rated health and sex. These results provide evidence for high reliability and good validity of the Mobility Assessment Tool in its Spanish and Portuguese versions used in Latin American populations. In addition, the Mobility Assessment Tool can detect mobility differences related to environmental features that cannot be captured by objective performance measures. © 2013 Japan Geriatrics Society.

Risk assessment tools to identify women with increased risk of osteoporotic fracture: complexity or simplicity? A systematic review.

PubMed

Rubin, Katrine Hass; Friis-Holmberg, Teresa; Hermann, Anne Pernille; Abrahamsen, Bo; Brixen, Kim

2013-08-01

A huge number of risk assessment tools have been developed. Far from all have been validated in external studies, more of them have absence of methodological and transparent evidence, and few are integrated in national guidelines. Therefore, we performed a systematic review to provide an overview of existing valid and reliable risk assessment tools for prediction of osteoporotic fractures. Additionally, we aimed to determine if the performance of each tool was sufficient for practical use, and last, to examine whether the complexity of the tools influenced their discriminative power. We searched PubMed, Embase, and Cochrane databases for papers and evaluated these with respect to methodological quality using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS) checklist. A total of 48 tools were identified; 20 had been externally validated, however, only six tools had been tested more than once in a population-based setting with acceptable methodological quality. None of the tools performed consistently better than the others and simple tools (i.e., the Osteoporosis Self-assessment Tool [OST], Osteoporosis Risk Assessment Instrument [ORAI], and Garvan Fracture Risk Calculator [Garvan]) often did as well or better than more complex tools (i.e., Simple Calculated Risk Estimation Score [SCORE], WHO Fracture Risk Assessment Tool [FRAX], and Qfracture). No studies determined the effectiveness of tools in selecting patients for therapy and thus improving fracture outcomes. High-quality studies in randomized design with population-based cohorts with different case mixes are needed. Copyright © 2013 American Society for Bone and Mineral Research.

Development of a Peer Teaching-Assessment Program and a Peer Observation and Evaluation Tool

PubMed Central

Trujillo, Jennifer M.; Barr, Judith; Gonyeau, Michael; Van Amburgh, Jenny A.; Matthews, S. James; Qualters, Donna

2008-01-01

Objectives To develop a formalized, comprehensive, peer-driven teaching assessment program and a valid and reliable assessment tool. Methods A volunteer taskforce was formed and a peer-assessment program was developed using a multistep, sequential approach and the Peer Observation and Evaluation Tool (POET). A pilot study was conducted to evaluate the efficiency and practicality of the process and to establish interrater reliability of the tool. Intra-class correlation coefficients (ICC) were calculated. Results ICCs for 8 separate lectures evaluated by 2-3 observers ranged from 0.66 to 0.97, indicating good interrater reliability of the tool. Conclusion Our peer assessment program for large classroom teaching, which includes a valid and reliable evaluation tool, is comprehensive, feasible, and can be adopted by other schools of pharmacy. PMID:19325963

Diagnostic flexible pharyngo-laryngoscopy: development of a procedure specific assessment tool using a Delphi methodology.

PubMed

Melchiors, Jacob; Henriksen, Mikael Johannes Vuokko; Dikkers, Frederik G; Gavilán, Javier; Noordzij, J Pieter; Fried, Marvin P; Novakovic, Daniel; Fagan, Johannes; Charabi, Birgitte W; Konge, Lars; von Buchwald, Christian

2018-05-01

Proper training and assessment of skill in flexible pharyngo-laryngoscopy are central in the education of otorhinolaryngologists. To facilitate an evidence-based approach to curriculum development in this field, a structured analysis of what constitutes flexible pharyngo-laryngoscopy is necessary. Our aim was to develop an assessment tool based on this analysis. We conducted an international Delphi study involving experts from twelve countries in five continents. Utilizing reiterative assessment, the panel defined the procedure and reached consensus (defined as 80% agreement) on the phrasing of an assessment tool. FIFTY PANELISTS COMPLETED THE DELPHI PROCESS. THE MEDIAN AGE OF THE PANELISTS WAS 44 YEARS (RANGE 33-64 YEARS). MEDIAN EXPERIENCE IN OTORHINOLARYNGOLOGY WAS 15 YEARS (RANGE 6-35 YEARS). TWENTY-FIVE WERE SPECIALIZED IN LARYNGOLOGY, 16 WERE HEAD AND NECK SURGEONS, AND NINE WERE GENERAL OTORHINOLARYNGOLOGISTS. AN ASSESSMENT TOOL WAS CREATED CONSISTING OF TWELVE DISTINCT ITEMS.: Conclusion The gathering of validity evidence for assessment of core procedural skills within Otorhinolaryngology is central to the development of a competence-based education. The use of an international Delphi panel allows for the creation of an assessment tool which is widely applicable and valid. This work allows for an informed approach to technical skills training for flexible pharyngo-laryngoscopy and as further validity evidence is gathered allows for a valid assessment of clinical performance within this important skillset.

Tool development to assess the work related neck and upper limb musculoskeletal disorders among female garment workers in Sri-Lanka.

PubMed

Amarasinghe, Nirmalie Champika; De AlwisSenevirathne, Rohini

2016-10-17

Musculoskeletal disorders (MSDs) have been identified as a predisposing factor for lesser productivity, but no validated tool has been developed to assess them in the Sri- Lankan context. To develop a validated tool to assess the neck and upper limb MSDs. It comprises three components: item selections, item reduction using principal component analysis, and validation. A tentative self-administrated questionnaire was developed, translated, and pre-tested. Four important domains - neck, shoulder, elbow and wrist - were identified through principal component analysis. Prevalence of any MSDs was 38.1% and prevalence of neck, shoulder, elbow and wrist MSDs are 12.85%, 13.71%, 12%, 13.71% respectively. Content and criterion validity of the tool was assessed. Separate ROC curves were produced and sensitivity and specificity of neck (83.1%, 71.7%), shoulder (97.6%, 91.9%), elbow (98.2%, 87.2%), and wrist (97.6%, 94.9%) was determined. Cronbach's Alpha and correlation coefficient was above 0.7. The tool has high sensitivity, specificity, internal consistency, and test re-test reliability.

Nutrition Environment Food Pantry Assessment Tool (NEFPAT): Development and Evaluation.

PubMed

Nikolaus, Cassandra J; Laurent, Emily; Loehmer, Emily; An, Ruopeng; Khan, Naiman; McCaffrey, Jennifer

2018-04-24

To develop and evaluate a nutrition environment assessment tool to assess the consumer nutrition environment and use of recommended practices in food pantries. The Nutrition Environment Food Pantry Assessment Tool (NEFPAT) was developed based on a literature review and guidance from professionals working with food pantries. The tool was pilot-tested at 9 food pantries, an expert panel assessed content validity, and interrater reliability was evaluated by pairs in 3 pantries. After revisions, the NEFPAT was used in 27 pantries. Pilot tests indicated positive appraisal for the NEFPAT and recommendations were addressed. The NEFPAT's 6 objectives and the overall tool were rated as content valid by experts, with an average section rating of 3.85 ± 0.10. Intraclass correlation coefficients for interrater reliability were >0.90. The NEFPAT is content valid with high interrater reliability. It provides baseline data that could be valuable for interventions within the nutrition environment of food pantries. Published by Elsevier Inc.

Consequential Validity and the Transformation of Tests from Measurement Tools to Policy Tools

ERIC Educational Resources Information Center

Welner, Kevin G.

2013-01-01

Background/Context: Recent U.S. policy has brought a shift in assessment use, from measurement tools to policy levers. In particular, testing has become a core part of teacher evaluation policies in many states, with test results becoming akin to a job evaluation. Purpose: To explore the notion of consequential validity in assessment use and…

Systematic review of methods for quantifying teamwork in the operating theatre

PubMed Central

Marshall, D.; Sykes, M.; McCulloch, P.; Shalhoub, J.; Maruthappu, M.

2018-01-01

Background Teamwork in the operating theatre is becoming increasingly recognized as a major factor in clinical outcomes. Many tools have been developed to measure teamwork. Most fall into two categories: self‐assessment by theatre staff and assessment by observers. A critical and comparative analysis of the validity and reliability of these tools is lacking. Methods MEDLINE and Embase databases were searched following PRISMA guidelines. Content validity was assessed using measurements of inter‐rater agreement, predictive validity and multisite reliability, and interobserver reliability using statistical measures of inter‐rater agreement and reliability. Quantitative meta‐analysis was deemed unsuitable. Results Forty‐eight articles were selected for final inclusion; self‐assessment tools were used in 18 and observational tools in 28, and there were two qualitative studies. Self‐assessment of teamwork by profession varied with the profession of the assessor. The most robust self‐assessment tool was the Safety Attitudes Questionnaire (SAQ), although this failed to demonstrate multisite reliability. The most robust observational tool was the Non‐Technical Skills (NOTECHS) system, which demonstrated both test–retest reliability (P > 0·09) and interobserver reliability (Rwg = 0·96). Conclusion Self‐assessment of teamwork by the theatre team was influenced by professional differences. Observational tools, when used by trained observers, circumvented this.

The Democritos Movement Screening Tool for Preschool Children (DEMOST-PRE©): development and factorial validity.

PubMed

Kambas, Antonis; Venetsanou, Fotini

2014-07-01

The aim of this study was (a) to develop an assessment tool (the Democritos Movement Screening Tool for Preschool Children - DEMOST-PRE), designed to provide preschool educators, clinicians and researchers with information about assessment and screening of the motor proficiency of children aged 4-6 years, as well as the development and control of movement programmes and (b) to assess its factorial validity. First, tool's content and face validity were established and its final structure was determined. Then, the DEMOST-PRE was administered to 435 children (197 girls) aged 48-71 months (M=60.48 months, SD=6.98). The factor analysis conducted revealed two distinct components. Present evidence combined with the DEMOST-PRE administrative traits make it promising for preschool aged children's assessment. Copyright © 2014 Elsevier Ltd. All rights reserved.

Validation of a novel duplex ultrasound objective structured assessment of technical skills (DUOSATS) for arterial stenosis detection.

PubMed

Jaffer, U; Singh, P; Pandey, V A; Aslam, M; Standfield, N J

2014-01-01

Duplex ultrasound facilitates bedside diagnosis and hence timely patient care. Its uptake has been hampered by training and accreditation issues. We have developed an assessment tool for Duplex arterial stenosis measurement for both simulator and patient based training. A novel assessment tool: duplex ultrasound assessment of technical skills was developed. A modified duplex ultrasound assessment of technical skills was used for simulator training. Novice, intermediate experience and expert users of duplex ultrasound were invited to participate. Participants viewed an instructional video and were allowed ample time to familiarize with the equipment. Participants' attempts were recorded and independently assessed by four experts using the modified duplex ultrasound assessment of technical skills. 'Global' assessment was also done on a four point Likert scale. Content, construct and concurrent validity as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. The simulator had good satisfaction rating from participants: median 4; range 3-5. Receiver operator characteristic analysis has established a cut point of 22/ 34 and 25/ 40 were most appropriate for simulator and patient based assessment respectively. We have validated a novel assessment tool for duplex arterial stenosis detection. Further work is underway to establish transference validity of simulator training to improved skill in scanning patients. We have developed and validated duplex ultrasound assessment of technical skills for simulator training.

Assessing Reliability and Validity of the "GroPromo" Audit Tool for Evaluation of Grocery Store Marketing and Promotional Environments

ERIC Educational Resources Information Center

Kerr, Jacqueline; Sallis, James F.; Bromby, Erica; Glanz, Karen

2012-01-01

Objective: To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Methods: Trained observers used the "GroPromo" instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity,…

Development and validation of a Malawian version of the primary care assessment tool.

PubMed

Dullie, Luckson; Meland, Eivind; Hetlevik, Øystein; Mildestvedt, Thomas; Gjesdal, Sturla

2018-05-16

Malawi does not have validated tools for assessing primary care performance from patients' experience. The aim of this study was to develop a Malawian version of Primary Care Assessment Tool (PCAT-Mw) and to evaluate its reliability and validity in the assessment of the core primary care dimensions from adult patients' perspective in Malawi. A team of experts assessed the South African version of the primary care assessment tool (ZA-PCAT) for face and content validity. The adapted questionnaire underwent forward and backward translation and a pilot study. The tool was then used in an interviewer administered cross-sectional survey in Neno district, Malawi, to test validity and reliability. Exploratory factor analysis was performed on a random half of the sample to evaluate internal consistency, reliability and construct validity of items and scales. The identified constructs were then tested with confirmatory factor analysis. Likert scale assumption testing and descriptive statistics were done on the final factor structure. The PCAT-Mw was further tested for intra-rater and inter-rater reliability. From the responses of 631 patients, a 29-item PCAT-Mw was constructed comprising seven multi-item scales, representing five primary care dimensions (first contact, continuity, comprehensiveness, coordination and community orientation). All the seven scales achieved good internal consistency, item-total correlations and construct validity. Cronbach's alpha coefficient ranged from 0.66 to 0.91. A satisfactory goodness of fit model was achieved (GFI = 0.90, CFI = 0.91, RMSEA = 0.05, PCLOSE = 0.65). The full range of possible scores was observed for all scales. Scaling assumptions tests were achieved for all except the two comprehensiveness scales. Intra-class correlation coefficient (ICC) was 0.90 (n = 44, 95% CI 0.81-0.94, p < 0.001) for intra-rater reliability and 0.84 (n = 42, 95% CI 0.71-0.96, p < 0.001) for inter-rater reliability. Comprehensive metric analyses supported the reliability and validity of PCAT-Mw in assessing the core concepts of primary care from adult patients' experience. This tool could be used for health service research in primary care in Malawi.

Construct Validation--Community College Instructional Development Inventory

ERIC Educational Resources Information Center

Xiong, Soua; Delgado, Nexi; Wood, J. Luke; Harris, Frank, III

2017-01-01

This white paper describes the construct validation of the Community College Instructional Development Inventory (CC-IDI). The CC-IDI is an institutional assessment tool designed to inform professional development programming for instructional faculty. The instrument was developed to serve as a standardized assessment tool to determine the…

A Rapid Assessment Tool for affirming good practice in midwifery education programming.

PubMed

Fullerton, Judith T; Johnson, Peter; Lobe, Erika; Myint, Khine Haymar; Aung, Nan Nan; Moe, Thida; Linn, Nay Aung

2016-03-01

to design a criterion-referenced assessment tool that could be used globally in a rapid assessment of good practices and bottlenecks in midwifery education programs. a standard tool development process was followed, to generate standards and reference criteria; followed by external review and field testing to document psychometric properties. review of standards and scoring criteria were conducted by stakeholders around the globe. Field testing of the tool was conducted in Myanmar. eleven of Myanmar׳s 22 midwifery education programs participated in the assessment. the clinimetric tool was demonstrated to have content validity and high inter-rater reliability in use. a globally validated tool, and accompanying user guide and handbook are now available for conducting rapid assessments of compliance with good practice criteria in midwifery education programming. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Overcoming redundancies in bedside nursing assessments by validating a parsimonious meta-tool: findings from a methodological exercise study.

PubMed

Palese, Alvisa; Marini, Eva; Guarnier, Annamaria; Barelli, Paolo; Zambiasi, Paola; Allegrini, Elisabetta; Bazoli, Letizia; Casson, Paola; Marin, Meri; Padovan, Marisa; Picogna, Michele; Taddia, Patrizia; Chiari, Paolo; Salmaso, Daniele; Marognolli, Oliva; Canzan, Federica; Ambrosi, Elisa; Saiani, Luisa; Grassetti, Luca

2016-10-01

There is growing interest in validating tools aimed at supporting the clinical decision-making process and research. However, an increased bureaucratization of clinical practice and redundancies in the measures collected have been reported by clinicians. Redundancies in clinical assessments affect negatively both patients and nurses. To validate a meta-tool measuring the risks/problems currently estimated by multiple tools used in daily practice. A secondary analysis of a database was performed, using a cross-validation and a longitudinal study designs. In total, 1464 patients admitted to 12 medical units in 2012 were assessed at admission with the Brass, Barthel, Conley and Braden tools. Pertinent outcomes such as the occurrence of post-discharge need for resources and functional decline at discharge, as well as falls and pressure sores, were measured. Explorative factor analysis of each tool, inter-tool correlations and a conceptual evaluation of the redundant/similar items across tools were performed. Therefore, the validation of the meta-tool was performed through explorative factor analysis, confirmatory factor analysis and the structural equation model to establish the ability of the meta-tool to predict the outcomes estimated by the original tools. High correlations between the tools have emerged (from r 0.428 to 0.867) with a common variance from 18.3% to 75.1%. Through a conceptual evaluation and explorative factor analysis, the items were reduced from 42 to 20, and the three factors that emerged were confirmed by confirmatory factor analysis. According to the structural equation model results, two out of three emerged factors predicted the outcomes. From the initial 42 items, the meta-tool is composed of 20 items capable of predicting the outcomes as with the original tools. © 2016 John Wiley & Sons, Ltd.

Exploring the Item Order Effect in a Geoscience Concept Inventory

ERIC Educational Resources Information Center

Undersander, Molly A.; Kettler, Richard M.; Stains, Marilyne

2017-01-01

Concept inventories have been determined to be useful assessment tools for evaluating students' knowledge, particularly in the sciences. However, these assessment tools must be validated to reflect as accurately as possible students' understanding of concepts. One possible threat to this validation is what previous literature calls the item order…

Cognitive assessment tools in Asia: a systematic review.

PubMed

Rosli, Roshaslina; Tan, Maw Pin; Gray, William Keith; Subramanian, Pathmawathi; Chin, Ai-Vyrn

2016-02-01

The prevalence of dementia is increasing in Asia than in any other continent. However, the applicability of the existing cognitive assessment tools is limited by differences in educational and cultural factors in this setting. We conducted a systematic review of published studies on cognitive assessments tools in Asia. We aimed to rationalize the results of available studies which evaluated the validity of cognitive tools for the detection of cognitive impairment and to identify the issues surrounding the available cognitive impairment screening tools in Asia. Five electronic databases (CINAHL, MEDLINE, Embase, Cochrane Library, and Science Direct) were searched using the keywords dementia Or Alzheimer Or cognitive impairment And screen Or measure Or test Or tool Or instrument Or assessment, and 2,381 articles were obtained. Thirty-eight articles, evaluating 28 tools in seven Asian languages, were included. Twenty-nine (76%) of the studies had been conducted in East Asia with only four studies conducted in South Asia and no study from northern, western, or central Asia or Indochina. Local language translations of the Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) were assessed in 15 and six studies respectively. Only three tools (the Korean Dementia Screening Questionnaire, the Picture-based Memory Intelligence Scale, and the revised Hasegawa Dementia Screen) were derived de novo from Asian populations. These tools were assessed in five studies. Highly variable cut-offs were reported for the MMSE (17-29/30) and MoCA (21-26/30), with 13/19 (68%) of studies reporting educational bias. Few cognitive assessment tools have been validated in Asia, with no published validation studies for many Asian nations and languages. In addition, many available tools display educational bias. Future research should include concerted efforts to develop culturally appropriate tools with minimal educational bias.

Hierarchical Clustering on the Basis of Inter-Job Similarity as a Tool in Validity Generalization

ERIC Educational Resources Information Center

Mobley, William H.; Ramsay, Robert S.

1973-01-01

The present research was stimulated by three related problems frequently faced in validation research: viable procedures for combining similar jobs in order to assess the validity of various predictors, for assessing groups of jobs represented in previous validity studies, and for assessing the applicability of validity findings between units.…

[Validation of a nutritional screening tool for hospitalized pediatric patients].

PubMed

Lama More, R A; Moráis López, A; Herrero Álvarez, M; Caraballo Chicano, S; Galera Martínez, R; López Ruzafa, E; Rodríguez Martínez, G; de la Mano Hernández, A; Rivero de la Rosa, M C

2012-01-01

Malnutrition among hospitalized patients has clinical implications, and interest has arisen to find screening tools able to identify subjects under risk. At present, there is no consensus about the most suitable nutrition screening tool for pediatric patients. To validate STAMP (Screening Tool for the Assessment of Malnutrition in Pediatrics) pediatric screening tool in Spain. Descriptive cross-sectional study of patients admitted to a 3rd level children's hospital with both medical and surgical specialities. During the first 24 hours of admission, STAMP screening tool was applied. For its validation, results were compared with those obtained from a nutritional assessment performed by specialist staff, which included clinical, anthropometric and body composition data. A sample of 250 children was studied. Nutritional assessment identified 64 patients (25.6%) under risk, 40 of whom were malnourished (16%). STAMP classified 48.4% of the patients as being under nutritional risk. This tool showed 75% sensitivity and 60.8% specificity when identifying patients under risk according to nutritional assessment. It showed 90% sensitivity and 59.5% specificity when identifying malnourished patients. Malnutrition was less frequent than that reported in other European countries, although diagnosis technique was different. STAMP is a simple and useful tool for nutritional screening, avoiding the need to assess all patients on admission in order to identify those under nutritional risk.

Development and psychometric validation of a scale to assess information needs in cardiac rehabilitation: the INCR Tool.

PubMed

Ghisi, Gabriela Lima de Melo; Grace, Sherry L; Thomas, Scott; Evans, Michael F; Oh, Paul

2013-06-01

To develop and psychometrically validate a tool to assess information needs in cardiac rehabilitation (CR) patients. After a literature search, 60 information items divided into 11 areas of needs were identified. To establish content validity, they were reviewed by an expert panel (N=10). Refined items were pilot-tested in 34 patients on a 5-point Likert-scale from 1 "really not helpful" to 5 "very important". A final version was generated and psychometrically tested in 203 CR patients. Test-retest reliability was assessed via the intraclass correlation coefficient (ICC), the internal consistency using Cronbach's alpha, and criterion validity was assessed with regard to patient's education and duration in CR. Five items were excluded after ICC analysis as well as one area of needs. All 10 areas were considered internally consistent (Cronbach's alpha>0.7). Criterion validity was supported by significant differences in mean scores by educational level (p<0.05) and duration in CR (p<0.001). The mean total score was 4.08 ± 0.53. Patients rated safety as their greatest information need. The INCR Tool was demonstrated to have good reliability and validity. This is an appropriate tool for application in clinical and research settings, assessing patients' needs during CR and as part of education programming. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Development and testing of a DVT risk assessment tool: providing evidence of validity and reliability.

PubMed

McCaffrey, Ruth; Bishop, Mary; Adonis-Rizzo, Marie; Williamson, Ellen; McPherson, Melanie; Cruikshank, Alice; Carrier, Vicki Jo; Sands, Simone; Pigano, Diane; Girard, Patricia; Lauzon, Cathy

2007-01-01

Hospital-acquired deep vein thrombosis (DVT) and pulmonary embolisms (PE) are preventable problems that can increase mortality. Early assessment and recognition of risk as well as initiating appropriate prevention measures can prevent DVT or PE. The purpose of this research project was to develop a DVT risk assessment tool and test the tool for validity and reliability. Three phases were undertaken in developing and testing the JFK Medical Center DVT risk assessment tool. Investigation and clarification of risk and predisposing factors for DVT were identified from the literature, expert nursing knowledge, and medical staff input. Second, item development and weighting were undertaken. Third, parametric testing for content validity measured the differences in mean assessment tool scores between a group of patients who developed DVT in the hospital and a demographically similar group who did not develop DVT. Interrater reliability was measured by having three different nurses score each patient and compare the differences in scores among the three. The DVT group had significantly higher scores on the JFK DVT assessment scale than did those who did not experience DVT. Interrater reliability showed a strong correlation among the scores of the three nurses (.98). Providing a valid and reliable tool for measuring the risk for DVT or PE in hospitalized patients will enable nurses to intervene early in patients at risk. Basing DVT risk assessment on the evidence provided in this study will assist nurses in becoming more confident in recognizing the necessity for interventions in hospitalized patients and decreasing risk. Nurses can now evaluate patients at risk for DVT or PE using the JFK Medial Center's risk assessment tool.

Validation of a novel venous duplex ultrasound objective structured assessment of technical skills for the assessment of venous reflux.

PubMed

Jaffer, Usman; Normahani, Pasha; Lackenby, Kimberly; Aslam, Mohammed; Standfield, Nigel J

2015-01-01

Duplex ultrasound measurement of reflux time is central to the diagnosis of venous incompetence. We have developed an assessment tool for Duplex measurement of venous reflux for both simulator and patient-based training. A novel assessment tool, Venous Duplex Ultrasound Assessment of Technical Skills (V-DUOSATS), was developed. A modified DUOSATS was used for simulator training. Participants of varying skill level were invited to viewed an instructional video and were allowed ample time to familiarize with the Duplex equipment. Attempts made by the participants were recorded and independently assessed by 3 expert assessors and 5 novice assessors using the modified V-DUOSATS. "Global" assessment was also done by expert assessors on a 4-point Likert scale. Content, construct, and concurrent validities as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. Receiver operator characteristic analysis-established cut points of 19/22 and 21/30 were most appropriate for simulator and patient-based assessment, respectively. We have validated a novel assessment tool for Duplex venous reflux measurement. Further work is required to establish transference validity of simulator training to improve skill in scanning patients. We have developed and validated V-DUOSATS for simulator training. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Performance comparison of SNP detection tools with illumina exome sequencing data—an assessment using both family pedigree information and sample-matched SNP array data

PubMed Central

Yi, Ming; Zhao, Yongmei; Jia, Li; He, Mei; Kebebew, Electron; Stephens, Robert M.

2014-01-01

To apply exome-seq-derived variants in the clinical setting, there is an urgent need to identify the best variant caller(s) from a large collection of available options. We have used an Illumina exome-seq dataset as a benchmark, with two validation scenarios—family pedigree information and SNP array data for the same samples, permitting global high-throughput cross-validation, to evaluate the quality of SNP calls derived from several popular variant discovery tools from both the open-source and commercial communities using a set of designated quality metrics. To the best of our knowledge, this is the first large-scale performance comparison of exome-seq variant discovery tools using high-throughput validation with both Mendelian inheritance checking and SNP array data, which allows us to gain insights into the accuracy of SNP calling through such high-throughput validation in an unprecedented way, whereas the previously reported comparison studies have only assessed concordance of these tools without directly assessing the quality of the derived SNPs. More importantly, the main purpose of our study was to establish a reusable procedure that applies high-throughput validation to compare the quality of SNP discovery tools with a focus on exome-seq, which can be used to compare any forthcoming tool(s) of interest. PMID:24831545

Development, Validation, and Verification of a Self-Assessment Tool to Estimate Agnibala (Digestive Strength).

PubMed

Singh, Aparna; Singh, Girish; Patwardhan, Kishor; Gehlot, Sangeeta

2017-01-01

According to Ayurveda, the traditional system of healthcare of Indian origin, Agni is the factor responsible for digestion and metabolism. Four functional states (Agnibala) of Agni have been recognized: regular, irregular, intense, and weak. The objective of the present study was to develop and validate a self-assessment tool to estimate Agnibala The developed tool was evaluated for its reliability and validity by administering it to 300 healthy volunteers of either gender belonging to 18 to 40-year age group. Besides confirming the statistical validity and reliability, the practical utility of the newly developed tool was also evaluated by recording serum lipid parameters of all the volunteers. The results show that the lipid parameters vary significantly according to the status of Agni The tool, therefore, may be used to screen normal population to look for possible susceptibility to certain health conditions. © The Author(s) 2016.

Obstetric care providers are able to assess psychosocial risks, identify and refer high-risk pregnant women: validation of a short assessment tool - the KINDEX Greek version.

PubMed

Spyridou, Andria; Schauer, Maggie; Ruf-Leuschner, Martina

2015-02-21

Prenatal assessment for psychosocial risk factors and prevention and intervention is scarce and, in most cases, nonexistent in obstetrical care. In this study we aimed to evaluate if the KINDEX, a short instrument developed in Germany, is a useful tool in the hands of non-trained medical staff, in order to identify and refer women in psychosocial risk to the adequate mental health and social services. We also examined the criterion-related concurrent validity of the tool through a validation interview carried out by an expert clinical psychologist. Our final objective was to achieve the cultural adaptation of the KINDEX Greek Version and to offer a valid tool for the psychosocial risk assessment to the obstetric care providers. Two obstetricians and five midwives carried out 93 KINDEX interviews (duration 20 minutes) with pregnant women to assess psychosocial risk factors present during pregnancy. Afterwards they referred women who they identified having two or more psychosocial risk factors to the mental health attention unit of the hospital. During the validation procedure an expert clinical psychologist carried out diagnostic interviews with a randomized subsample of 50 pregnant women based on established diagnostic instruments for stress and psychopathology, like the PSS-14, ESI, PDS, HSCL-25. Significant correlations between the results obtained through the assessment using the KINDEX and the risk areas of stress, psychopathology and trauma load assessed in the validation interview demonstrate the criterion-related concurrent validity of the KINDEX. The referral accuracy of the medical staff is confirmed through comparisons between pregnant women who have and have not been referred to the mental health attention unit. Prenatal screenings for psychosocial risks like the KINDEX are feasible in public health settings in Greece. In addition, validity was confirmed in high correlations between the KINDEX results and the results of the validation interviews. The KINDEX Greek version can be considered a valid tool, which can be used by non-trained medical staff providing obstetrical care to identify high-risk women and refer them to adequate mental health and social services. These kind of assessments are indispensable for the promotion of a healthy family environment and child development.

Development and validation of an observation tool for the assessment of nursing pain management practices in intensive care unit in a standardized clinical simulation setting.

PubMed

Gosselin, Emilie; Bourgault, Patricia; Lavoie, Stephan; Coleman, Robin-Marie; Méziat-Burdin, Anne

2014-12-01

Pain management in the intensive care unit is often inadequate. There is no tool available to assess nursing pain management practices. The aim of this study was to develop and validate a measuring tool to assess nursing pain management in the intensive care unit during standardized clinical simulation. A literature review was performed to identify relevant components demonstrating optimal pain management in adult intensive care units and to integrate them in an observation tool. This tool was submitted to an expert panel and pretested. It was then used to assess pain management practice during 26 discrete standardized clinical simulation sessions with intensive care nurses. The Nursing Observation Tool for Pain Management (NOTPaM) contains 28 statements grouped into 8 categories, which are grouped into 4 dimensions: subjective assessment, objective assessment, interventions, and reassessment. The tool's internal consistency was calculated at a Cronbach's alpha of 0.436 for the whole tool; the alpha varies from 0.328 to 0.518 for each dimension. To evaluate the inter-rater reliability, intra-class correlation coefficient was used, which was calculated at 0.751 (p < .001) for the whole tool, with variations from 0.619 to 0.920 (p < .01) between dimensions. The expert panel was satisfied with the content and face validity of the tool. The psychometric qualities of the NOTPaM developed in this study are satisfactory. However, the tool could be improved with slight modifications. Nevertheless, it was useful in assessing intensive care nurses' pain management in a standardized clinical simulation. The NOTPaM is the first tool created for this purpose. Copyright © 2014 American Society for Pain Management Nursing. Published by Elsevier Inc. All rights reserved.

Children's Physical Activity While Gardening: Development of a Valid and Reliable Direct Observation Tool.

PubMed

Myers, Beth M; Wells, Nancy M

2015-04-01

Gardens are a promising intervention to promote physical activity (PA) and foster health. However, because of the unique characteristics of gardening, no extant tool can capture PA, postures, and motions that take place in a garden. The Physical Activity Research and Assessment tool for Garden Observation (PARAGON) was developed to assess children's PA levels, tasks, postures, and motions, associations, and interactions while gardening. PARAGON uses momentary time sampling in which a trained observer watches a focal child for 15 seconds and then records behavior for 15 seconds. Sixty-five children (38 girls, 27 boys) at 4 elementary schools in New York State were observed over 8 days. During the observation, children simultaneously wore Actigraph GT3X+ accelerometers. The overall interrater reliability was 88% agreement, and Ebel was .97. Percent agreement values for activity level (93%), garden tasks (93%), motions (80%), associations (95%), and interactions (91%) also met acceptable criteria. Validity was established by previously validated PA codes and by expected convergent validity with accelerometry. PARAGON is a valid and reliable observation tool for assessing children's PA in the context of gardening.

Development, Evaluation, and Validation of Environmental Assessment Tools to Evaluate the College Nutrition Environment

ERIC Educational Resources Information Center

Freedman, Marjorie R.

2010-01-01

Objective: To develop, evaluate, and validate 2 nutrition environment assessment tools (surveys), for specific use in combating overweight on college/university campuses. Participants and Methods: Invitations to complete surveys were e-mailed to food service and health center directors at 47 universities, Winter 2008. Overall response rate was…

The effectiveness of environment assessment tools to guide refurbishment of Australian residential aged care facilities: A systematic review.

PubMed

Neylon, Samantha; Bulsara, Caroline; Hill, Anne-Marie

2017-06-01

To determine applicability of environment assessment tools in guiding minor refurbishments of Australian residential aged care facilities. Studies conducted in residential aged care settings using assessment tools which address the physical environment were eligible for inclusion in a systematic review. Given these studies are limited, tools which have not yet been utilised in research settings were also included. Tools were analysed using a critical appraisal screen. Forty-three publications met the inclusion criteria. Ten environment assessment tools were identified, of which four addressed all seven minor refurbishment domains of lighting, colour and contrast, sound, flooring, furniture, signage and way finding. Only one had undergone reliability and validity testing. There are four tools which may be suitable to use for minor refurbishment of Australian residential aged care facilities. Data on their reliability, validity and quality are limited. © 2017 AJA Inc.

Validity of a self-report survey tool measuring the nutrition and physical activity environment of primary schools.

PubMed

Nathan, Nicole; Wolfenden, Luke; Morgan, Philip J; Bell, Andrew C; Barker, Daniel; Wiggers, John

2013-06-13

Valid tools measuring characteristics of the school environment associated with the physical activity and dietary behaviours of children are needed to accurately evaluate the impact of initiatives to improve school environments. The aim of this study was to assess the validity of Principal self-report of primary school healthy eating and physical activity environments. Primary school Principals (n = 42) in New South Wales, Australia were invited to complete a telephone survey of the school environment; the School Environment Assessment Tool - SEAT. Equivalent observational data were collected by pre-service teachers located within the school. The SEAT, involved 65 items that assessed food availability via canteens, vending machines and fundraisers and the presence of physical activity facilities, equipment and organised physical activities. Kappa statistics were used to assess agreement between the two measures. Almost 70% of the survey demonstrated moderate to almost perfect agreement. Substantial agreement was found for 10 of 13 items assessing foods sold for fundraising, 3 of 6 items assessing physical activity facilities of the school, and both items assessing organised physical activities that occurred at recess and lunch and school sport. Limited agreement was found for items assessing foods sold through canteens and access to small screen recreation. The SEAT provides researchers and policy makers with a valid tool for assessing aspects of the school food and physical activity environment.

Reliability, Validity, and Factor Structure of the Current Assessment Practice Evaluation-Revised (CAPER) in a National Sample.

PubMed

Lyon, Aaron R; Pullmann, Michael D; Dorsey, Shannon; Martin, Prerna; Grigore, Alexandra A; Becker, Emily M; Jensen-Doss, Amanda

2018-05-11

Measurement-based care (MBC) is an increasingly popular, evidence-based practice, but there are no tools with established psychometrics to evaluate clinician use of MBC practices in mental health service delivery. The current study evaluated the reliability, validity, and factor structure of scores generated from a brief, standardized tool to measure MBC practices, the Current Assessment Practice Evaluation-Revised (CAPER). Survey data from a national sample of 479 mental health clinicians were used to conduct exploratory and confirmatory factor analyses, as well as reliability and validity analyses (e.g., relationships between CAPER subscales and clinician MBC attitudes). Analyses revealed competing two- and three-factor models. Regardless of the model used, scores from CAPER subscales demonstrated good reliability and convergent and divergent validity with MBC attitudes in the expected directions. The CAPER appears to be a psychometrically sound tool for assessing clinician MBC practices. Future directions for development and application of the tool are discussed.

Visual Impairment Screening Assessment (VISA) tool: pilot validation.

PubMed

Rowe, Fiona J; Hepworth, Lauren R; Hanna, Kerry L; Howard, Claire

2018-03-06

To report and evaluate a new Vision Impairment Screening Assessment (VISA) tool intended for use by the stroke team to improve identification of visual impairment in stroke survivors. Prospective case cohort comparative study. Stroke units at two secondary care hospitals and one tertiary centre. 116 stroke survivors were screened, 62 by naïve and 54 by non-naïve screeners. Both the VISA screening tool and the comprehensive specialist vision assessment measured case history, visual acuity, eye alignment, eye movements, visual field and visual inattention. Full completion of VISA tool and specialist vision assessment was achieved for 89 stroke survivors. Missing data for one or more sections typically related to patient's inability to complete the assessment. Sensitivity and specificity of the VISA screening tool were 90.24% and 85.29%, respectively; the positive and negative predictive values were 93.67% and 78.36%, respectively. Overall agreement was significant; k=0.736. Lowest agreement was found for screening of eye movement and visual inattention deficits. This early validation of the VISA screening tool shows promise in improving detection accuracy for clinicians involved in stroke care who are not specialists in vision problems and lack formal eye training, with potential to lead to more prompt referral with fewer false positives and negatives. Pilot validation indicates acceptability of the VISA tool for screening of visual impairment in stroke survivors. Sensitivity and specificity were high indicating the potential accuracy of the VISA tool for screening purposes. Results of this study have guided the revision of the VISA screening tool ahead of full clinical validation. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

PubMed

Vemer, P; Corro Ramos, I; van Voorn, G A K; Al, M J; Feenstra, T L

2016-04-01

A trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool providing model users with a structured view into the validation status of HE decision models, to address this trade-off. A Delphi panel was organized, and was completed by a workshop during an international conference. The proposed tool was constructed iteratively based on comments from, and the discussion amongst, panellists. During the Delphi process, comments were solicited on the importance and feasibility of possible validation techniques for modellers, their relevance for decision makers, and the overall structure and formulation in the tool. The panel consisted of 47 experts in HE modelling and HE decision making from various professional and international backgrounds. In addition, 50 discussants actively engaged in the discussion at the conference workshop and returned 19 questionnaires with additional comments. The final version consists of 13 items covering all relevant aspects of HE decision models: the conceptual model, the input data, the implemented software program, and the model outcomes. Assessment of the Validation Status of Health-Economic decision models (AdViSHE) is a validation-assessment tool in which model developers report in a systematic way both on validation efforts performed and on their outcomes. Subsequently, model users can establish whether confidence in the model is justified or whether additional validation efforts should be undertaken. In this way, AdViSHE enhances transparency of the validation status of HE models and supports efficient model validation.

Development and psychometric testing of the Carter Assessment of Critical Thinking in Midwifery (Preceptor/Mentor version).

PubMed

Carter, Amanda G; Creedy, Debra K; Sidebotham, Mary

2016-03-01

develop and test a tool designed for use by preceptors/mentors to assess undergraduate midwifery students׳ critical thinking in practice. a descriptive cohort design was used. participants worked in a range of maternity settings in Queensland, Australia. 106 midwifery clinicians who had acted in the role of preceptor for undergraduate midwifery students. this study followed a staged model for tool development recommended by DeVellis (2012). This included generation of items, content validity testing through mapping of draft items to critical thinking concepts and expert review, administration of items to a convenience sample of preceptors, and psychometric testing. A 24 item tool titled the XXXX Assessment of Critical Thinking in Midwifery (CACTiM) was completed by registered midwives in relation to students they had recently preceptored in the clinical environment. ratings by experts revealed a content validity index score of 0.97, representing good content validity. An evaluation of construct validity through factor analysis generated three factors: 'partnership in practice', 'reflection on practice' and 'practice improvements'. The scale demonstrated good internal reliability with a Cronbach alpha coefficient of 0.97. The mean total score for the CACTiM scale was 116.77 (SD=16.68) with a range of 60-144. Total and subscale scores correlated significantly. the CACTiM (Preceptor/Mentor version) was found to be a valid and reliable tool for use by preceptors to assess critical thinking in undergraduate midwifery students. given the importance of critical thinking skills for midwifery practice, mapping and assessing critical thinking development in students׳ practice across an undergraduate programme is vital. The CACTiM (Preceptor/Mentor version) has utility for clinical education, research and practice. The tool can inform and guide preceptors׳ assessment of students׳ critical thinking in practice. The availability of a reliable and valid tool can be used to research the development of critical thinking in practice. Crown Copyright © 2015. Published by Elsevier Ltd. All rights reserved.

The risk of bias in systematic reviews tool showed fair reliability and good construct validity.

PubMed

Bühn, Stefanie; Mathes, Tim; Prengel, Peggy; Wegewitz, Uta; Ostermann, Thomas; Robens, Sibylle; Pieper, Dawid

2017-11-01

There is a movement from generic quality checklists toward a more domain-based approach in critical appraisal tools. This study aimed to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR), that is, the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability, and applicability. Validation study with four reviewers based on 16 systematic reviews in the field of occupational health. Interrater reliability (IRR) of all four raters was highest for domain 2 (Fleiss' kappa κ = 0.56) and lowest for domain 4 (κ = 0.04). For ROBIS, median IRR was κ = 0.52 (range 0.13-0.88) for the experienced pair of raters compared to κ = 0.32 (range 0.12-0.76) for the less experienced pair of raters. The percentage of "yes" scores of each review of ROBIS ratings was strongly correlated with the AMSTAR ratings (r s  = 0.76; P = 0.01). ROBIS has fair reliability and good construct validity to assess the risk of bias in systematic reviews. More validation studies are needed to investigate reliability and applicability, in particular. Copyright © 2017 Elsevier Inc. All rights reserved.

Suitability of measurements used to assess mental health outcomes in men and women trafficked for sexual and labour exploitation: a systematic review.

PubMed

Doherty, S; Oram, S; Siriwardhana, C; Abas, M

2016-05-01

Trafficking is a global human rights violation with multiple and complex mental health consequences. Valid and reliable mental health assessment tools are needed to inform health-care provision. We reviewed mental health assessment tools used in research with men and women trafficked for sexual and labour exploitation. We searched nine electronic databases (PsycINFO, Ovid Medline, PubMed, Embase, Assia, the Web of Science, Global Health, Google Scholar, and Open Grey) and hand-searched the reference lists of relevant identified studies. Seven studies were included in this Review. Six of the studies screened for post-traumatic stress disorder, depression, and anxiety; one study screened for harmful use or abuse of alcohol and used a diagnostic tool to assess post-traumatic stress disorder, depression, and anxiety. Two studies included men in their sample population. Although the reported prevalence of mental health problems was high, little information was provided about the validity, reliability, and cultural appropriateness of assessment tools. Further research is needed to determine which assessment tools are culturally appropriate, valid, and reliable for trafficked people. Copyright © 2016 Elsevier Ltd. All rights reserved.

Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

PubMed

Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

2009-09-01

The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools.

PubMed

Griesinger, Claudius; Desprez, Bertrand; Coecke, Sandra; Casey, Warren; Zuang, Valérie

This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the test method for a given purpose. Relevance encapsulates the scientific basis of the test method, its capacity to predict adverse effects in the "target system" (i.e. human health or the environment) as well as its applicability for the intended purpose. In this chapter we focus on the validation of non-animal in vitro alternative testing methods and review the concepts, challenges, processes and tools fundamental to the validation of in vitro methods intended for hazard testing of chemicals. We explore major challenges and peculiarities of validation in this area. Based on the notion that validation per se is a scientific endeavour that needs to adhere to key scientific principles, namely objectivity and appropriate choice of methodology, we examine basic aspects of study design and management, and provide illustrations of statistical approaches to describe predictive performance of validated test methods as well as their reliability.

Methods and Practices of Investigators for Determining Participants’ Decisional Capacity and Comprehension of Protocols

PubMed Central

Kon, Alexander A.; Klug, Michael

2010-01-01

Ethicists recommend that investigators assess subjects’ comprehension prior to accepting their consent as valid. Because children represent an at-risk population, ensuring adequate comprehension in pediatric research is vital. We surveyed all corresponding authors of research articles published over a six-month period in five leading adult and pediatric journals. Our goal was to assess how often subject’s comprehension or decisional capacity was assessed in the consent process, whether there was any difference between adult and pediatric research projects, and the rate at which investigators use formal or validated tools to assess capacity. Responses from 102 authors were analyzed (response rate 56%). Approximately two-thirds of respondents stated that they assessed comprehension or decisional capacity prior to accepting consent, and we found no difference between adult and pediatric researchers. Nine investigators used a formal questionnaire, and three used a validated tool. These findings suggest that fewer than expected investigators assess comprehension and decisional capacity, and that the use of standardized and validated tools is the exception rather than the rule. PMID:19385838

Cognitive assessment: A challenge for occupational therapists in Brazil

PubMed Central

Conti, Juliana

2017-01-01

Cognitive impairment is a common dysfunction after neurological injury. Cognitive assessment tools can help the therapist understand how impairments are affecting functional status and quality of life. Objective The aim of the study was to identify instruments for cognitive assessment that Occupational Therapists (OT) can use in clinical practice. Methods The instruments published in English and Portuguese between 1999 and 2016 were systematically reviewed. Results The search identified 17 specific instruments for OT not validated in Brazilian Portuguese, 10 non-specific instruments for OT not validated in Brazilian Portuguese, and 25 instruments validated for Portuguese, only one of which was specific for OT (Lowenstein Occupational Therapy Cognitive Assessment). Conclusion There are few assessment cognitive tools validated for use in the Brazilian culture and language. The majority of the instruments appear not to be validated for use by OT in clinical practice. PMID:29213503

Improvement of communication and interpersonal competence in telenursing--development of a self-assessment tool.

PubMed

Johnson, Christina; Wilhelmsson, Susan; Börjeson, Sussanne; Lindberg, Malou

2015-06-01

The aim of this study was to develop a self-assessment tool aiming to raise telenurses' awareness of their communication and interpersonal competence, and highlight areas in need of improvement. Several studies have revealed the need for development of communication competence in telenursing. Structured analyses of conversations with patients/callers, is one way to increase telenurses' awareness of their unique communication and interpersonal competence. Instrument development, Validation assessment using the method Content Validity Index. The process to determine content validity was done in two stages; the development stage and the assessment stage. The development stage started with a literature search. The assessment stage was separated into two phases, assessment by an expert group and assessment and test by telenurses. The telenurses also participated in consensus discussions. A telenursing self-assessment tool with 58 items was developed. The items were sorted into five sections according to the nursing process. This study describes the thorough development process of the telenursing self-assessment tool to be used by telenurses in order to become aware of their unique communication and interpersonal competence when analysing their own conversations with patients/callers. As a formative tool it is meant to provide self-direction, feedback and coaching, and create learning opportunities. The self-assessment tool helps the telenurse to follow the nursing process, to be patient-centred, and it is meant to provide self-direction, feedback, and coaching, as well as create learning opportunities. The tool can contribute to the development of communication and interpersonal competence in telephone advice nursing. Further development of the tool may provide an objective scoring instrument for evaluating communication training and education in the field. © 2014 John Wiley & Sons Ltd.

Development of a Content-Valid Standardized Orthopedic Assessment Tool (SOAT)

ERIC Educational Resources Information Center

Lafave, Mark; Katz, Larry; Butterwick, Dale

2008-01-01

Content validation of an instrument that measures student performance in OSCE-type practical examinations is a critical step in a tool's overall validity and reliability [Hopkins (1998), "Educational and Psychological Measurement and Evaluation" (8th ed.). Toronto: Allyn & Bacon]. The purpose of the paper is to outline the process…

Validity and reliability of Internet-based physiotherapy assessment for musculoskeletal disorders: a systematic review.

PubMed

Mani, Suresh; Sharma, Shobha; Omar, Baharudin; Paungmali, Aatit; Joseph, Leonard

2017-04-01

Purpose The purpose of this review is to systematically explore and summarise the validity and reliability of telerehabilitation (TR)-based physiotherapy assessment for musculoskeletal disorders. Method A comprehensive systematic literature review was conducted using a number of electronic databases: PubMed, EMBASE, PsycINFO, Cochrane Library and CINAHL, published between January 2000 and May 2015. The studies examined the validity, inter- and intra-rater reliabilities of TR-based physiotherapy assessment for musculoskeletal conditions were included. Two independent reviewers used the Quality Appraisal Tool for studies of diagnostic Reliability (QAREL) and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool to assess the methodological quality of reliability and validity studies respectively. Results A total of 898 hits were achieved, of which 11 articles based on inclusion criteria were reviewed. Nine studies explored the concurrent validity, inter- and intra-rater reliabilities, while two studies examined only the concurrent validity. Reviewed studies were moderate to good in methodological quality. The physiotherapy assessments such as pain, swelling, range of motion, muscle strength, balance, gait and functional assessment demonstrated good concurrent validity. However, the reported concurrent validity of lumbar spine posture, special orthopaedic tests, neurodynamic tests and scar assessments ranged from low to moderate. Conclusion TR-based physiotherapy assessment was technically feasible with overall good concurrent validity and excellent reliability, except for lumbar spine posture, orthopaedic special tests, neurodynamic testa and scar assessment.

The live donor assessment tool: a psychosocial assessment tool for live organ donors.

PubMed

Iacoviello, Brian M; Shenoy, Akhil; Braoude, Jenna; Jennings, Tiane; Vaidya, Swapna; Brouwer, Julianna; Haydel, Brandy; Arroyo, Hansel; Thakur, Devendra; Leinwand, Joseph; Rudow, Dianne LaPointe

2015-01-01

Psychosocial evaluation is an important part of the live organ donor evaluation process, yet it is not standardized across institutions, and although tools exist for the psychosocial evaluation of organ recipients, none exist to assess donors. We set out to develop a semistructured psychosocial evaluation tool (the Live Donor Assessment Tool, LDAT) to assess potential live organ donors and to conduct preliminary analyses of the tool's reliability and validity. Review of the literature on the psychosocial variables associated with treatment adherence, quality of life, live organ donation outcome, and resilience, as well as review of the procedures for psychosocial evaluation at our center and other centers around the country, identified 9 domains to address; these domains were distilled into several items each, in collaboration with colleagues at transplant centers across the country, for a total of 29 items. Four raters were trained to use the LDAT, and they retrospectively scored 99 psychosocial evaluations conducted on live organ donor candidates. Reliability of the LDAT was assessed by calculating the internal consistency of the items in the scale and interrater reliability between raters; validity was estimated by comparing LDAT scores between those with a "positive" evaluation outcome and "negative" outcome. The LDAT was found to have good internal consistency, inter-rater reliability, and showed signs of validity: LDAT scores differentiated the positive vs. negative outcome groups. The LDAT demonstrated good reliability and validity, but future research on the LDAT and the ability to implement the LDAT prospectively is warranted. Copyright © 2015 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.

A Student Assessment Tool for Standardized Patient Simulations (SAT-SPS): Psychometric analysis.

PubMed

Castro-Yuste, Cristina; García-Cabanillas, María José; Rodríguez-Cornejo, María Jesús; Carnicer-Fuentes, Concepción; Paloma-Castro, Olga; Moreno-Corral, Luis Javier

2018-05-01

The evaluation of the level of clinical competence acquired by the student is a complex process that must meet various requirements to ensure its quality. The psychometric analysis of the data collected by the assessment tools used is a fundamental aspect to guarantee the student's competence level. To conduct a psychometric analysis of an instrument which assesses clinical competence in nursing students at simulation stations with standardized patients in OSCE-format tests. The construct of clinical competence was operationalized as a set of observable and measurable behaviors, measured by the newly-created Student Assessment Tool for Standardized Patient Simulations (SAT-SPS), which was comprised of 27 items. The categories assigned to the items were 'incorrect or not performed' (0), 'acceptable' (1), and 'correct' (2). 499 nursing students. Data were collected by two independent observers during the assessment of the students' performance at a four-station OSCE with standardized patients. Descriptive statistics were used to summarize the variables. The difficulty levels and floor and ceiling effects were determined for each item. Reliability was analyzed using internal consistency and inter-observer reliability. The validity analysis was performed considering face validity, content and construct validity (through exploratory factor analysis), and criterion validity. Internal reliability and inter-observer reliability were higher than 0.80. The construct validity analysis suggested a three-factor model accounting for 37.1% of the variance. These three factors were named 'Nursing process', 'Communication skills', and 'Safe practice'. A significant correlation was found between the scores obtained and the students' grades in general, as well as with the grades obtained in subjects with clinical content. The assessment tool has proven to be sufficiently reliable and valid for the assessment of the clinical competence of nursing students using standardized patients. This tool has three main components: the nursing process, communication skills, and safety management. Copyright © 2018 Elsevier Ltd. All rights reserved.

Evaluating trauma team performance in a Level I trauma center: Validation of the trauma team communication assessment (TTCA-24).

PubMed

DeMoor, Stephanie; Abdel-Rehim, Shady; Olmsted, Richard; Myers, John G; Parker-Raley, Jessica

2017-07-01

Nontechnical skills (NTS), such as team communication, are well-recognized determinants of trauma team performance and good patient care. Measuring these competencies during trauma resuscitations is essential, yet few valid and reliable tools are available. We aimed to demonstrate that the Trauma Team Communication Assessment (TTCA-24) is a valid and reliable instrument that measures communication effectiveness during activations. Two tools with adequate psychometric strength (Trauma Nontechnical Skills Scale [T-NOTECHS], Team Emergency Assessment Measure [TEAM]) were identified during a systematic review of medical literature and compared with TTCA-24. Three coders used each tool to evaluate 35 stable and 35 unstable patient activations (defined according to Advanced Trauma Life Support criteria). Interrater reliability was calculated between coders using the intraclass correlation coefficient. Spearman rank correlation coefficient was used to establish concurrent validity between TTCA-24 and the other two validated tools. Coders achieved an intraclass correlation coefficient of 0.87 for stable patient activations and 0.78 for unstable activations scoring excellent on the interrater agreement guidelines. The median score for each assessment showed good team communication for all 70 videos (TEAM, 39.8 of 54; T-NOTECHS, 17.4 of 25; and TTCA-24, 87.4 of 96). A significant correlation between TTTC-24 and T-NOTECHS was revealed (p = 0.029), but no significant correlation between TTCA-24 and TEAM (p = 0.77). Team communication was rated slightly better across all assessments for stable versus unstable patient activations, but not statistically significant. TTCA-24 correlated with T-NOTECHS, an instrument measuring nontechnical skills for trauma teams, but not TEAM, a tool that assesses communication in generic emergency settings. TTCA-24 is a reliable and valid assessment that can be a useful adjunct when evaluating interpersonal and team communication during trauma activations. Diagnostic tests or criteria, level II.

Validity and reliability of wii fit balance board for the assessment of balance of healthy young adults and the elderly.

PubMed

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-10-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86-0.99) for the elderly people and positive correlations (r = 0.58-0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability.

Predictive Validity of Measures of the Pathfinder Scaling Algorithm on Programming Performance: Alternative Assessment Strategy for Programming Education

ERIC Educational Resources Information Center

Lau, Wilfred W. F.; Yuen, Allan H. K.

2009-01-01

Recent years have seen a shift in focus from assessment of learning to assessment for learning and the emergence of alternative assessment methods. However, the reliability and validity of these methods as assessment tools are still questionable. In this article, we investigated the predictive validity of measures of the Pathfinder Scaling…

Development of Reliable and Validated Tools to Evaluate Technical Resuscitation Skills in a Pediatric Simulation Setting: Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics.

PubMed

Faudeux, Camille; Tran, Antoine; Dupont, Audrey; Desmontils, Jonathan; Montaudié, Isabelle; Bréaud, Jean; Braun, Marc; Fournier, Jean-Paul; Bérard, Etienne; Berlengi, Noémie; Schweitzer, Cyril; Haas, Hervé; Caci, Hervé; Gatin, Amélie; Giovannini-Chami, Lisa

2017-09-01

To develop a reliable and validated tool to evaluate technical resuscitation skills in a pediatric simulation setting. Four Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics (RESCAPE) evaluation tools were created, following international guidelines: intraosseous needle insertion, bag mask ventilation, endotracheal intubation, and cardiac massage. We applied a modified Delphi methodology evaluation to binary rating items. Reliability was assessed comparing the ratings of 2 observers (1 in real time and 1 after a video-recorded review). The tools were assessed for content, construct, and criterion validity, and for sensitivity to change. Inter-rater reliability, evaluated with Cohen kappa coefficients, was perfect or near-perfect (>0.8) for 92.5% of items and each Cronbach alpha coefficient was ≥0.91. Principal component analyses showed that all 4 tools were unidimensional. Significant increases in median scores with increasing levels of medical expertise were demonstrated for RESCAPE-intraosseous needle insertion (P = .0002), RESCAPE-bag mask ventilation (P = .0002), RESCAPE-endotracheal intubation (P = .0001), and RESCAPE-cardiac massage (P = .0037). Significantly increased median scores over time were also demonstrated during a simulation-based educational program. RESCAPE tools are reliable and validated tools for the evaluation of technical resuscitation skills in pediatric settings during simulation-based educational programs. They might also be used for medical practice performance evaluations. Copyright © 2017 Elsevier Inc. All rights reserved.

Motivational Interviewing Skills in Health Care Encounters (MISHCE): Development and psychometric testing of an assessment tool.

PubMed

Petrova, Tatjana; Kavookjian, Jan; Madson, Michael B; Dagley, John; Shannon, David; McDonough, Sharon K

2015-01-01

Motivational interviewing (MI) has demonstrated a significant impact as an intervention strategy for addiction management, change in lifestyle behaviors, and adherence to prescribed medication and other treatments. Key elements to studying MI include training in MI of professionals who will use it, assessment of skills acquisition in trainees, and the use of a validated skills assessment tool. The purpose of this research project was to develop a psychometrically valid and reliable tool that has been designed to assess MI skills competence in health care provider trainees. The goal was to develop an assessment tool that would evaluate the acquisition and use of specific MI skills and principles, as well as the quality of the patient-provider therapeutic alliance in brief health care encounters. To address this purpose, specific steps were followed, beginning with a literature review. This review contributed to the development of relevant conceptual and operational definitions, selecting a scaling technique and response format, and methods for analyzing validity and reliability. Internal consistency reliability was established on 88 video recorded interactions. The inter-rater and test-retest reliability were established using randomly selected 18 from the 88 interactions. The assessment tool Motivational Interviewing Skills for Health Care Encounters (MISHCE) and a manual for use of the tool were developed. Validity and reliability of MISHCE were examined. Face and content validity were supported with well-defined conceptual and operational definitions and feedback from an expert panel. Reliability was established through internal consistency, inter-rater reliability, and test-retest reliability. The overall internal consistency reliability (Cronbach's alpha) for all fifteen items was 0.75. MISHCE demonstrated good inter-rater reliability and good to excellent test-retest reliability. MISHCE assesses the health provider's level of knowledge and skills in brief disease management encounters. MISHCE also evaluates quality of the patient-provider therapeutic alliance, i.e., the "flow" of the interaction. Copyright © 2015 Elsevier Inc. All rights reserved.

Tool for Evaluating the Ways Nurses Assess Pain (TENAP): psychometric properties assessment.

PubMed

Ng, Siok Qi; Brammer, Jillian; Creedy, Debra K; Klainin-Yobas, Piyanee

2014-12-01

Elderly people with cognitive impairment are at risk for under-treatment of pain due to their inability to communicate. Poor knowledge and attitudes of nurses toward pain in this population may result in inadequate pain assessment. This study used a descriptive correlational design to develop and validate a tool to assess nurses' knowledge, attitudes, and reported practice of pain assessment in cognitively impaired elderly patients in acute care settings. The Tool for Evaluating the ways Nurses Assess Pain (TENAP) has two sections: (1) nurses' knowledge and attitudes about pain assessment and management and (2) two vignettes to assess reported practice. Content validity was established by an expert panel of three geriatric-trained nurse clinicians, and pilot tested with a convenience sample of 10 nurses. The psychometric properties were tested with a sample of 263 Registered and Enrolled nurses working in medical wards of two public hospitals in Singapore. The final version of TENAP comprised 29 items. Content validity index ranged from 0.84 to 1.00. The scale took 10 to 15 minutes to complete and items were easily understood. Results from the factor analysis suggested that Section A demonstrated one factor (13 items) while Section B had two distinct factors (16 items), one for each vignette, supporting construct validity of the scale. Cronbach's alphas for all factors were acceptable. TENAP was feasible, valid, and reliable for assessing nurses' knowledge, attitudes, and reported practice of pain assessment in cognitively-impaired elderly patients. Further testing of the tool with a larger sample of nurses in other practice contexts is needed. Copyright © 2014 American Society for Pain Management Nursing. Published by Elsevier Inc. All rights reserved.

Development of a Multi-Domain Assessment Tool for Quality Improvement Projects.

PubMed

Rosenbluth, Glenn; Burman, Natalie J; Ranji, Sumant R; Boscardin, Christy K

2017-08-01

Improving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education. We developed and validated a tool to assess QI projects with learner engagement across the educational continuum. After reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool. The resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance. The MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels.

The development and validity of the Salford Gait Tool: an observation-based clinical gait assessment tool.

PubMed

Toro, Brigitte; Nester, Christopher J; Farren, Pauline C

2007-03-01

To develop the construct, content, and criterion validity of the Salford Gait Tool (SF-GT) and to evaluate agreement between gait observations using the SF-GT and kinematic gait data. Tool development and comparative evaluation. University in the United Kingdom. For designing construct and content validity, convenience samples of 10 children with hemiplegic, diplegic, and quadriplegic cerebral palsy (CP) and 152 physical therapy students and 4 physical therapists were recruited. For developing criterion validity, kinematic gait data of 13 gait clusters containing 56 children with hemiplegic, diplegic, and quadriplegic CP and 11 neurologically intact children was used. For clinical evaluation, a convenience sample of 23 pediatric physical therapists participated. We developed a sagittal plane observational gait assessment tool through a series of design, test, and redesign iterations. The tool's grading system was calibrated using kinematic gait data of 13 gait clusters and was evaluated by comparing the agreement of gait observations using the SF-GT with kinematic gait data. Criterion standard kinematic gait data. There was 58% mean agreement based on grading categories and 80% mean agreement based on degree estimations evaluated with the least significant difference method. The new SF-GT has good concurrent criterion validity.

The Johns Hopkins Fall Risk Assessment Tool: A Study of Reliability and Validity.

PubMed

Poe, Stephanie S; Dawson, Patricia B; Cvach, Maria; Burnett, Margaret; Kumble, Sowmya; Lewis, Maureen; Thompson, Carol B; Hill, Elizabeth E

Patient falls and fall-related injury remain a safety concern. The Johns Hopkins Fall Risk Assessment Tool (JHFRAT) was developed to facilitate early detection of risk for anticipated physiologic falls in adult inpatients. Psychometric properties in acute care settings have not yet been fully established; this study sought to fill that gap. Results indicate that the JHFRAT is reliable, with high sensitivity and negative predictive validity. Specificity and positive predictive validity were lower than expected.

The assessment of creativity in creativity/psychopathology research - a systematic review.

PubMed

Thys, E; Sabbe, B; De Hert, M

2014-01-01

The possible link between creativity and psychopathology has been a long time focus of research up to the present day. However, this research is hampered by methodological problems, especially the definition and assessment of creativity. This makes interpretation and comparison of studies difficult and possibly accounts for the contradictory results of this research. In this systematic review of the literature, research articles in the field of creativity and psychopathology were searched for creativity assessment tools. The tools used in the collected articles are presented and discussed. The results indicate that a multitude of creativity assessment tools were used, that many studies only used one tool to assess creativity and that most of these tools were only used in a limited number of studies. A few assessment tools stand out by a more frequent use, also outside psychopathological research, and more solid psychometric properties. Most scales used to evaluate creativity have poor psychometric properties. The scattered methodology to assess creativity compromises the generalizability and validity of this research. The field should creatively develop new validated instruments.

Developing and Validating a Tool to Assess Ethical Decision-Making Ability of Nursing Students, Using Rubrics

ERIC Educational Resources Information Center

Indhraratana, Apinya; Kaemkate, Wannee

2012-01-01

The aim of this paper is to develop a reliable and valid tool to assess ethical decision-making ability of nursing students using rubrics. A proposed ethical decision making process, from reviewing related literature was used as a framework for developing the rubrics. Participants included purposive sample of 86 nursing students from the Royal…

Quality Assessment of Comparative Diagnostic Accuracy Studies: Our Experience Using a Modified Version of the QUADAS-2 Tool

ERIC Educational Resources Information Center

Wade, Ros; Corbett, Mark; Eastwood, Alison

2013-01-01

Assessing the quality of included studies is a vital step in undertaking a systematic review. The recently revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool (QUADAS-2), which is the only validated quality assessment tool for diagnostic accuracy studies, does not include specific criteria for assessing comparative studies. As…

Development and Construct Validation of the Interprofessional Attitudes Scale

PubMed Central

Norris, Jeffrey; Carpenter, Joan G.; Eaton, Jacqueline; Guo, Jia-Wen; Lassche, Madeline; Pett, Marjorie A.; Blumenthal, Donald K.

2015-01-01

Purpose Training of health professionals requires development of interprofessional competencies and assessment of these competencies. No validated tools exist to assess all four competency domains described in the 2011 Core Competencies for Interprofessional Collaborative Practice (the IPEC Report). The purpose of this study was to develop and validate a scale based on the IPEC competency domains that assesses interprofessional attitudes of students in the health professions. Method In 2012, a survey tool was developed and administered to 1,549 students from the University of Utah Health Science Center, an academic health center composed of four schools and colleges (Health, Medicine, Nursing, and Pharmacy). Exploratory and confirmatory factor analyses (EFA and CFA) were performed to validate the assessment tool, eliminate redundant questions, and to identify subscales. Results The EFA and CFA focused on aligning subscales with IPEC core competencies, and demonstrating good construct validity and internal consistency reliability. A response rate of 45% (n = 701) was obtained. Responses with complete data (n=678) were randomly split into two datasets which were independently analyzed using EFA and CFA. The EFA produced a 27-item scale, with five subscales (Cronbach’s alpha coefficients: 0.62 to 0.92). CFA indicated the content of the five subscales was consistent with the EFA model. Conclusions The Interprofessional Attitudes Scale (IPAS) is a novel tool that, compared to previous tools, better reflects current trends in interprofessional competencies. The IPAS should be useful to health sciences educational institutions and others training people to work collaboratively in interprofessional teams. PMID:25993280

Critical thinking skills in midwifery practice: Development of a self-assessment tool for students.

PubMed

Carter, Amanda G; Creedy, Debra K; Sidebotham, Mary

2017-07-01

Develop and test a tool designed for use by pre-registration midwifery students to self-appraise their critical thinking in practice. A descriptive cohort design was used. All students (n=164) enrolled in a three-year Bachelor of Midwifery program in Queensland, Australia. The staged model for tool development involved item generation, mapping draft items to critical thinking concepts and expert review to test content validity, pilot testing of the tool to a convenience sample of students, and psychometric testing. Students (n=126, 76.8% response rate) provided demographic details, completed the new tool, and five questions from the Motivated Strategies for Learning Questionnaire (MSLQ) via an online platform or paper version. A high content validity index score of 0.97 was achieved through expert review. Construct validity via factor analysis revealed four factors: seeks information, reflects on practice, facilitates shared decision making, and evaluates practice. The mean total score for the tool was 124.98 (SD=12.58). Total and subscale scores correlated significantly. The scale achieved good internal reliability with a Cronbach's alpha coefficient of 0.92. Concurrent validity with the MSLQ subscale was 0.35 (p<0.001). This study established the reliability and validity of the CACTiM - student version for use by pre-registration midwifery students to self-assess critical thinking in practice. Critical thinking skills are vital for safe and effective midwifery practice. Students' assessment of their critical thinking development throughout their pre-registration programme makes these skills explicit, and could guide teaching innovation to address identified deficits. The availability of a reliable and valid tool assists research into the development of critical thinking in education and practice. Crown Copyright © 2017. Published by Elsevier Ltd. All rights reserved.

Validating an Environmental Education Field Day Observation Tool

ERIC Educational Resources Information Center

Carlson, Stephan P.; Heimlich, Joe E.; Storksdieck, Martin

2011-01-01

Environmental Field Days (EFD) are held throughout the country and provide a unique opportunity to involve students in real world science. A study to assess the validity of an observation tool for EFD programs was conducted at the Metro Water Festival with fifth grade students. Items from the observation tool were mapped to students' evaluation…

Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men.

PubMed

Hussein, Ahmed A; Sexton, Kevin J; May, Paul R; Meng, Maxwell V; Hosseini, Abolfazl; Eun, Daniel D; Daneshmand, Siamak; Bochner, Bernard H; Peabody, James O; Abaza, Ronney; Skinner, Eila C; Hautmann, Richard E; Guru, Khurshid A

2018-04-13

We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men. A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation. The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains. We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.

A Pathway to Freedom: An Evaluation of Screening Tools for the Identification of Trafficking Victims.

PubMed

Bespalova, Nadejda; Morgan, Juliet; Coverdale, John

2016-02-01

Because training residents and faculty to identify human trafficking victims is a major public health priority, the authors review existing assessment tools. PubMed and Google were searched using combinations of search terms including human, trafficking, sex, labor, screening, identification, and tool. Nine screening tools that met the inclusion criteria were found. They varied greatly in length, format, target demographic, supporting resources, and other parameters. Only two tools were designed specifically for healthcare providers. Only one tool was formally assessed to be valid and reliable in a pilot project in trafficking victim service organizations, although it has not been validated in the healthcare setting. This toolbox should facilitate the education of resident physicians and faculty in screening for trafficking victims, assist educators in assessing screening skills, and promote future research on the identification of trafficking victims.

The IDEA Assessment Tool: Assessing the Reporting, Diagnostic Reasoning, and Decision-Making Skills Demonstrated in Medical Students' Hospital Admission Notes.

PubMed

Baker, Elizabeth A; Ledford, Cynthia H; Fogg, Louis; Way, David P; Park, Yoon Soo

2015-01-01

Construct: Clinical skills are used in the care of patients, including reporting, diagnostic reasoning, and decision-making skills. Written comprehensive new patient admission notes (H&Ps) are a ubiquitous part of student education but are underutilized in the assessment of clinical skills. The interpretive summary, differential diagnosis, explanation of reasoning, and alternatives (IDEA) assessment tool was developed to assess students' clinical skills using written comprehensive new patient admission notes. The validity evidence for assessment of clinical skills using clinical documentation following authentic patient encounters has not been well documented. Diagnostic justification tools and postencounter notes are described in the literature (1,2) but are based on standardized patient encounters. To our knowledge, the IDEA assessment tool is the first published tool that uses medical students' H&Ps to rate students' clinical skills. The IDEA assessment tool is a 15-item instrument that asks evaluators to rate students' reporting, diagnostic reasoning, and decision-making skills based on medical students' new patient admission notes. This study presents validity evidence in support of the IDEA assessment tool using Messick's unified framework, including content (theoretical framework), response process (interrater reliability), internal structure (factor analysis and internal-consistency reliability), and relationship to other variables. Validity evidence is based on results from four studies conducted between 2010 and 2013. First, the factor analysis (2010, n = 216) yielded a three-factor solution, measuring patient story, IDEA, and completeness, with reliabilities of .79, .88, and .79, respectively. Second, an initial interrater reliability study (2010) involving two raters demonstrated fair to moderate consensus (κ = .21-.56, ρ =.42-.79). Third, a second interrater reliability study (2011) with 22 trained raters also demonstrated fair to moderate agreement (intraclass correlations [ICCs] = .29-.67). There was moderate reliability for all three skill domains, including reporting skills (ICC = .53), diagnostic reasoning skills (ICC = .64), and decision-making skills (ICC = .63). Fourth, there was a significant correlation between IDEA rating scores (2010-2013) and final Internal Medicine clerkship grades (r = .24), 95% confidence interval (CI) [.15, .33]. The IDEA assessment tool is a novel tool with validity evidence to support its use in the assessment of students' reporting, diagnostic reasoning, and decision-making skills. The moderate reliability achieved supports formative or lower stakes summative uses rather than high-stakes summative judgments.

Use of dietary assessment tools in randomized trials evaluating diet-based interventions in pregnancy: a systematic review of literature.

PubMed

Al Wattar, Bassel H; Mylrea-Lowndes, Bronacha; Morgan, Catrin; Moore, Amanda P; Thangaratinam, Shakila

2016-12-01

Accurate assessment of dietary intake in interventional trials is the key to evaluate changes in dietary behaviour and compliance. We evaluated the use of dietary assessment tools in randomized trials on diet-based interventions in pregnancy by a systematic review. We updated our previous search (until January 2012) on trials of diet and lifestyle interventions in pregnancy using Medline and EMBASE up to December 2015. Two independent reviewers undertook study selection and data extraction. We assessed the characteristics of dietary assessment tools, the timing and frequency of use and any validation undertaken.Two-thirds (39/58, 67%) of the included studies used some form of tools to assess dietary intake. Multiple days' food diaries were the most commonly used (23/39, 59%). Three studies (3/39, 8%) validated the used tools in a pregnant population. Three studies (3/39, 8%) prespecified the criteria for adherence to the intervention. The use of dietary assessment tools was not associated with study quality, year of publication, journal impact factor, type of journal and the study sample size. Although self-reporting dietary assessment tools are widely used in interventional dietary trials in pregnancy, the quality and applicability of existing tools are low.

The development and validation of an instrument to measure the quality of health research reports in the lay media.

PubMed

Zeraatkar, Dena; Obeda, Michael; Ginsberg, Jeffrey S; Hirsh, Jack

2017-04-20

The media serves as an important link between medical research, as reported in scholarly sources, and the public and has the potential to act as a powerful tool to improve public health. However, concerns about the reliability of health research reports have been raised. Tools to monitor the quality of health research reporting in the media are needed to identify areas of weakness in health research reporting and to subsequently work towards the efficient use of the lay media as a public health tool through which the public's health behaviors can be improved. We developed the Quality Index for health-related Media Reports (QIMR) as a tool to monitor the quality of health research reports in the lay media. The tool was developed according to themes generated from interviews with health journalists and researchers. Item and domain characteristics and scale reliability were assessed. The scale was correlated with a global quality assessment score and media report word count to provide evidence towards its construct validity. The items and domains of the QIMR demonstrated acceptable validity and reliability. Items from the 'validity' domain were negatively skewed, suggesting possible floor effect. These items were not eliminated due to acceptable content and face validity. QIMR total scores produced a strong correlation with raters' global assessment and a moderate correlation with media report word count, providing evidence towards the construct validity of the instrument. The results of this investigation indicate that QIMR can adequately measure the quality of health research reports, with acceptable reliability and validity.

The ELPAT living organ donor Psychosocial Assessment Tool (EPAT): from 'what' to 'how' of psychosocial screening - a pilot study.

PubMed

Massey, Emma K; Timmerman, Lotte; Ismail, Sohal Y; Duerinckx, Nathalie; Lopes, Alice; Maple, Hannah; Mega, Inês; Papachristou, Christina; Dobbels, Fabienne

2018-01-01

Thorough psychosocial screening of donor candidates is required in order to minimize potential negative consequences and to strive for optimal safety within living donation programmes. We aimed to develop an evidence-based tool to standardize the psychosocial screening process. Key concepts of psychosocial screening were used to structure our tool: motivation and decision-making, personal resources, psychopathology, social resources, ethical and legal factors and information and risk processing. We (i) discussed how each item per concept could be measured, (ii) reviewed and rated available validated tools, (iii) where necessary developed new items, (iv) assessed content validity and (v) pilot-tested the new items. The resulting ELPAT living organ donor Psychosocial Assessment Tool (EPAT) consists of a selection of validated questionnaires (28 items in total), a semi-structured interview (43 questions) and a Red Flag Checklist. We outline optimal procedures and conditions for implementing this tool. The EPAT and user manual are available from the authors. Use of this tool will standardize the psychosocial screening procedure ensuring that no psychosocial issues are overlooked and ensure that comparable selection criteria are used and facilitate generation of comparable psychosocial data on living donor candidates. © 2017 Steunstichting ESOT.

Development of an interprofessional lean facilitator assessment scale.

PubMed

Bravo-Sanchez, Cindy; Dorazio, Vincent; Denmark, Robert; Heuer, Albert J; Parrott, J Scott

2018-05-01

High reliability is important for optimising quality and safety in healthcare organisations. Reliability efforts include interprofessional collaborative practice (IPCP) and Lean quality/process improvement strategies, which require skilful facilitation. Currently, no validated Lean facilitator assessment tool for interprofessional collaboration exists. This article describes the development and pilot evaluation of such a tool; the Interprofessional Lean Facilitator Assessment Scale (ILFAS), which measures both technical and 'soft' skills, which have not been measured in other instruments. The ILFAS was developed using methodologies and principles from Lean/Shingo, IPCP, metacognition research and Bloom's Taxonomy of Learning Domains. A panel of experts confirmed the initial face validity of the instrument. Researchers independently assessed five facilitators, during six Lean sessions. Analysis included quantitative evaluation of rater agreement. Overall inter-rater agreement of the assessment of facilitator performance was high (92%), and discrepancies in the agreement statistics were analysed. Face and content validity were further established, and usability was evaluated, through primary stakeholder post-pilot feedback, uncovering minor concerns, leading to tool revision. The ILFAS appears comprehensive in the assessment of facilitator knowledge, skills, abilities, and may be useful in the discrimination between facilitators of different skill levels. Further study is needed to explore instrument performance and validity.

Evaluation of Knowla: An Online Assessment and Learning Tool

ERIC Educational Resources Information Center

Thompson, Meredith Myra; Braude, Eric John

2016-01-01

The assessment of learning in large online courses requires tools that are valid, reliable, easy to administer, and can be automatically scored. We have evaluated an online assessment and learning tool called Knowledge Assembly, or Knowla. Knowla measures a student's knowledge in a particular subject by having the student assemble a set of…

Creation of a Tool for Assessing Knowledge in Evidence-Based Decision-Making in Practicing Health Care Providers.

PubMed

Spurr, Kathy; Dechman, Gail; Lackie, Kelly; Gilbert, Robert

2016-01-01

Evidence-based decision-making (EBDM) is the process health care providers (HCPs) use to identify and appraise potential evidence. It supports the integration of best research evidence with clinical expertise and patient values into the decision-making process for patient care. Competence in this process is essential to delivery of optimal care. There is no objective tool that assesses EBDM across HCP groups. This research aimed to develop a content valid tool to assess knowledge of the principles of evidence-based medicine and the EBDM process, for use with all HCPs. A Delphi process was used in the creation of the tool. Pilot testing established its content validity with the added benefit of evaluating HCPs' knowledge of EBDM. Descriptive statistics and multivariate mixed models were used to evaluate individual survey responses in total, as well as within each EBDM component. The tool consisted of 26 multiple-choice questions. A total of 12,884 HCPs in Nova Scotia were invited to participate in the web-based validation study, yielding 818 (6.3%) participants, 471 of whom completed all questions. The mean overall score was 68%. Knowledge in one component, integration of evidence with clinical expertise and patient preferences, was identified as needing development across all HCPs surveyed. A content valid tool for assessing HCP EBDM knowledge was created and can be used to support the development of continuing education programs to enhance EBDM competency.

The Development and Use of a Concept Mapping Assessment Tool with Young Children on Family Visits to a Live Butterfly Exhibit

ERIC Educational Resources Information Center

Mesa, Jennifer Cheryl

2010-01-01

Although young children are major audiences of science museums, limited evidence exists documenting changes in children's knowledge in these settings due in part to the limited number of valid and reliable assessment tools available for use with this population. The purposes of this study were to develop and validate a concept mapping assessment…

Development and Initial Validation of the NyTid Test: A Movement Assessment Tool for Compulsory School Pupils

ERIC Educational Resources Information Center

Tidén, Anna; Lundqvist, Carolina; Nyberg, Marie

2015-01-01

This study presents the development process and initial validation of the NyTid test, a process-oriented movement assessment tool for compulsory school pupils. A sample of 1,260 (627 girls and 633 boys; mean age of 14.39) Swedish school children participated in the study. In the first step, exploratory factor analyses (EFAs) were performed in…

Validity, efficacy and reliability of 3 nutritional screening tools regarding the nutritional assessment in different social and health areas.

PubMed

Castro-Vega, Iciar; Veses Martín, Silvia; Cantero Llorca, Juana; Barrios Marta, Cristina; Bañuls, Celia; Hernández-Mijares, Antonio

2018-03-09

Nutritional screening allows for the detection of nutritional risk. Validated tools should be implemented, and their usefulness should be contrasted with a gold standard. The aim of this study is to discover the validity, efficacy and reliability of 3 nutritional screening tools in relation to complete nutritional assessment. A sub-analysis of a cross-sectional and descriptive study on the prevalence of disease-related malnutrition. The sample was selected from outpatients, hospitalized and institutionalized patients. MUST, MNAsf and MST screening were employed. A nutritional assessment of all the patients was undertaken. The SENPE-SEDOM consensus was used for the diagnosis. In the outpatients, both MUST and MNAsf have a similar validity in relation to the nutritional assessment (AUC 0.871 and 0.883, respectively). In the institutionalized patients, the MUST screening method is the one that shows the greatest validity (AUC 0.815), whereas in the hospitalized patients, the most valid methods are both MUST and MST (AUC 0.868 and 0.853, respectively). It is essential to use nutritional screening to invest the available resources wisely. Based on our results, MUST is the most suitable screening method in hospitalized and institutionalized patients. Copyright © 2017 Elsevier España, S.L.U. All rights reserved.

The Computerized Perceptual Motor Skills Assessment: A new visual perceptual motor skills evaluation tool for children in early elementary grades.

PubMed

Howe, Tsu-Hsin; Chen, Hao-Ling; Lee, Candy Chieh; Chen, Ying-Dar; Wang, Tien-Ni

2017-10-01

Visual perceptual motor skills have been proposed as underlying courses of handwriting difficulties. However, there is no evaluation tool currently available to assess these skills comprehensively and to serve as a sensitive measure. The purpose of this study was to validate the Computerized Perceptual Motor Skills Assessment (CPMSA), a newly developed evaluation tool for children in early elementary grades. Its test-retest reliability, concurrent validity, discriminant validity, and responsiveness were examined in 43 typically developing children and 26 children with handwriting difficulty. The CPMSA demonstrated excellent reliability across all subtests with intra-class correlation coefficients (ICCs)≥0.80. Significant moderate correlations between the domains of the CPMSA and corresponding gold standards including Beery VMI, the TVPS-3, and the eye-hand coordination subtest of the DTVP-2 demonstrated good concurrent validity. In addition, the CPMSA showed evidence of discriminant validity in samples of children with and without handwriting difficulty. This article provides evidence in support of the CPMSA. The CPMSA is a reliable, valid, and promising measure of visual perceptual motor skills for children in early elementary grades. Directions for future study and improvements to the assessment are discussed. Copyright © 2017. Published by Elsevier Ltd.

Measuring patient-provider communication skills in Rwanda: Selection, adaptation and assessment of psychometric properties of the Communication Assessment Tool.

PubMed

Cubaka, Vincent Kalumire; Schriver, Michael; Vedsted, Peter; Makoul, Gregory; Kallestrup, Per

2018-04-23

To identify, adapt and validate a measure for providers' communication and interpersonal skills in Rwanda. After selection, translation and piloting of the measure, structural validity, test-retest reliability, and differential item functioning were assessed. Identification and adaptation: The 14-item Communication Assessment Tool (CAT) was selected and adapted. Content validation found all items highly relevant in the local context except two, which were retained upon understanding the reasoning applied by patients. Eleven providers and 291 patients were involved in the field-testing. Confirmatory factor analysis showed a good fit for the original one factor model. Test-retest reliability assessment revealed a mean quadratic weighted Kappa = 0.81 (range: 0.69-0.89, N = 57). The average proportion of excellent scores was 15.7% (SD: 24.7, range: 9.9-21.8%, N = 180). Differential item functioning was not observed except for item 1, which focuses on greetings, for age groups (p = 0.02, N = 180). The Kinyarwanda version of CAT (K-CAT) is a reliable and valid patient-reported measure of providers' communication and interpersonal skills. K-CAT was validated on nurses and its use on other types of providers may require further validation. K-CAT is expected to be a valuable feedback tool for providers in practice and in training. Copyright © 2018 Elsevier B.V. All rights reserved.

NASA EEE Parts and Advanced Interconnect Program (AIP)

NASA Technical Reports Server (NTRS)

Gindorf, T.; Garrison, A.

1996-01-01

none given From Program Objectives: I. Accelerate the readiness of new technologies through development of validation, assessment and test method/tools II. Provide NASA Projects infusion paths for emerging technologies III. Provide NASA Projects technology selection, application and validation guidelines for harware and processes IV. Disseminate quality assurance, reliability, validation, tools and availability information to the NASA community.

Measuring assessment standards in undergraduate medical programs: Development and validation of AIM tool.

PubMed

Sajjad, Madiha; Khan, Rehan Ahmed; Yasmeen, Rahila

2018-01-01

To develop a tool to evaluate faculty perceptions of assessment quality in an undergraduate medical program. The Assessment Implementation Measure (AIM) tool was developed by a mixed method approach. A preliminary questionnaire developed through literature review was submitted to a panel of 10 medical education experts for a three-round 'Modified Delphi technique'. Panel agreement of > 75% was considered the criterion for inclusion of items in the questionnaire. Cognitive pre-testing of five faculty members was conducted. Pilot study was done with 30 randomly selected faculty members. Content validity index (CVI) was calculated for individual items (I-CVI) and composite scale (S-CVI). Cronbach's alpha was calculated to determine the internal consistency reliability of the tool. The final AIM tool had 30 items after the Delphi process. S-CVI was 0.98 with the S-CVI/Avg method and 0.86 by S-CVI/UA method, suggesting good content validity. Cut-off value of < 0.9 I-CVI was taken as criterion for item deletion. Cognitive pre-testing revealed good item interpretation. Cronbach's alpha calculated for the AIM was 0.9, whereas Cronbach's alpha for the four domains ranged from 0.67 to 0.80. 'AIM' is a relevant and useful instrument with good content validity and reliability of results, and may be used to evaluate the teachers´ perceptions about assessment quality.

Measurement properties of tools used to assess suicidality in autistic and general population adults: A systematic review.

PubMed

Cassidy, S A; Bradley, L; Bowen, E; Wigham, S; Rodgers, J

2018-05-05

Adults diagnosed with autism are at significantly increased risk of suicidal thoughts, suicidal behaviours and dying by suicide. However, it is unclear whether any validated tools are currently available to effectively assess suicidality in autistic adults in research and clinical practice. This is crucial for understanding and preventing premature death by suicide in this vulnerable group. This two stage systematic review therefore aimed to identify tools used to assess suicidality in autistic and general population adults, evaluate these tools for their appropriateness and measurement properties, and make recommendations for appropriate selection of suicidality assessment tools in research and clinical practice. Three databases were searched (PsycInfo, Medline and Web of Knowledge). Four frequently used suicidality assessment tools were identified, and subsequently rated for quality of the evidence in support of their measurement properties using the COSMIN checklist. Despite studies having explored suicidality in autistic adults, none had utilised a validated tool. Overall, there was lack of evidence in support of suicidality risk assessments successfully predicting future suicide attempts. We recommend adaptations to current suicidality assessment tools and priorities for future research, in order to better conceptualise suicidality and its measurement in autism. Copyright © 2018 The Authors. Published by Elsevier Ltd.. All rights reserved.

Is a sphygmomanometer a valid and reliable tool to measure the isometric strength of hip muscles? A systematic review.

PubMed

Toohey, Liam Anthony; De Noronha, Marcos; Taylor, Carolyn; Thomas, James

2015-02-01

Muscle strength measurement is a key component of physiotherapists' assessment and is frequently used as an outcome measure. A sphygmomanometer is an instrument commonly used to measure blood pressure that can be potentially used as a tool to assess isometric muscle strength. To systematically review the evidence on the reliability and validity of a sphygmomanometer for measuring isometric strength of hip muscles. A literature search was conducted across four databases. Studies were eligible if they presented data on reliability and/or validity, used a sphygmomanometer to measure isometric muscle strength of the hip region, and were peer reviewed. The individual studies were evaluated for quality using a standardized critical appraisal tool. A total of 644 articles were screened for eligibility, with five articles chosen for inclusion. The use of a sphygmomanometer to objectively assess isometric muscle strength of the hip muscles appears to be reliable with intraclass correlation coefficient values ranging from 0.66 to 0.94 in elderly and young populations. No studies were identified that have assessed the validity of a sphygmomanometer. The sphygmomanometer appears to be reliable for assessment of isometric muscle strength around the hip joint, but further research is warranted to establish its validity.

[SCREENING OF NUTRITIONAL STATUS AMONG ELDERLY PEOPLE AT FAMILY MEDICINE].

PubMed

Račić, M; Ivković, N; Kusmuk, S

2015-11-01

The prevalence of malnutrition in elderly is high. Malnutrition or risk of malnutrition can be detected by use of nutritional screening or assessment tools. This systematic review aimed to identify tools that would be reliable, valid, sensitive and specific for nutritional status screening in patients older than 65 at family medicine. The review was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Studies were retrieved using MEDLINE (via Ovid), PubMed and Cochrane Library electronic databases and by manual searching of relevant articles listed in reference list of key publications. The electronic databases were searched using defined key words adapted to each database and using MESH terms. Manual revision of reviews and original articles was performed using Electronic Journals Library. Included studies involved development and validation of screening tools in the community-dwelling elderly population. The tools, subjected to validity and reliability testing for use in the community-dwelling elderly population were Mini Nutritional Assessment (MNA), Mini Nutritional Assessment-Short Form (MNA-SF), Nutrition Screening Initiative (NSI), which includes DETERMINE list, Level I and II Screen, Seniors in the Community: Risk Evaluation for Eating, and Nutrition (SCREEN I and SCREEN II), Subjective Global Assessment (SGA), Nutritional Risk Index (NRI), and Malaysian and South African tool. MNA and MNA-SF appear to have highest reliability and validity for screening of community-dwelling elderly, while the reliability and validity of SCREEN II are good. The authors conclude that whilst several tools have been developed, most have not undergone extensive testing to demonstrate their ability to identify nutritional risk. MNA and MNA-SF have the highest reliability and validity for screening of nutritional status in the community-dwelling elderly, and the reliability and validity of SCREEN II are satisfactory. These instruments also contain all three nutritional status indicators and are practical for use in family medicine. However, the gold standard for screening cannot be set because testing of reliability and continuous validation in the study with a higher level of evidence need to be conducted in family medicine.

Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study.

PubMed

Morris, Christopher; Hoogenes, Jen; Shayegan, Bobby; Matsumoto, Edward D

2017-01-01

As urology training shifts toward competency-based frameworks, the need for tools for high stakes assessment of trainees is crucial. Validated assessment metrics are lacking for many robot-assisted radical prostatectomy (RARP). As it is quickly becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment tool for training is timely. We recruited 13 expert RARP surgeons from the United States and Canada to serve as our Delphi panel. Using an initial inventory developed via a modified Delphi process with urology residents, fellows, and staff at our institution, panelists iteratively rated each step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited for each item to determine proper step placement, wording, and suggestions. Panelist's responses were compiled and the inventory was edited through three iterations, after which 100% consensus was achieved. The initial inventory steps were decreased by 13% and a skip pattern was incorporated. The final RARP stepwise inventory was comprised of 13 critical steps with 52 sub-steps. There was no attrition throughout the Delphi process. Our Delphi study resulted in a comprehensive inventory of intraoperative RARP steps with excellent consensus. This final inventory will be used to develop a valid and psychometrically sound intraoperative assessment tool for use during RARP training and evaluation, with the aim of increasing competency of all trainees. Copyright® by the International Brazilian Journal of Urology.

Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study

PubMed Central

Morris, Christopher; Hoogenes, Jen; Shayegan, Bobby; Matsumoto, Edward D.

2017-01-01

ABSTRACT Introduction As urology training shifts toward competency-based frameworks, the need for tools for high stakes assessment of trainees is crucial. Validated assessment metrics are lacking for many robot-assisted radical prostatectomy (RARP). As it is quickly becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment tool for training is timely. Materials and methods We recruited 13 expert RARP surgeons from the United States and Canada to serve as our Delphi panel. Using an initial inventory developed via a modified Delphi process with urology residents, fellows, and staff at our institution, panelists iteratively rated each step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited for each item to determine proper step placement, wording, and suggestions. Results Panelist’s responses were compiled and the inventory was edited through three iterations, after which 100% consensus was achieved. The initial inventory steps were decreased by 13% and a skip pattern was incorporated. The final RARP stepwise inventory was comprised of 13 critical steps with 52 sub-steps. There was no attrition throughout the Delphi process. Conclusions Our Delphi study resulted in a comprehensive inventory of intraoperative RARP steps with excellent consensus. This final inventory will be used to develop a valid and psychometrically sound intraoperative assessment tool for use during RARP training and evaluation, with the aim of increasing competency of all trainees. PMID:28379668

Assessment and classification of cancer breakthrough pain: a systematic literature review.

PubMed

Haugen, Dagny Faksvåg; Hjermstad, Marianne Jensen; Hagen, Neil; Caraceni, Augusto; Kaasa, Stein

2010-06-01

Temporal variations in cancer pain intensity are highly prevalent, and are often difficult to manage. However, the phenomenon is not well understood: several definitions and approaches to classification and bedside assessment of cancer breakthrough pain (BTP) have been described. The present study is a systematic review of published literature on cancer BTP to answer the following questions: which terms and definitions have been used; are there validated assessment tools; which domains of BTP do the tools delineate, and which items do they contain; how have assessment tools been applied within clinical studies; and are there validated classification systems for BTP. A systematic search of the peer-reviewed literature was performed using five major databases. Of 375 titles and abstracts initially identified, 51 articles were examined in detail. Analysis of these publications indicates a range of overlapping but distinct definitions have been used to characterize BTP; 42 of the included papers presented one or more ways of classifying BTP; and while 10 tools to assess patients' experience of BTP were identified, only 2 have been partially validated. We conclude that there is no widely accepted definition, classification system or well-validated assessment tool for cancer-related breakthrough pain, but there is strong concurrence on most of its key attributes. With further work in this area, an internationally agreed upon definition and classification system for cancer-related breakthrough pain, and a standard approach on how to measure it, hold the promise to improve patient care and support research in this poor-prognosis cancer pain syndrome.

School-based behavioral assessment tools are reliable and valid for measurement of fruit and vegetable intake, physical activity, and television viewing in young children.

PubMed

Economos, Christina D; Sacheck, Jennifer M; Kwan Ho Chui, Kenneth; Irizarry, Laura; Irizzary, Laura; Guillemont, Juliette; Collins, Jessica J; Hyatt, Raymond R

2008-04-01

Interventions aiming to modify the dietary and physical activity behaviors of young children require precise and accurate measurement tools. As part of a larger community-based project, three school-based questionnaires were developed to assess (a) fruit and vegetable intake, (b) physical activity and television (TV) viewing, and (c) perceived parental support for diet and physical activity. Test-retest reliability was performed on all questionnaires and validity was measured for fruit and vegetable intake, physical activity, and TV viewing. Eighty-four school children (8.3+/-1.1 years) were studied. Test-retest reliability was performed by administering questionnaires twice, 1 to 2 hours apart. Validity of the fruit and vegetable questionnaire was measured by direct observation, while the physical activity and TV questionnaire was validated by a parent phone interview. All three questionnaires yielded excellent test-retest reliability (P<0.001). The majority of fruit and vegetable questions and the questions regarding specific physical activities and TV viewing were valid. Low validity scores were found for questions on watching TV during breakfast or dinner. These questionnaires are reliable and valid tools to assess fruit and vegetable intake, physical activity, and TV viewing behaviors in early elementary school-aged children. Methods for assessment of children's TV viewing during meals should be further investigated because of parent-child discrepancies.

School environment assessment tools to address behavioural risk factors of non-communicable diseases: A scoping review.

PubMed

Saluja, Kiran; Rawal, Tina; Bassi, Shalini; Bhaumik, Soumyadeep; Singh, Ankur; Park, Min Hae; Kinra, Sanjay; Arora, Monika

2018-06-01

We aimed to identify, describe and analyse school environment assessment (SEA) tools that address behavioural risk factors (unhealthy diet, physical inactivity, tobacco and alcohol consumption) for non-communicable diseases (NCD). We searched in MEDLINE and Web of Science, hand-searched reference lists and contacted experts. Basic characteristics, measures assessed and measurement properties (validity, reliability, usability) of identified tools were extracted. We narratively synthesized the data and used content analysis to develop a list of measures used in the SEA tools. Twenty-four SEA tools were identified, mostly from developed countries. Out of these, 15 were questionnaire based, 8 were checklists or observation based tools and one tool used a combined checklist/observation based and telephonic questionnaire approach. Only 1 SEA tool had components related to all the four NCD risk factors, 2 SEA tools has assessed three NCD risk factors (diet/nutrition, physical activity, tobacco), 10 SEA tools has assessed two NCD risk factors (diet/nutrition and physical activity) and 11 SEA tools has assessed only one of the NCD risk factor. Several measures were used in the tools to assess the four NCD risk factors, but tobacco and alcohol was sparingly included. Measurement properties were reported for 14 tools. The review provides a comprehensive list of measures used in SEA tools which could be a valuable resource to guide future development of such tools. A valid and reliable SEA tool which could simultaneously evaluate all NCD risk factors, that has been tested in different settings with varying resource availability is needed.

A Turkish Version of the Critical-Care Pain Observation Tool: Reliability and Validity Assessment.

PubMed

Aktaş, Yeşim Yaman; Karabulut, Neziha

2017-08-01

The study aim was to evaluate the validity and reliability of the Critical-Care Pain Observation Tool in critically ill patients. A repeated measures design was used for the study. A convenience sample of 66 patients who had undergone open-heart surgery in the cardiovascular surgery intensive care unit in Ordu, Turkey, was recruited for the study. The patients were evaluated by using the Critical-Care Pain Observation Tool at rest, during a nociceptive procedure (suctioning), and 20 minutes after the procedure while they were conscious and intubated after surgery. The Turkish version of the Critical-Care Pain Observation Tool has shown statistically acceptable levels of validity and reliability. Inter-rater reliability was supported by moderate-to-high-weighted κ coefficients (weighted κ coefficient = 0.55 to 1.00). For concurrent validity, significant associations were found between the scores on the Critical-Care Pain Observation Tool and the Behavioral Pain Scale scores. Discriminant validity was also supported by higher scores during suctioning (a nociceptive procedure) versus non-nociceptive procedures. The internal consistency of the Critical-Care Pain Observation Tool was 0.72 during a nociceptive procedure and 0.71 during a non-nociceptive procedure. The validity and reliability of the Turkish version of the Critical-Care Pain Observation Tool was determined to be acceptable for pain assessment in critical care, especially for patients who cannot communicate verbally. Copyright © 2016 American Society of PeriAnesthesia Nurses. Published by Elsevier Inc. All rights reserved.

The Arabic version of the Mayo-Portland Adaptability Inventory 4: a validation study.

PubMed

Hamed, Razan; Tariah, Hashem Abu; Malkawi, Somaya; Holm, Margo B

2012-09-01

The Mayo-Portland Adaptability Inventory 4 (MPAI-4) is a valid and reliable assessment tool to detect clinical impairments in patients with acquired brain injury. The tool is widely used by rehabilitation therapists worldwide, given its good psychometric properties and its availability in several languages. The purpose of this study was to translate the tool into Arabic and to examine its validity and reliability with multiple sclerosis and stroke patients. A total of 128 participants were enrolled in this study: 49 with multiple sclerosis, 17 with stroke, and 62 healthy adults. The psychometric properties of discriminative and convergent construct validity as well as test-retest reliability were tested. The translated tool, the Arabic-MPAI-4 (A-MPAI-4), significantly discriminated among the three subgroups (F=50.93, P<0.001), correlated moderately but significantly with the Arabic version of the Performance Assessment of Self-Care Skills Self-Report as a measure of functional independence in daily activities (r=-0.35, P<0.001), and showed good stability over time (r=0.73, P<0.001). The A-MPAI-4 is a valid and reliable tool for clinical use with multiple sclerosis and stroke patients who speak Arabic.

Validation of Caregiver-Centered Delirium Detection Tools: A Systematic Review.

PubMed

Rosgen, Brianna; Krewulak, Karla; Demiantschuk, Danielle; Ely, E Wesley; Davidson, Judy E; Stelfox, Henry T; Fiest, Kirsten M

2018-04-18

To summarize the validity of caregiver-centered delirium detection tools in hospitalized adults and assess associated patient and caregiver outcomes. Systematic review. We searched MEDLINE, EMBASE, PsycINFO, CINAHL, and Scopus from inception to May 15, 2017. Hospitalized adults. Caregiver-centered delirium detection tools. We drafted a protocol from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Two reviewers independently completed abstract and full-text review, data extraction, and quality assessment. We summarized findings using descriptive statistics including mean, median, standard deviation, range, frequencies (percentages), and Cohen's kappa. Studies that reported on the validity of caregiver-centered delirium detection tools or associated patient and caregiver outcomes and were cohort or cross-sectional in design were included. We reviewed 6,056 titles and abstracts, included 6 articles, and identified 6 caregiver-centered tools. All tools were designed to be administered in several minutes or less and had 11 items or fewer. Three tools were caregiver administered (completed independently by caregivers): Family Confusion Assessment Method (FAM-CAM), Informant Assessment of Geriatric Delirium (I-AGeD), and Sour Seven. Three tools were caregiver informed (administered by a healthcare professional using caregiver input): Single Question in Delirium (SQiD), Single Screening Question Delirium (SSQ-Delirium), and Stressful Caregiving Response to Experiences of Dying. Caregiver-administered tools had better psychometric properties (FAM-CAM sensitivity 75%, 95% confidence interval (CI)=35-95%, specificity 91%, 95% CI=74-97%; Sour Seven positive predictive value 89.5%, negative predictive value 90%) than caregiver-informed tools (SQiD: sensitivity 80%, 95% CI=28.4-99.5%; specificity 71%, 95% CI=41.9-91.6%; SSQ-Delirium sensitivity 79.6%, specificity 56.1%). Delirium detection is essential for appropriate delirium management. Caregiver-centered delirium detection tools show promise in improving delirium detection and associated patient and caregiver outcomes. Comparative studies using larger sample sizes and multiple centers are required to determine validity and reliability characteristics. © 2018, Copyright the Authors Journal compilation © 2018, The American Geriatrics Society.

NASA Aerospace Flight Battery Systems Program Update

NASA Technical Reports Server (NTRS)

Manzo, Michelle; ODonnell, Patricia

1997-01-01

The objectives of NASA's Aerospace Flight Battery Systems Program is to: develop, maintain and provide tools for the validation and assessment of aerospace battery technologies; accelerate the readiness of technology advances and provide infusion paths for emerging technologies; provide NASA projects with the required database and validation guidelines for technology selection of hardware and processes relating to aerospace batteries; disseminate validation and assessment tools, quality assurance, reliability, and availability information to the NASA and aerospace battery communities; and ensure that safe, reliable batteries are available for NASA's future missions.

Evaluation in industry of a draft code of practice for manual handling.

PubMed

Ashby, Liz; Tappin, David; Bentley, Tim

2004-05-01

This paper reports findings from a study which evaluated the draft New Zealand Code of Practice for Manual Handling. The evaluation assessed the ease of use, applicability and validity of the Code and in particular the associated manual handling hazard assessment tools, within New Zealand industry. The Code was studied in a sample of eight companies from four sectors of industry. Subjective feedback and objective findings indicated that the Code was useful, applicable and informative. The manual handling hazard assessment tools incorporated in the Code could be adequately applied by most users, with risk assessment outcomes largely consistent with the findings of researchers using more specific ergonomics methodologies. However, some changes were recommended to the risk assessment tools to improve usability and validity. The evaluation concluded that both the Code and the tools within it would benefit from simplification, improved typography and layout, and industry-specific information on manual handling hazards.

Validity and Reliability of Wii Fit Balance Board for the Assessment of Balance of Healthy Young Adults and the Elderly

PubMed Central

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-01-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86–0.99) for the elderly people and positive correlations (r = 0.58–0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability. PMID:24259769

Measuring pain in patients undergoing hemodialysis: a review of pain assessment tools

PubMed Central

Upadhyay, Chandani; Cameron, Karen; Murphy, Laura; Battistella, Marisa

2014-01-01

Background Patients undergoing hemodialysis frequently report pain with multifactorial causes, not limited to that experienced directly from hemodialysis treatment. Their pain may be nociceptive, neuropathic, somatic or visceral in nature. Despite this, pain in this population remains under-recognized and under-treated. Although several tools have been used to measure pain in patients undergoing hemodialysis as reported in the literature, none of them have been validated specifically in this population. The objective for this review was to compare and contrast these pain assessment tools and discuss their clinical utility in this patient population. Methods To identify pain assessment tools studied in patients undergoing hemodialysis, a literature search was performed in PubMed and Medline. An expert panel of dialysis and pain clinicians reviewed each tool. Each pain assessment tool was assessed on how it is administered and scored, its psychometric properties such as reliability, validity and responsiveness to change, and its clinical utility in a hemodialysis population. Brief Pain Inventory, McGill Pain Questionnaire, Pain Management Index, Edmonton Symptom Assessment System, Visual Analogue Scale and Faces Pain Scale were evaluated and compared. Results This assessment will help clinicians practicing in nephrology to determine which of these pain assessment tools is best suited for use in their individual clinical practice. PMID:25852910

Family-Based Benchmarking of Copy Number Variation Detection Software.

PubMed

Nutsua, Marcel Elie; Fischer, Annegret; Nebel, Almut; Hofmann, Sylvia; Schreiber, Stefan; Krawczak, Michael; Nothnagel, Michael

2015-01-01

The analysis of structural variants, in particular of copy-number variations (CNVs), has proven valuable in unraveling the genetic basis of human diseases. Hence, a large number of algorithms have been developed for the detection of CNVs in SNP array signal intensity data. Using the European and African HapMap trio data, we undertook a comparative evaluation of six commonly used CNV detection software tools, namely Affymetrix Power Tools (APT), QuantiSNP, PennCNV, GLAD, R-gada and VEGA, and assessed their level of pair-wise prediction concordance. The tool-specific CNV prediction accuracy was assessed in silico by way of intra-familial validation. Software tools differed greatly in terms of the number and length of the CNVs predicted as well as the number of markers included in a CNV. All software tools predicted substantially more deletions than duplications. Intra-familial validation revealed consistently low levels of prediction accuracy as measured by the proportion of validated CNVs (34-60%). Moreover, up to 20% of apparent family-based validations were found to be due to chance alone. Software using Hidden Markov models (HMM) showed a trend to predict fewer CNVs than segmentation-based algorithms albeit with greater validity. PennCNV yielded the highest prediction accuracy (60.9%). Finally, the pairwise concordance of CNV prediction was found to vary widely with the software tools involved. We recommend HMM-based software, in particular PennCNV, rather than segmentation-based algorithms when validity is the primary concern of CNV detection. QuantiSNP may be used as an additional tool to detect sets of CNVs not detectable by the other tools. Our study also reemphasizes the need for laboratory-based validation, such as qPCR, of CNVs predicted in silico.

Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit.

PubMed

Karnoe, Astrid; Furstrand, Dorthe; Christensen, Karl Bang; Norgaard, Ole; Kayser, Lars

2018-05-10

To achieve full potential in user-oriented eHealth projects, we need to ensure a match between the eHealth technology and the user's eHealth literacy, described as knowledge and skills. However, there is a lack of multifaceted eHealth literacy assessment tools suitable for screening purposes. The objective of our study was to develop and validate an eHealth literacy assessment toolkit (eHLA) that assesses individuals' health literacy and digital literacy using a mix of existing and newly developed scales. From 2011 to 2015, scales were continuously tested and developed in an iterative process, which led to 7 tools being included in the validation study. The eHLA validation version consisted of 4 health-related tools (tool 1: "functional health literacy," tool 2: "health literacy self-assessment," tool 3: "familiarity with health and health care," and tool 4: "knowledge of health and disease") and 3 digitally-related tools (tool 5: "technology familiarity," tool 6: "technology confidence," and tool 7: "incentives for engaging with technology") that were tested in 475 respondents from a general population sample and an outpatient clinic. Statistical analyses examined floor and ceiling effects, interitem correlations, item-total correlations, and Cronbach coefficient alpha (CCA). Rasch models (RM) examined the fit of data. Tools were reduced in items to secure robust tools fit for screening purposes. Reductions were made based on psychometrics, face validity, and content validity. Tool 1 was not reduced in items; it consequently consists of 10 items. The overall fit to the RM was acceptable (Anderson conditional likelihood ratio, CLR=10.8; df=9; P=.29), and CCA was .67. Tool 2 was reduced from 20 to 9 items. The overall fit to a log-linear RM was acceptable (Anderson CLR=78.4, df=45, P=.002), and CCA was .85. Tool 3 was reduced from 23 to 5 items. The final version showed excellent fit to a log-linear RM (Anderson CLR=47.7, df=40, P=.19), and CCA was .90. Tool 4 was reduced from 12 to 6 items. The fit to a log-linear RM was acceptable (Anderson CLR=42.1, df=18, P=.001), and CCA was .59. Tool 5 was reduced from 20 to 6 items. The fit to the RM was acceptable (Anderson CLR=30.3, df=17, P=.02), and CCA was .94. Tool 6 was reduced from 5 to 4 items. The fit to a log-linear RM taking local dependency (LD) into account was acceptable (Anderson CLR=26.1, df=21, P=.20), and CCA was .91. Tool 7 was reduced from 6 to 4 items. The fit to a log-linear RM taking LD and differential item functioning into account was acceptable (Anderson CLR=23.0, df=29, P=.78), and CCA was .90. The eHLA consists of 7 short, robust scales that assess individual's knowledge and skills related to digital literacy and health literacy. ©Astrid Karnoe, Dorthe Furstrand, Karl Bang Christensen, Ole Norgaard, Lars Kayser. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 10.05.2018.

Methodological challenges of validating a clinical decision-making tool in the practice environment.

PubMed

Brennan, Caitlin W; Daly, Barbara J

2015-04-01

Validating a measurement tool intended for use in the practice environment poses challenges that may not be present when validating a tool intended solely for research purposes. The aim of this article is to describe the methodological challenges of validating a clinical decision-making tool, the Oncology Acuity Tool, which nurses use to make nurse assignment and staffing decisions prospectively each shift. Data were derived from a larger validation study, during which several methodological challenges arose. Revisions to the tool, including conducting iterative feedback cycles with end users, were necessary before the validation study was initiated. The "true" value of patient acuity is unknown, and thus, two approaches to inter-rater reliability assessment were used. Discordant perspectives existed between experts and end users. Balancing psychometric rigor with clinical relevance may be achieved through establishing research-practice partnerships, seeking active and continuous feedback with end users, and weighing traditional statistical rules of thumb with practical considerations. © The Author(s) 2014.

Embedded performance validity testing in neuropsychological assessment: Potential clinical tools.

PubMed

Rickards, Tyler A; Cranston, Christopher C; Touradji, Pegah; Bechtold, Kathleen T

2018-01-01

The article aims to suggest clinically-useful tools in neuropsychological assessment for efficient use of embedded measures of performance validity. To accomplish this, we integrated available validity-related and statistical research from the literature, consensus statements, and survey-based data from practicing neuropsychologists. We provide recommendations for use of 1) Cutoffs for embedded performance validity tests including Reliable Digit Span, California Verbal Learning Test (Second Edition) Forced Choice Recognition, Rey-Osterrieth Complex Figure Test Combination Score, Wisconsin Card Sorting Test Failure to Maintain Set, and the Finger Tapping Test; 2) Selecting number of performance validity measures to administer in an assessment; and 3) Hypothetical clinical decision-making models for use of performance validity testing in a neuropsychological assessment collectively considering behavior, patient reporting, and data indicating invalid or noncredible performance. Performance validity testing helps inform the clinician about an individual's general approach to tasks: response to failure, task engagement and persistence, compliance with task demands. Data-driven clinical suggestions provide a resource to clinicians and to instigate conversation within the field to make more uniform, testable decisions to further the discussion, and guide future research in this area.

Development and validation of the Learning Disabilities Needs Assessment Tool (LDNAT), a HoNOS-based needs assessment tool for use with people with intellectual disability.

PubMed

Painter, J; Trevithick, L; Hastings, R P; Ingham, B; Roy, A

2016-12-01

In meeting the needs of individuals with intellectual disabilities (ID) who access health services, a brief, holistic assessment of need is useful. This study outlines the development and testing of the Learning Disabilities Needs Assessment Tool (LDNAT), a tool intended for this purpose. An existing mental health (MH) tool was extended by a multidisciplinary group of ID practitioners. Additional scales were drafted to capture needs across six ID treatment domains that the group identified. LDNAT ratings were analysed for the following: item redundancy, relevance, construct validity and internal consistency (n = 1692); test-retest reliability (n = 27); and concurrent validity (n = 160). All LDNAT scales were deemed clinically relevant with little redundancy apparent. Principal component analysis indicated three components (developmental needs, challenging behaviour, MH and well-being). Internal consistency was good (Cronbach alpha 0.80). Individual item test-retest reliability was substantial-near perfect for 20 scales and slight-fair for three scales. Overall reliability was near perfect (intra-class correlation = 0.91). There were significant associations with five of six condition-specific measures, i.e. the Waisman Activities of Daily Living Scale (general ability/disability), Threshold Assessment Grid (risk), Behaviour Problems Inventory for Individuals with Intellectual Disabilities-Short Form (challenging behaviour) Social Communication Questionnaire (autism) and a bespoke physical health questionnaire. Additionally, the statistically significant correlations between these tools and the LDNAT components made sense clinically. There were no statistically significant correlations with the Psychiatric Assessment Schedules for Adults with Developmental Disabilities (a measure of MH symptoms in people with ID). The LDNAT had clinically utility when rating the needs of people with ID prior to condition-specific assessment(s). Analyses of internal and external validity were promising. Further evaluation of its sensitivity to changes in needs is now required. © 2016 MENCAP and International Association of the Scientific Study of Intellectual and Developmental Disabilities and John Wiley & Sons Ltd.

Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training.

PubMed

Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

2014-03-01

Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists⁄surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The 'Endoscopy Task Load Index' was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers.

Appraising the risk matrix 2000 static sex offender risk assessment tool.

PubMed

Tully, Ruth J; Browne, Kevin D

2015-02-01

This critical appraisal explores the reliability and validity of the Risk Matrix 2000 static sex offender risk assessment tool that is widely used in the United Kingdom. The Risk Matrix 2000 has to some extent been empirically validated for use with adult male sex offenders; however, this review highlights that further research into the validity of this static tool with sex offender subgroups or types is necessary in order to improve practical utility. The Risk Matrix 2000 relies on static risk predictors, thus it is limited in scope. This article argues that the addition of dynamic items that have been shown to be predictive of sexual recidivism would further enhance the tool. The paper argues that adding dynamic risk items would fit better with a rehabilitative approach to sex offender risk management and assessment. This would also provide a means by which to effectively plan sex offender treatment and evaluate individual offenders' progress in treatment; however, difficulties remain in identifying and assessing dynamic risk factors of sexual offending and so further research is required. © The Author(s) 2013.

Pain assessment tools: is the content appropriate for use in palliative care?

PubMed

Hølen, Jacob Chr; Hjermstad, Marianne Jensen; Loge, Jon Håvard; Fayers, Peter M; Caraceni, Augusto; De Conno, Franco; Forbes, Karen; Fürst, Carl Johan; Radbruch, Lukas; Kaasa, Stein

2006-12-01

Inadequate pain assessment prevents optimal treatment in palliative care. The content of pain assessment tools might limit their usefulness for proper pain assessment, but data on the content validity of the tools are scarce. The objective of this study was to examine the content of the existing pain assessment tools, and to evaluate the appropriateness of different dimensions and items for pain assessment in palliative care. A systematic search was performed to find pain assessment tools for patients with advanced cancer who were receiving palliative care. An ad hoc search with broader search criteria supplemented the systematic search. The items of the identified tools were allocated to appropriate dimensions. This was reviewed by an international panel of experts, who also evaluated the relevance of the different dimensions for pain assessment in palliative care. The systematic literature search generated 16 assessment tools while the ad hoc search generated 64. Ten pain dimensions containing 1,011 pain items were identified by the experts. The experts ranked intensity, temporal pattern, treatment and exacerbating/relieving factors, location, and interference with health-related quality of life as the most important dimensions. None of the assessment tools covered these dimensions satisfactorily. Most items were related to interference (231) and intensity (138). Temporal pattern (which includes breakthrough pain), ranked as the second most important dimension, was covered by 29 items only. Many tools include dimensions and items of limited relevance for patients with advanced cancer. This might reduce compliance and threaten the validity of the assessment. New tools should reflect the clinical relevance of different dimensions and be user-friendly.

Psychometric properties of the Peer Proficiency Assessment (PEPA): a tool for evaluation of undergraduate peer counselors' motivational interviewing fidelity.

PubMed

Mastroleo, Nadine R; Mallett, Kimberly A; Turrisi, Rob; Ray, Anne E

2009-09-01

Despite the expanding use of undergraduate student peer counseling interventions aimed at reducing college student drinking, few programs evaluate peer counselors' competency to conduct these interventions. The present research describes the development and psychometric assessments of the Peer Proficiency Assessment (PEPA), a new tool for examining Motivational Interviewing adherence in undergraduate student peer delivered interventions. Twenty peer delivered sessions were evaluated by master and undergraduate student coders using a cross-validation design to examine peer based alcohol intervention sessions. Assessments revealed high inter-rater reliability between student and master coders and good correlations between previously established fidelity tools. Findings lend support for the use of the PEPA to examine peer counselor competency. The PEPA, training for use, inter-rater reliability information, construct and predictive validity, and tool usefulness are described.

Furthering the Validity of a Tool to Assess Simulated Pregnancy Options Counseling Skills.

PubMed

Lupi, Carla; Ward-Peterson, Melissa; Coxe, Stefany; Minor, Suzanne; Eliacin, Irmanie; Obeso, Vivian

2016-10-01

To further the validity of a tool to assess nondirective pregnancy options counseling skills. Using a cross-sectional design, we explored four sources of construct validity evidence for an objective structured clinical examination for training and assessment of nondirective pregnancy options counseling: content, response process, internal structure, and relations to other variables. Content of the previously developed tool was enhanced through input from five family medicine educators. The objective structured clinical examination was implemented in a family medicine clerkship with third-year medical students from 2014 to 2015 using trained raters. Response process was addressed after a pilot round. Three new raters evaluated videotapes of 46 performances. Cronbach's alpha, intraclass correlation coefficients, and Spearman's rho were estimated with 95% confidence intervals. The content validity was affirmed. Cronbach's alpha was 0.71. According to Landis and Koch's criteria, all but two items unique to the clinical situation of pregnancy options counseling generated substantial to perfect agreement (0.62-1.00). Relations to other variables within the checklist were strong, ranging from 0.66 to 0.87. This tool for assessing pregnancy options counseling skills has excellent content and strong internal structure. Further work to improve the Global Rating Scale may be necessary for summative use.

The application of health literacy measurement tools (collective or individual domains) in assessing chronic disease management: a systematic review protocol.

PubMed

Shum, Jessica; Poureslami, Iraj; Doyle-Waters, Mary M; FitzGerald, J Mark

2016-06-07

The term "health literacy" (HL) was first coined in 1974, and its most common definition is currently defined as a person's ability to access, understand, evaluate, communicate, and use health information to make decisions for one's health. The previous systematic reviews assessing the effect of existing HL measurement tools on health outcomes have simply searched for the term "health literacy" only to identify measures instead of incorporating either one or more of the five domains in their search. Furthermore, as the domain "use" is fairly new, few studies have actually assessed this domain. In this protocol, we propose to identify and assess HL measures that applied the mentioned five domains either collectively or individually in assessing chronic disease management, in particular for asthma and chronic obstructive pulmonary disease (COPD). The ultimate goal is to provide recommendations towards the development and validation of a patient-centric HL measurement tool for the two diseases. A comprehensive, electronic search will be conducted to identify potential studies dating from 1974 to 2016 from databases such as Embase, MEDLINE, CINAHL, Cochrane Central Register of Controlled Trials, Web of Science, ERIC, PsycINFO, and HAPI. Database searches will be complemented with grey literature. Two independent reviewers will perform tool selection, study selection, data extraction, and quality assessment using pre-designed study forms. Any disagreement will be resolved through discussion or a third reviewer. Only studies that have developed or validated HL measurement tools (including one or more of the five domains mentioned above) among asthma and COPD patients will be included. Information collected from the studies will include instrument details such as versions, purpose, underlying constructs, administration, mapping of items onto the five domains, internal structure, scoring, response processes, standard error of measurement (SEM), correlation with other variables, clinically important difference, and item response theory (IRT)-based analyses. The identified strengths and weaknesses as well as reliability, validity, responsiveness, and interpretability of the tools from the validation studies will also be assessed using the COSMIN checklist. A synthesis will be presented for all tools in relation to asthma and COPD management. This systematic review will be one of several key contributions central to a global evidence-based strategy funded by the Canadian Institutes of Health Research (CIHR) for measuring HL in patients with asthma and COPD, highlighting the gaps and inconsistencies of domains between existing tools. The knowledge generated from this review will provide the team information on (1) the five-domain model and cross domains, (2) underlying constructs, (3) tool length, (4) time for completion, (5) reading level, and (6) format for development of the proposed tool. Other aspects of the published validation studies such as reliability coefficients, SEM, correlations with other variables, clinically important difference, and IRT-based analyses will be important for comparison purposes when testing, interpreting, and validating the developed tool. PROSPERO CRD42016037532.

Gathering Validity Evidence for Surgical Simulation: A Systematic Review.

PubMed

Borgersen, Nanna Jo; Naur, Therese M H; Sørensen, Stine M D; Bjerrum, Flemming; Konge, Lars; Subhi, Yousif; Thomsen, Ann Sofia S

2018-06-01

To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.

The Use of Authentic Assessment to Report Accountability Data on Young Children's Language, Literacy and Pre-Math Competency

ERIC Educational Resources Information Center

Gao, Xin; Grisham-Brown, Jennifer

2011-01-01

This validity study examined the validity of Assessment, Evaluation, and Programming System, 2nd Edition (AEPS®), a curriculum-based, authentic assessment for infants and young children. The primary purposes were to: a) examine whether the AEPS® is a concurrently valid tool for measuring young children's language, literacy and pre-math skills for…

Simulation for Prediction of Entry Article Demise (SPEAD): An Analysis Tool for Spacecraft Safety Analysis and Ascent/Reentry Risk Assessment

NASA Technical Reports Server (NTRS)

Ling, Lisa

2014-01-01

For the purpose of performing safety analysis and risk assessment for a potential off-nominal atmospheric reentry resulting in vehicle breakup, a synthesis of trajectory propagation coupled with thermal analysis and the evaluation of node failure is required to predict the sequence of events, the timeline, and the progressive demise of spacecraft components. To provide this capability, the Simulation for Prediction of Entry Article Demise (SPEAD) analysis tool was developed. The software and methodology have been validated against actual flights, telemetry data, and validated software, and safety/risk analyses were performed for various programs using SPEAD. This report discusses the capabilities, modeling, validation, and application of the SPEAD analysis tool.

Assessing attachment in school-aged children: Do the School-Age Assessment of Attachment and Family Drawings work together as complementary tools?

PubMed

Carr-Hopkins, Rebecca; De Burca, Calem; Aldridge, Felicity A

2017-07-01

Our goal was to identify an assessment package that could improve treatment planning for troubled children and their families. To assess the validity of our tools, we tested the relations among the School-Age Assessment of Attachment, the Family Drawing and children's risk status. We used the Dynamic-Maturational Model of Attachment and Adaptation to interpret the assessments in the hope of identifying a gradient of risk, and explore whether a new coding method improved the validity of Family Drawings and their utility as a tool to complement the School-Age Assessment of Attachment. The participants were 89 children, aged between 5 and 12 years; 32 children were involved with mental health services or child protection. Each child completed a School-Age Assessment of Attachment and a Family Drawing. Both assessments differentiated between clinical and normative referrals with moderate effect sizes when dichotomizing risk versus non-risk attachment. When the analysis incorporated a gradient of six attachment classifications, the effect sizes decreased, but specificity of risk increased. The School-Age Assessment of Attachment had greater validity for discriminating risk, and type of risk, than the Family Drawings. With a School-Age Assessment of Attachment and family history, the Family Drawing can provide information about distress that some children do not provide verbally. Integration of the two assessment tools alongside information about parental and family functioning appears to be the key to formulating children's problems.

Validity of the Nintendo Wii® balance board for the assessment of standing balance in Parkinson's disease.

PubMed

Holmes, Jeffrey D; Jenkins, Mary E; Johnson, Andrew M; Hunt, Michael A; Clark, Ross A

2013-04-01

Impaired postural stability places individuals with Parkinson's at an increased risk for falls. Given the high incidence of fall-related injuries within this population, ongoing assessment of postural stability is important. To evaluate the validity of the Nintendo Wii(®) balance board as a measurement tool for the assessment of postural stability in individuals with Parkinson's. Twenty individuals with Parkinson's participated. Subjects completed testing on two balance tasks with eyes open and closed on a Wii(®) balance board and biomechanical force platform. Bland-Altman plots and a two-way, random-effects, single measure intraclass correlation coefficient model were used to assess concurrent validity of centre-of-pressure data. Concurrent validity was demonstrated to be excellent across balance tasks (intraclass correlation coefficients = 0.96, 0.98, 0.92, 0.94). This study suggests that the Wii(®) balance board is a valid tool for the quantification of postural stability among individuals with Parkinson's.

The development of a multimedia online language assessment tool for young children with autism.

PubMed

Lin, Chu-Sui; Chang, Shu-Hui; Liou, Wen-Ying; Tsai, Yu-Show

2013-10-01

This study aimed to provide early childhood special education professionals with a standardized and comprehensive language assessment tool for the early identification of language learning characteristics (e.g., hyperlexia) of young children with autism. In this study, we used computer technology to develop a multi-media online language assessment tool that presents auditory or visual stimuli. This online comprehensive language assessment consists of six subtests: decoding, homographs, auditory vocabulary comprehension, visual vocabulary comprehension, auditory sentence comprehension, and visual sentence comprehension. Three hundred typically developing children and 35 children with autism from Tao-Yuan County in Taiwan aged 4-6 participated in this study. The Cronbach α values of the six subtests ranged from .64 to .97. The variance explained by the six subtests ranged from 14% to 56%, the current validity of each subtest with the Peabody Picture Vocabulary Test-Revised ranged from .21 to .45, and the predictive validity of each subtest with WISC-III ranged from .47 to .75. This assessment tool was also found to be able to accurately differentiate children with autism up to 92%. These results indicate that this assessment tool has both adequate reliability and validity. Additionally, 35 children with autism have completed the entire assessment in this study without exhibiting any extremely troubling behaviors. However, future research is needed to increase the sample size of both typically developing children and young children with autism and to overcome the technical challenges associated with internet issues. Copyright © 2013 Elsevier Ltd. All rights reserved.

The COA360: a tool for assessing the cultural competency of healthcare organizations.

PubMed

LaVeist, Thomas A; Relosa, Rachel; Sawaya, Nadia

2008-01-01

The U.S. Census Bureau projects that by 2050, non-Hispanic whites will be in the numerical minority. This rapid diversification requires healthcare organizations to pay closer attention to cross-cultural issues if they are to meet the healthcare needs of the nation and continue to maintain a high standard of care. Although scorecards and benchmarking are widely used to gauge healthcare organizations' performance in various areas, these tools have been underused in relation to cultural preparedness or initiatives. The likely reason for this is the lack of a validated tool specifically designed to examine cultural competency. Existing validated cultural competency instruments evaluate individuals, not organizations. In this article, we discuss a study to validate the Cultural Competency Organizational Assessment--360 or the COA360, an instrument designed to appraise a healthcare organization's cultural competence. The Office of Minority Health and the Joint Commission have each developed standards for measuring the cultural competency of organizations. The COA360 is designed to assess adherence to both of these sets of standards. For this validation study, we enlisted a panel of national experts. The panel rated each dimension of the COA360, and the combination of items for each of the scale's 14 dimensions was rated above 4.13 (on 5-point scale). Our conclusion points to the validity of the COA360. As such, it is a valuable tool not only for assessing a healthcare organization's cultural readiness but also for benchmarking its progress in addressing cultural and diversity issues.

Teamwork Assessment Tools in Modern Surgical Practice: A Systematic Review

PubMed Central

Whittaker, George; Abboudi, Hamid; Khan, Muhammed Shamim; Dasgupta, Prokar; Ahmed, Kamran

2015-01-01

Introduction. Deficiencies in teamwork skills have been shown to contribute to the occurrence of adverse events during surgery. Consequently, several teamwork assessment tools have been developed to evaluate trainee nontechnical performance. This paper aims to provide an overview of these instruments and review the validity of each tool. Furthermore, the present paper aims to review the deficiencies surrounding training and propose several recommendations to address these issues. Methods. A systematic literature search was conducted to identify teamwork assessment tools using MEDLINE (1946 to August 2015), EMBASE (1974 to August 2015), and PsycINFO (1806 to August 2015) databases. Results. Eight assessment tools which encompass aspects of teamwork were identified. The Nontechnical Skills for Surgeons (NOTSS) assessment was found to possess the highest level of validity from a variety of sources; reliability and acceptability have also been established for this tool. Conclusions. Deficits in current surgical training pathways have prompted several recommendations to meet the evolving requirements of surgeons. Recommendations from the current paper include integration of teamwork training and assessment into medical school curricula, standardised formal training of assessors to ensure accurate evaluation of nontechnical skill acquisition, and integration of concurrent technical and nontechnical skills training throughout training. PMID:26425732

A scoping review of assessment tools for laparoscopic suturing.

PubMed

Bilgic, Elif; Endo, Satoshi; Lebedeva, Ekaterina; Takao, Madoka; McKendy, Katherine M; Watanabe, Yusuke; Feldman, Liane S; Vassiliou, Melina C

2018-05-03

A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions. Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills. Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences. Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.

The SAFE-T assessment tool: derivation and validation of a web-based application for point-of-care evaluation of gastroenterology fellow performance in colonoscopy.

PubMed

Kumar, Navin L; Kugener, Guillaume; Perencevich, Molly L; Saltzman, John R

2018-01-01

Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise assessment tool to evaluate real-time fellow performance in colonoscopy administered via a web-based application. The Skill Assessment in Fellow Endoscopy Training (SAFE-T) tool was derived as a novel 5-question evaluation tool that captures both summative and formative feedback adapted into a web-based application. A prospective study of 15 gastroenterology fellows (5 fellows each from years 1 to 3 of training) was performed using the SAFE-T tool. An independent reviewer evaluated a subset of these procedures and completed the SAFE-T tool and Mayo Colonoscopy Skills Assessment Tool (MCSAT) for reliability testing. Twenty-six faculty completed 350 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score (year 1, 2.00; year 2, 3.84; year 3, 4.28) differentiated each sequential fellow year of training (P < .0001). The mean SAFE-T overall score decreased with increasing case complexity score, with straightforward cases compared with average cases (4.07 vs 3.50, P < .0001), and average cases compared with challenging cases (3.50 vs 3.08, P = .0134). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a kappa agreement statistic of 0.898 (P < .0001). Correlation of the SAFE-T overall score with the MCSAT overall hands-on and individual motor scores was excellent (each r > 0.90, P < .0001). We developed and validated the SAFE-T assessment tool, a concise and web-based means of assessing real-time gastroenterology fellow performance in colonoscopy. Copyright © 2018 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

Development and Validation of a Fine-Motor Assessment Tool for Use with Young Children in a Chinese Population

ERIC Educational Resources Information Center

Siu, Andrew M. H.; Lai, Cynthia Y. Y.; Chiu, Amy S. M.; Yip, Calvin C. K.

2011-01-01

Objectives: Most of the fine-motor assessment tools used in Hong Kong have been designed in Western countries, so there is a need to develop a standardized assessment which is relevant to the culture and daily living tasks of the local (that is, Chinese) population. This study aimed to (1) develop a fine-motor assessment tool (the Hong Kong…

The Assessment, Development, Assurance Pharmacist's Tool (ADAPT) for ensuring quality implementation of health promotion programs.

PubMed

Truong, Hoai-An; Taylor, Catherine R; DiPietro, Natalie A

2012-02-10

To develop and validate the Assessment, Development, Assurance Pharmacist's Tool (ADAPT), an instrument for pharmacists and student pharmacists to use in developing and implementing health promotion programs. The 36-item ADAPT instrument was developed using the framework of public health's 3 core functions (assessment, policy development, and assurance) and 10 essential services. The tool's content and usage was assessed and conducted through peer-review and initial validity testing processes. Over 20 faculty members, preceptors, and student pharmacists at 5 institutions involved in planning and implementing health promotion initiatives reviewed the instrument and conducted validity testing. The instrument took approximately 15 minutes to complete and the findings resulted in changes and improvements to elements of the programs evaluated. The ADAPT instrument fills a need to more effectively plan, develop, implement, and evaluate pharmacist-directed public health programs that are evidence-based, high-quality, and compliant with laws and regulations and facilitates documentation of pharmacists' contributions to public health.

Initial Steps in Creating a Developmentally Valid Tool for Observing/Assessing Rope Jumping

ERIC Educational Resources Information Center

Roberton, Mary Ann; Thompson, Gregory; Langendorfer, Stephen J.

2017-01-01

Background: Valid motor development sequences show the various behaviors that children display as they progress toward competence in specific motor skills. Teachers can use these sequences to observe informally or formally assess their students. While longitudinal study is ultimately required to validate developmental sequences, there are earlier,…

Assessing the Efficacy of the Measure of Understanding of Macroevolution as a Valid Tool for Undergraduate Non-Science Majors

ERIC Educational Resources Information Center

Romine, William Lee; Walter, Emily Marie

2014-01-01

Efficacy of the Measure of Understanding of Macroevolution (MUM) as a measurement tool has been a point of contention among scholars needing a valid measure for knowledge of macroevolution. We explored the structure and construct validity of the MUM using Rasch methodologies in the context of a general education biology course designed with an…

Multifactorial screening for fall risk in community-dwelling older adults in the primary care office: development of the fall risk assessment & screening tool.

PubMed

Renfro, Mindy Oxman; Fehrer, Steven

2011-01-01

Unintentional falls is an increasing public health problem as incidence of falls rises and the population ages. The Centers for Disease Control and Prevention reports that 1 in 3 adults aged 65 years and older will experience a fall this year; 20% to 30% of those who fall will sustain a moderate to severe injury. Physical therapists caring for older adults are usually engaged with these patients after the first injury fall and may have little opportunity to abate fall risk before the injuries occur. This article describes the content selection and development of a simple-to-administer, multifactorial, Fall Risk Assessment & Screening Tool (FRAST), designed specifically for use in primary care settings to identify those older adults with high fall risk. Fall Risk Assessment & Screening Tool incorporates previously validated measures within a new multifactorial tool and includes targeted recommendations for intervention. Development of the multifactorial FRAST used a 5-part process: identification of significant fall risk factors, review of best evidence, selection of items, creation of the scoring grid, and development of a recommended action plan. Fall Risk Assessment & Screening Tool has been developed to assess fall risk in the target population of older adults (older than 65 years) living and ambulating independently in the community. Many fall risk factors have been considered and 15 items selected for inclusion. Fall Risk Assessment & Screening Tool includes 4 previously validated measures to assess balance, depression, falls efficacy, and home safety. Reliability and validity studies of FRAST are under way. Fall risk for community-dwelling older adults is an urgent, multifactorial, public health problem. Providing primary care practitioners (PCPs) with a very simple screening tool is imperative. Fall Risk Assessment & Screening Tool was created to allow for safe, quick, and low-cost administration by minimally trained office staff with interpretation and follow-up provided by the PCP.

Using qualitative methods to improve questionnaires for Spanish speakers: assessing face validity of a food behavior checklist.

PubMed

Banna, Jinan C; Vera Becerra, Luz E; Kaiser, Lucia L; Townsend, Marilyn S

2010-01-01

Development of outcome measures relevant to health nutrition behaviors requires a rigorous process of testing and revision. Whereas researchers often report performance of quantitative data collection to assess questionnaire validity and reliability, qualitative testing procedures are often overlooked. This report outlines a procedure for assessing face validity of a Spanish-language dietary assessment tool. Reviewing the literature produced no rigorously validated Spanish-language food behavior assessment tools for the US Department of Agriculture's food assistance and education programs. In response to this need, this study evaluated the face validity of a Spanish-language food behavior checklist adapted from a 16-item English version of a food behavior checklist shown to be valid and reliable for limited-resource English speakers. The English version was translated using rigorous methods involving initial translation by one party and creation of five possible versions. Photos were modified based on client input and new photos were taken as necessary. A sample of low-income, Spanish-speaking women completed cognitive interviews (n=20). Spanish translation experts (n=7) fluent in both languages and familiar with both cultures made minor modifications but essentially approved client preferences. The resulting checklist generated a readability score of 93, indicating low reading difficulty. The Spanish-language checklist has adequate face validity in the target population and is ready for further validation using convergent measures. At the conclusion of testing, this instrument may be used to evaluate nutrition education interventions in California. These qualitative procedures provide a framework for designing evaluation tools for low-literate audiences participating in the US Department of Agriculture food assistance and education programs. Copyright 2010 American Dietetic Association. Published by Elsevier Inc. All rights reserved.

Using Qualitative Methods to Improve Questionnaires for Spanish Speakers: Assessing Face Validity of a Food Behavior Checklist

PubMed Central

BANNA, JINAN C.; VERA BECERRA, LUZ E.; KAISER, LUCIA L.; TOWNSEND, MARILYN S.

2015-01-01

Development of outcome measures relevant to health nutrition behaviors requires a rigorous process of testing and revision. Whereas researchers often report performance of quantitative data collection to assess questionnaire validity and reliability, qualitative testing procedures are often overlooked. This report outlines a procedure for assessing face validity of a Spanish-language dietary assessment tool. Reviewing the literature produced no rigorously validated Spanish-language food behavior assessment tools for the US Department of Agriculture’s food assistance and education programs. In response to this need, this study evaluated the face validity of a Spanish-language food behavior checklist adapted from a 16-item English version of a food behavior checklist shown to be valid and reliable for limited-resource English speakers. The English version was translated using rigorous methods involving initial translation by one party and creation of five possible versions. Photos were modified based on client input and new photos were taken as necessary. A sample of low-income, Spanish-speaking women completed cognitive interviews (n=20). Spanish translation experts (n=7) fluent in both languages and familiar with both cultures made minor modifications but essentially approved client preferences. The resulting checklist generated a readability score of 93, indicating low reading difficulty. The Spanish-language checklist has adequate face validity in the target population and is ready for further validation using convergent measures. At the conclusion of testing, this instrument may be used to evaluate nutrition education interventions in California. These qualitative procedures provide a framework for designing evaluation tools for low-literate audiences participating in the US Department of Agriculture food assistance and education programs. PMID:20102831

Automated Writing Evaluation for Formative Assessment of Second Language Writing: Investigating the Accuracy and Usefulness of Feedback as Part of Argument-Based Validation

ERIC Educational Resources Information Center

Ranalli, Jim; Link, Stephanie; Chukharev-Hudilainen, Evgeny

2017-01-01

An increasing number of studies on the use of tools for automated writing evaluation (AWE) in writing classrooms suggest growing interest in their potential for formative assessment. As with all assessments, these applications should be validated in terms of their intended interpretations and uses. A recent argument-based validation framework…

Tools used to assess medical students competence in procedural skills at the end of a primary medical degree: a systematic review.

PubMed

Morris, Marie C; Gallagher, Tom K; Ridgway, Paul F

2012-01-01

The objective was to systematically review the literature to identify and grade tools used for the end point assessment of procedural skills (e.g., phlebotomy, IV cannulation, suturing) competence in medical students prior to certification. The authors searched eight bibliographic databases electronically - ERIC, Medline, CINAHL, EMBASE, Psychinfo, PsychLIT, EBM Reviews and the Cochrane databases. Two reviewers independently reviewed the literature to identify procedural assessment tools used specifically for assessing medical students within the PRISMA framework, the inclusion/exclusion criteria and search period. Papers on OSATS and DOPS were excluded as they focused on post-registration assessment and clinical rather than simulated competence. Of 659 abstracted articles 56 identified procedural assessment tools. Only 11 specifically assessed medical students. The final 11 studies consisted of 1 randomised controlled trial, 4 comparative and 6 descriptive studies yielding 12 heterogeneous procedural assessment tools for analysis. Seven tools addressed four discrete pre-certification skills, basic suture (3), airway management (2), nasogastric tube insertion (1) and intravenous cannulation (1). One tool used a generic assessment of procedural skills. Two tools focused on postgraduate laparoscopic skills and one on osteopathic students and thus were not included in this review. The levels of evidence are low with regard to reliability - κ = 0.65-0.71 and minimum validity is achieved - face and content. In conclusion, there are no tools designed specifically to assess competence of procedural skills in a final certification examination. There is a need to develop standardised tools with proven reliability and validity for assessment of procedural skills competence at the end of medical training. Medicine graduates must have comparable levels of procedural skills acquisition entering the clinical workforce irrespective of the country of training.

Derivation of a Performance Checklist for Ultrasound-Guided Arthrocentesis Using the Modified Delphi Method.

PubMed

Kunz, Derek; Pariyadath, Manoj; Wittler, Mary; Askew, Kim; Manthey, David; Hartman, Nicholas

2017-06-01

Arthrocentesis is an important skill for physicians in multiple specialties. Recent studies indicate a superior safety and performance profile for this procedure using ultrasound guidance for needle placement, and improving quality of care requires a valid measurement of competency using this modality. We endeavored to create a validated tool to assess the performance of this procedure using the modified Delphi technique and experts in multiple disciplines across the United States. We derived a 22-item checklist designed to assess competency for the completion of ultrasound-guided arthrocentesis, which demonstrated a Cronbach's alpha of 0.89, indicating an excellent degree of internal consistency. Although we were able to demonstrate content validity for this tool, further validity evidence should be acquired after the tool is used and studied in clinical and simulated contexts. © 2017 by the American Institute of Ultrasound in Medicine.

Citizenship and recovery: two intertwined concepts for civic-recovery.

PubMed

Pelletier, Jean-François; Corbière, Marc; Lecomte, Tania; Briand, Catherine; Corrigan, Patrick; Davidson, Larry; Rowe, Michael

2015-03-04

Validation of the psychometric properties of a new measure of citizenship was required for a research project in the province of Quebec, Canada. This study was meant to study the interplay between recovery- and citizenship-oriented supportive employment. As recovery and citizenship were expected to be two related concepts, convergent validity between the Citizenship Measure (CM) and the Recovery Assessment Scale (RAS) was tested. Study objectives were to: 1) conduct exploratory factor analyses on the CM and confirmatory factor analysis on the RAS tools (construct validity), 2) calculate Cronbach's alphas for each dimension emerging from objective 1 (reliability), and 3) calculate correlations between all dimensions from both tools (convergent validity). Data were collected from 174 individuals with serious mental illness, working in social firms. Serious mental illnesses include major depression, schizophrenia, bipolar disorder, obsessive compulsive disorder, panic disorder, post traumatic stress disorder and borderline personality disorder. Five factors emerged from the exploratory factor analysis of the CM, with good reliability. Confirmatory factor analyses showed that the short and the long versions of the RAS present satisfactory results. Finally, the correlation matrix indicated that all dimensions from both tools are significantly correlated, thus confirming their convergent validity. This study confirms the validity and reliability of two tools, CM and RAS. These tools can be used in combination to assess citizenship and recovery, both of which may be combined in the new concept of civic-recovery.

Investigating the Validity of Two Widely Used Quantitative Text Tools

ERIC Educational Resources Information Center

Cunningham, James W.; Hiebert, Elfrieda H.; Mesmer, Heidi Anne

2018-01-01

In recent years, readability formulas have gained new prominence as a basis for selecting texts for learning and assessment. Variables that quantitative tools count (e.g., word frequency, sentence length) provide valid measures of text complexity insofar as they accurately predict representative and high-quality criteria. The longstanding…

Seeking Empirical Validity in an Assurance of Learning System

ERIC Educational Resources Information Center

Avery, Sherry L.; McWhorter, Rochell R.; Lirely, Roger; Doty, H. Harold

2014-01-01

Business schools have established measurement tools to support their assurance of learning (AoL) systems and to assess student achievement of learning objectives. However, business schools have not required their tools to be empirically validated, thus ensuring that they measure what they are intended to measure. The authors propose confirmatory…

Measuring couple relationship quality in a rural African population: Validation of a Couple Functionality Assessment Tool in Malawi.

PubMed

Ruark, Allison; Chase, Rachel; Hembling, John; Davis, Valerie Rhoe; Perrin, Paul Clayton; Brewster-Lee, Dorothy

2017-01-01

Available data suggest that individual and family well-being are linked to the quality of women's and men's couple relationships, but few tools exist to assess couple relationship functioning in low- and middle-income countries. In response to this gap, Catholic Relief Services has developed a Couple Functionality Assessment Tool (CFAT) to capture valid and reliable data on various domains of relationship quality. This tool is designed to be used by interventions which aim to improve couple and family well-being as a means of measuring the effectiveness of these interventions, particularly related to couple relationship quality. We carried out a validation study of the CFAT among 401 married and cohabiting adults (203 women and 198 men) in rural Chikhwawa District, Malawi. Using psychometric scales, the CFAT addressed six domains of couple relationship quality (intimacy, partner support, sexual satisfaction, gender roles, decision-making, and communication and conflict management), and included questions on intimate partner violence. We used exploratory factor analysis to assess scale performance of each domain and produce a shortened Relationship Quality Index (RQI) composed of items from five relationship quality domains. This article reports the performance of the RQI. Internal reliability and validity of the RQI were found to be good. Regression analyses examined the relationship of the RQI to outcomes important to health and development: intra-household cooperation, positive health behaviors, intimate partner violence, and gender-equitable norms. We found many significant correlations between RQI scores and these couple- and family-level development issues. There is a need to further validate the tool with use in other populations as well as to continue to explore whether the observed linkages between couple functionality and development outcomes are causal relationships.

Measuring couple relationship quality in a rural African population: Validation of a Couple Functionality Assessment Tool in Malawi

PubMed Central

Chase, Rachel; Hembling, John; Davis, Valerie Rhoe; Perrin, Paul Clayton; Brewster-Lee, Dorothy

2017-01-01

Available data suggest that individual and family well-being are linked to the quality of women’s and men’s couple relationships, but few tools exist to assess couple relationship functioning in low- and middle-income countries. In response to this gap, Catholic Relief Services has developed a Couple Functionality Assessment Tool (CFAT) to capture valid and reliable data on various domains of relationship quality. This tool is designed to be used by interventions which aim to improve couple and family well-being as a means of measuring the effectiveness of these interventions, particularly related to couple relationship quality. We carried out a validation study of the CFAT among 401 married and cohabiting adults (203 women and 198 men) in rural Chikhwawa District, Malawi. Using psychometric scales, the CFAT addressed six domains of couple relationship quality (intimacy, partner support, sexual satisfaction, gender roles, decision-making, and communication and conflict management), and included questions on intimate partner violence. We used exploratory factor analysis to assess scale performance of each domain and produce a shortened Relationship Quality Index (RQI) composed of items from five relationship quality domains. This article reports the performance of the RQI. Internal reliability and validity of the RQI were found to be good. Regression analyses examined the relationship of the RQI to outcomes important to health and development: intra-household cooperation, positive health behaviors, intimate partner violence, and gender-equitable norms. We found many significant correlations between RQI scores and these couple- and family-level development issues. There is a need to further validate the tool with use in other populations as well as to continue to explore whether the observed linkages between couple functionality and development outcomes are causal relationships. PMID:29190769

Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate-the learning curve for endoscopy training

PubMed Central

Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

2014-01-01

BACKGROUND: Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. OBJECTIVE: To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. METHODS: The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists/surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The ‘Endoscopy Task Load Index’ was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. RESULTS: From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. DISCUSSION: The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers. PMID:24619638

Part 1: Pressure ulcer assessment - the development of Critical Care Pressure Ulcer Assessment Tool made Easy (CALCULATE).

PubMed

Richardson, Annette; Barrow, Isabel

2015-11-01

Critically ill patients are at high risk of developing pressure ulcers resulting in serious untoward patient and health care system outcomes. Pressure ulcer prevention is therefore an important patient safety priority and establishing a structured approach to pressure ulcer risk assessment to identify patients at risk is a critical first step. The literature was searched using three electronic databases from 2000 to 2011 to identify papers reporting on pressure ulcer risk factors and assessment in adult critical care. The review and appraisal of papers were conducted by two critical care nurses. Papers underwent detailed review if they met inclusion criteria where they identified pressure ulcer assessment scores, scales or risk factors and related to adult critical care patients Seven papers were reviewed. No single assessment tool was sufficiently validated for critically ill patients and seven key critical care risk factors were identified. These risk factors were: mechanical ventilation, impaired circulation, dialysis, long surgery, low protein and too unstable to turn. The tool Critical Care Pressure Ulcer Assessment Tool made Easy (CALCULATE) was developed utilizing the risk factors from the literature and expert critical care nursing consensus decision-making. In the absence of current consensus, valid assessment scales and limited evidence for the most appropriate pressure ulcer assessment for critically ill patients, this assessment tool offers an easy, appropriate alternative for critically ill patients than existing tools primarily validated for acute care wards. 'CALCULATE' offers an important contribution towards the advancement and development of critical care pressure ulcer risk assessment. Future research is needed to further enhance and inform pressure ulcer risk assessment of the critically ill patients. The identification of critical care risk factors may be an indicative method of assessing pressure ulcer risk in the critically ill patients. © 2015 British Association of Critical Care Nurses.

Procedure-specific assessment tool for flexible pharyngo-laryngoscopy: gathering validity evidence and setting pass-fail standards.

PubMed

Melchiors, Jacob; Petersen, K; Todsen, T; Bohr, A; Konge, Lars; von Buchwald, Christian

2018-06-01

The attainment of specific identifiable competencies is the primary measure of progress in the modern medical education system. The system, therefore, requires a method for accurately assessing competence to be feasible. Evidence of validity needs to be gathered before an assessment tool can be implemented in the training and assessment of physicians. This evidence of validity must according to the contemporary theory on validity be gathered from specific sources in a structured and rigorous manner. The flexible pharyngo-laryngoscopy (FPL) is central to the otorhinolaryngologist. We aim to evaluate the flexible pharyngo-laryngoscopy assessment tool (FLEXPAT) created in a previous study and to establish a pass-fail level for proficiency. Eighteen physicians with different levels of experience (novices, intermediates, and experienced) were recruited to the study. Each performed an FPL on two patients. These procedures were video recorded, blinded, and assessed by two specialists. The score was expressed as the percentage of a possible max score. Cronbach's α was used to analyze internal consistency of the data, and a generalizability analysis was performed. The scores of the three different groups were explored, and a pass-fail level was determined using the contrasting groups' standard setting method. Internal consistency was strong with a Cronbach's α of 0.86. We found a generalizability coefficient of 0.72 sufficient for moderate stakes assessment. We found a significant difference between the novice and experienced groups (p < 0.001) and strong correlation between experience and score (Pearson's r = 0.75). The pass/fail level was established at 72% of the maximum score. Applying this pass-fail level in the test population resulted in half of the intermediary group receiving a failing score. We gathered validity evidence for the FLEXPAT according to the contemporary framework as described by Messick. Our results support a claim of validity and are comparable to other studies exploring clinical assessment tools. The high rate of physicians underperforming in the intermediary group demonstrates the need for continued educational intervention. Based on our work, we recommend the use of the FLEXPAT in clinical assessment of FPL and the application of a pass-fail level of 72% for proficiency.

Reliability and Validity of the Alberta Context Tool (ACT) with Professional Nurses: Findings from a Multi-Study Analysis

PubMed Central

Squires, Janet E.; Hayduk, Leslie; Hutchinson, Alison M.; Mallick, Ranjeeta; Norton, Peter G.; Cummings, Greta G.; Estabrooks, Carole A.

2015-01-01

Although organizational context is central to evidence-based practice, underdeveloped measurement hindersitsassessment. The Alberta Context Tool, comprised of 59 items that tap10 modifiable contextual concepts, was developed to address this gap. The purpose of this study to examine the reliability and validity of scores obtained when the Alberta Context Tool is completed by professional nurses across different healthcare settings. Five separate studies (N = 2361 nurses across different care settings) comprised the study sample. Reliability and validity were assessed. Cronbach’s alpha exceeded 0.70 for9/10 Alberta Context Tool concepts. Item-total correlations exceeded acceptable standards for 56/59items. Confirmatory Factor Analysescoordinated acceptably with the Alberta Context Tool’s proposed latent structure. The mean values for each Alberta Context Tool concept increased from low to high levels of research utilization(as hypothesized) further supporting its validity. This study provides robust evidence forreliability and validity of scores obtained with the Alberta Context Tool when administered to professional nurses. PMID:26098857

Validation of a global scale to assess the quality of interprofessional teamwork in mental health settings.

PubMed

Tomizawa, Ryoko; Yamano, Mayumi; Osako, Mitue; Hirabayashi, Naotugu; Oshima, Nobuo; Sigeta, Masahiro; Reeves, Scott

2017-12-01

Few scales currently exist to assess the quality of interprofessional teamwork through team members' perceptions of working together in mental health settings. The purpose of this study was to revise and validate an interprofessional scale to assess the quality of teamwork in inpatient psychiatric units and to use it multi-nationally. A literature review was undertaken to identify evaluative teamwork tools and develop an additional 12 items to ensure a broad global focus. Focus group discussions considered adaptation to different care systems using subjective judgements from 11 participants in a pre-test of items. Data quality, construct validity, reproducibility, and internal consistency were investigated in the survey using an international comparative design. Exploratory factor analysis yielded five factors with 21 items: 'patient/community centred care', 'collaborative communication', 'interprofessional conflict', 'role clarification', and 'environment'. High overall internal consistency, reproducibility, adequate face validity, and reasonable construct validity were shown in the USA and Japan. The revised Collaborative Practice Assessment Tool (CPAT) is a valid measure to assess the quality of interprofessional teamwork in psychiatry and identifies the best strategies to improve team performance. Furthermore, the revised scale will generate more rigorous evidence for collaborative practice in psychiatry internationally.

A Comparative Study of Adolescent Risk Assessment Instruments: Predictive and Incremental Validity

ERIC Educational Resources Information Center

Welsh, Jennifer L.; Schmidt, Fred; McKinnon, Lauren; Chattha, H. K.; Meyers, Joanna R.

2008-01-01

Promising new adolescent risk assessment tools are being incorporated into clinical practice but currently possess limited evidence of predictive validity regarding their individual and/or combined use in risk assessments. The current study compares three structured adolescent risk instruments, Youth Level of Service/Case Management Inventory…

External validation of a simple clinical tool used to predict falls in people with Parkinson disease

PubMed Central

Duncan, Ryan P.; Cavanaugh, James T.; Earhart, Gammon M.; Ellis, Terry D.; Ford, Matthew P.; Foreman, K. Bo; Leddy, Abigail L.; Paul, Serene S.; Canning, Colleen G.; Thackeray, Anne; Dibble, Leland E.

2015-01-01

Background Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD. METHODS We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment. RESULTS The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76 –0.89), comparable to the developmental study. CONCLUSION The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual’s risk of an impending fall. PMID:26003412

External validation of a simple clinical tool used to predict falls in people with Parkinson disease.

PubMed

Duncan, Ryan P; Cavanaugh, James T; Earhart, Gammon M; Ellis, Terry D; Ford, Matthew P; Foreman, K Bo; Leddy, Abigail L; Paul, Serene S; Canning, Colleen G; Thackeray, Anne; Dibble, Leland E

2015-08-01

Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD. We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment. The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76-0.89), comparable to the developmental study. The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual's risk of an impending fall. Copyright © 2015 Elsevier Ltd. All rights reserved.

A Novel Tool for Assessment of Emergency Medicine Resident Skill in Determining Diagnosis and Management for Emergent Electrocardiograms: A Multicenter Study.

PubMed

Hartman, Nicholas D; Wheaton, Natasha B; Williamson, Kelly; Quattromani, Erin N; Branzetti, Jeremy B; Aldeen, Amer Z

2016-12-01

Reading emergent electrocardiograms (ECGs) is one of the emergency physician's most crucial tasks, yet no well-validated tool exists to measure resident competence in this skill. To assess validity of a novel tool measuring emergency medicine resident competency for interpreting, and responding to, critical ECGs. In addition, we aim to observe trends in this skill for resident physicians at different levels of training. This is a multi-center, prospective study of postgraduate year (PGY) 1-4 residents at five emergency medicine (EM) residency programs in the United States. An assessment tool was created that asks the physician to identify either the ECG diagnosis or the best immediate management. One hundred thirteen EM residents from five EM residency programs submitted completed assessment surveys, including 43 PGY-1s, 33 PGY-2s, and 37 PGY-3/4s. PGY-3/4s averaged 74.6% correct (95% confidence interval [CI] 70.9-78.4) and performed significantly better than PGY-1s, who averaged 63.2% correct (95% CI 58.0-68.3). PGY-2s averaged 69.0% (95% CI 62.2-73.7). Year-to-year differences were more pronounced in management than in diagnosis. Residency training in EM seems to be associated with improved ability to interpret "critical" ECGs as measured by our assessment tool. This lends validity evidence for the tool by correlating with a previously observed association between residency training and improved ECG interpretation. Resident skill in ECG interpretation remains less than ideal. Creation of this sort of tool may allow programs to assess resident performance as well as evaluate interventions designed to improve competency. Copyright © 2016 Elsevier Inc. All rights reserved.

Validation of an organizational communication climate assessment toolkit.

PubMed

Wynia, Matthew K; Johnson, Megan; McCoy, Thomas P; Griffin, Leah Passmore; Osborn, Chandra Y

2010-01-01

Effective communication is critical to providing quality health care and can be affected by a number of modifiable organizational factors. The authors performed a prospective multisite validation study of an organizational communication climate assessment tool in 13 geographically and ethnically diverse health care organizations. Communication climate was measured across 9 discrete domains. Patient and staff surveys with matched items in each domain were developed using a national consensus process, which then underwent psychometric field testing and assessment of domain coherence. The authors found meaningful within-site and between-site performance score variability in all domains. In multivariable models, most communication domains were significant predictors of patient-reported quality of care and trust. The authors conclude that these assessment tools provide a valid empirical assessment of organizational communication climate in 9 domains. Assessment results may be useful to track organizational performance, to benchmark, and to inform tailored quality improvement interventions.

Nutrition screening tools: does one size fit all? A systematic review of screening tools for the hospital setting.

PubMed

van Bokhorst-de van der Schueren, Marian A E; Guaitoli, Patrícia Realino; Jansma, Elise P; de Vet, Henrica C W

2014-02-01

Numerous nutrition screening tools for the hospital setting have been developed. The aim of this systematic review is to study construct or criterion validity and predictive validity of nutrition screening tools for the general hospital setting. A systematic review of English, French, German, Spanish, Portuguese and Dutch articles identified via MEDLINE, Cinahl and EMBASE (from inception to the 2nd of February 2012). Additional studies were identified by checking reference lists of identified manuscripts. Search terms included key words for malnutrition, screening or assessment instruments, and terms for hospital setting and adults. Data were extracted independently by 2 authors. Only studies expressing the (construct, criterion or predictive) validity of a tool were included. 83 studies (32 screening tools) were identified: 42 studies on construct or criterion validity versus a reference method and 51 studies on predictive validity on outcome (i.e. length of stay, mortality or complications). None of the tools performed consistently well to establish the patients' nutritional status. For the elderly, MNA performed fair to good, for the adults MUST performed fair to good. SGA, NRS-2002 and MUST performed well in predicting outcome in approximately half of the studies reviewed in adults, but not in older patients. Not one single screening or assessment tool is capable of adequate nutrition screening as well as predicting poor nutrition related outcome. Development of new tools seems redundant and will most probably not lead to new insights. New studies comparing different tools within one patient population are required. Copyright © 2013 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

Measuring activities and participation in persons with haemophilia: A systematic review of commonly used instruments.

PubMed

Timmer, M A; Gouw, S C; Feldman, B M; Zwagemaker, A; de Kleijn, P; Pisters, M F; Schutgens, R E G; Blanchette, V; Srivastava, A; David, J A; Fischer, K; van der Net, J

2018-03-01

Monitoring clinical outcome in persons with haemophilia (PWH) is essential in order to provide optimal treatment for individual patients and compare effectiveness of treatment strategies. Experience with measurement of activities and participation in haemophilia is limited and consensus on preferred tools is lacking. The aim of this study was to give a comprehensive overview of the measurement properties of a selection of commonly used tools developed to assess activities and participation in PWH. Electronic databases were searched for articles that reported on reliability, validity or responsiveness of predetermined measurement tools (5 self-reported and 4 performance based measurement tools). Methodological quality of the studies was assessed according to the COSMIN checklist. Best evidence synthesis was used to summarize evidence on the measurement properties. The search resulted in 3453 unique hits. Forty-two articles were included. The self-reported Haemophilia Acitivity List (HAL), Pediatric HAL (PedHAL) and the performance based Functional Independence Score in Haemophilia (FISH) were studied most extensively. Methodological quality of the studies was limited. Measurement error, cross-cultural validity and responsiveness have been insufficiently evaluated. Albeit based on limited evidence, the measurement properties of the PedHAL, HAL and FISH are currently considered most satisfactory. Further research needs to focus on measurement error, responsiveness, interpretability and cross-cultural validity of the self-reported tools and validity of performance based tools which are able to assess limitations in sports and leisure activities. © 2018 The Authors. Haemophilia Published by John Wiley & Sons Ltd.

Model Modules to Assist Assessing and Controlling SCC

DOT National Transportation Integrated Search

2008-04-04

This project developed and validated tools to assist in integrity assessment and management both forms of SCC. Because the understanding that underlies integrity management tools was most comprehensive for high-pH SCC, development targeted NN-pH SCC,...

Quantitative Analysis of the Rubric as an Assessment Tool: An Empirical Study of Student Peer-Group Rating

ERIC Educational Resources Information Center

Hafner, John C.; Hafner, Patti M.

2003-01-01

Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool "in the hands of the students." This study focuses on the validity and reliability of the rubric as…

Health related quality of life assessment in metastatic disease of the spine: a systematic review.

PubMed

Street, John; Berven, Sigurd; Fisher, Charles; Ryken, Timothy

2009-10-15

Systematic literature review. To examine the available literature on health related quality of life (HRQOL) assessment in metastatic disease of the spine and identify the optimal functional outcome scales to be used in developing a disease-specific tool. There is a lack of consensus in the use of HRQOL measures in patients with metastatic spine disease. A systematic review was conducted using MEDLINE, EMBASE, the Science Citation Index (ISI), the Cumulative Index to Nursing and Allied Health Literature, the PsycINFO, the Allied and Complementary Medicine (AMED), Cochrane Reviews and Global Health databases for clinical studies addressing metastatic spine disease from 1966 through 2008. The validity of outcome tools was established by linkage analysis with the International Classification of Functioning Disability and Health (ICF). One hundred forty-one clinical studies met inclusion criteria including 10,347 patients. Only 5 moderate grade and 1 high grade study were identified. Thirty- four studies used a patient self-assessment instrument to assess health status. None of the instruments were validated for metastatic spine patients. The most commonly used Pi-by-no tools were SF-36, SIP 5, and the ADL. None of the studies defined health related quality of life (HRQOL) or justified the choice of instrument. The most commonly used cancer-specific tools were ECOG, EORTC QCQ-C30, and EUROQOL 5D. Based on frequency of citation and on correlation with the International Classification of Functioning Disability and Health, the ECOG and SF36 were judged as most valid and reliable. A systematic review of the available evidence suggests that valid and reliable health related quality of life measures exist for the assessment of oncology patients; however, a disease-specific tool for metastatic spine disease awaits development. Until such time as a disease-specific tool is available, we recommend that the ECOG and SF-36 be considered for use in studies addressing the outcome assessment of patients with metastatic spine disease.

[Hand fine motor skills and use of both hand and arm in subjects after a stroke: a systematic review].

PubMed

Ostolaza, M; Abudarham, J; Dilascio, S; Drault-Boedo, E; Gallo, S; Garcete, A; Kramer, M; Maiaru, M; Mendelevich, A; Modica, M; Peralta, F; Sanchez-Correa, C

2017-04-01

In clinical practice it is important to be able to assess the function of the upper limb of the patient who has suffered a stroke. There is currently no systemic review that could identify assessment tools for the 'fine use of the hand' and 'use of both hand and arm'. Primary, to identify observational tools which can assess the fine use of the hand and the use of both hand and arm in patients with stroke sequels. Secondary, to analyze the bias risk in the included articles, describing and categorizing the clinical utility, validity and reliability. A search was carried in Medline, LILACS, SciELO and Open Grey, which included articles published until October 2015. Studies that validate assessing tools of the upper limb in subjects with a stroke sequel which evaluate the fine use of the hand and the use of both hand and arm were included. Eleven tools in evaluate observational haven been selected, which assess the fine use of the hand and the use of hand and arm. In every case both validity and reliability have been reported, but clinical utility has been less considered for assessment. The studies that researched these tools showed a high risk of bias in their development. ARAT-19 showed a lower bias risk, but when it has to do with applicability and the reference trial is taken into account, the level of concern is high.

Evaluating the Diagnostic Validity of a Facet-Based Formative Assessment System

ERIC Educational Resources Information Center

DeBarger, Angela Haydel; DiBello, Louis; Minstrell, Jim; Feng, Mingyu; Stout, William; Pellegrino, James; Haertel, Geneva; Harris, Christopher; Ructinger, Liliana

2011-01-01

This paper describes methods for an alignment study and psychometric analyses of a formative assessment system, Diagnoser Tools for physics. Diagnoser Tools begin with facet clusters as the interpretive framework for designing questions and instructional activities. Thus each question in the diagnostic assessments includes distractors that…

Knowing Every Child: Validation of the Holistic Student Assessment (HSA) as a Measure of Social-Emotional Development.

PubMed

Malti, Tina; Zuffianò, Antonio; Noam, Gil G

2018-04-01

Knowing every child's social-emotional development is important as it can support prevention and intervention approaches to meet the developmental needs and strengths of children. Here, we discuss the role of social-emotional assessment tools in planning, implementing, and evaluating preventative strategies to promote mental health in all children and adolescents. We, first, selectively review existing tools and identify current gaps in the measurement literature. Next, we introduce the Holistic Student Assessment (HSA), a tool that is based in our social-emotional developmental theory, The Clover Model, and designed to measure social-emotional development in children and adolescents. Using a sample of 5946 students (51% boys, M age  = 13.16 years), we provide evidence for the psychometric validity of the self-report version of the HSA. First, we document the theoretically expected 7-dimension factor structure in a calibration sub-sample (n = 984) and cross-validate its structure in a validation sub-sample (n = 4962). Next, we show measurement invariance across development, i.e., late childhood (9- to 11-year-olds), early adolescence (12- to 14-year-olds), and middle adolescence (15- to 18-year-olds), and evidence for the HSA's construct validity in each age group. The findings support the robustness of the factor structure and confirm its developmental sensitivity. Structural equation modeling validity analysis in a multiple-group framework indicates that the HSA is associated with mental health in expected directions across ages. Overall, these findings show the psychometric properties of the tool, and we discuss how social-emotional tools such as the HSA can guide future research and inform large-scale dissemination of preventive strategies.

Implementation of a digital preparation validation tool in dental skills laboratory training.

PubMed

Kozarovska, A; Larsson, C

2018-05-01

To describe the implementation of a digital tool for preparation validation and evaluate it as an aid in students' self-assessment. Students at the final semester of skills laboratory training were asked to use a digital preparation validation tool (PVT) when performing two different tasks; preparation of crowns for teeth 11 and 21. The students were divided into two groups. Group A self-assessed and scanned all three attempts at 21 ("prep-and-scan"). Group B self-assessed all attempts chose the best one and scanned it ("best-of-three"). The situation was reversed for 11. The students assessed five parameters of the preparation and marked them as approved (A) or failed (F). These marks were compared with the information from the PVT. The students also completed a questionnaire. Each question was rated from 1 to 5. Teachers' opinions were collected at staff meetings throughout the project. Most students in the "prep-and-scan" groups showed an increase in agreement between their self-assessment and the information from the PVT, whereas students in the "best-of-three" groups showed lower levels of agreement. All students rated the PVT positively. Most strongly agreed that the tool was helpful in developing skills (mean 4.15), easy to use (mean 4.23) and that it added benefits in comparison to existing assessment tools (mean 4.05). They did not however, fully agree that the tool is time efficient (mean 2.55), and they did not consider it a substitute for verbal teacher feedback. Teachers' feedback suggested advantages of the tool in the form of ease of use, visual aid and increasing interest and motivation during skills laboratory training however, they did not notice a reduction in need of verbal feedback. Within the limitations of the study, our conclusion is that a digital PVT may be a valuable adjunct to other assessment tools in skills laboratory training. © 2017 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

[Emotional Intelligence Index: a tool for the routine assessment of mental health promotion programs in schools].

PubMed

Veltro, Franco; Ialenti, Valentina; Morales García, Manuel Alejandro; Gigantesco, Antonella

2016-01-01

After critical examination of several aspects relating to the evaluation of some dimensions of emotional intelligence through self-assessment tools, is described the procedure of construction and validation of an Index for its measurement, conceived only for the routine assessment of health promotion programs mental in schools that include among their objectives the improvement of emotional intelligence specifically "outcome-oriented". On the basis of the two most common international tools, are listed 27 items plus 6 of control, illustrated two Focus Group (FG) of students (face validity). The scale obtained by FG was administered to 300 students, and the results were submitted to factorial analysis (construct validity). It was also evaluated the internal consistency with Cronbach's Alpha and studied concurrent validity with the emotional quotient inventory, a scale of perceived self-efficacy and a stress test rating. From the analysis of FG all the original items were modified, deleted 4, and reduced the encoding system from 6 to 4 levels of Likert scale. Of the 23 items included in the analysis have emerged five factors (intra-psychic dimension, interpersonal, impulsivity, adaptive coping, sense of self-efficacy) for a total of 15 items. Very satisfactory were the results of the validation process of internal consistency (0.72) and the concurrent validity. The results are positive. It is obtained in fact the shortest routine assessment tool currently available in Italy which constitutes a real Index, for which compilation are required on average 3 minutes. Is emphasized the characteristic of an Index, and not of questionnaire or interview for clinical use, highlighting the only specific use for mental health promotion programs in schools.

The development and assessment of an evaluation tool for pediatric resident competence in leading simulated pediatric resuscitations.

PubMed

Grant, Estée C; Grant, Vincent J; Bhanji, Farhan; Duff, Jonathan P; Cheng, Adam; Lockyer, Jocelyn M

2012-07-01

It is critical that competency in pediatric resuscitation is achieved and assessed during residency or post graduate medical training. The purpose of this study was to create and evaluate a tool to measure all elements of pediatric resuscitation team leadership competence. An initial set of items, derived from a literature review and a brainstorming session, were refined to a 26 item assessment tool through the use of Delphi methodology. The tool was tested using videos of standardized resuscitations. A psychometric assessment of the evidence for instrument validity and reliability was undertaken. The performance of 30 residents on two videotaped scenarios was assessed by 4 pediatricians using the tool, with 12 items assessing 'leadership and communication skills' (LCS) and 14 items assessing 'knowledge and clinical skills' (KCS). The instrument showed evidence of reliability; the Cronbach's alpha and generalizability co-efficients for the overall instrument were α=0.818 and Ep(2)=0.76, for LCS were α=0.827 and Ep(2)=0.844, and for KCS were α=0.673 and Ep(2)=0.482. While validity was initially established through literature review and brainstorming by the panel of experts, it was further built through the high strength of correlation between global scores and scores for overall performance (r=0.733), LCS (r=0.718) and KCS (r=0.662) as well as the factor analysis which accounted for 40.2% of the variance. The results of the study demonstrate that the instrument is a valid and reliable tool to evaluate pediatric resuscitation team leader competence. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

Development and validation of a new assessment tool for suturing skills in medical students.

PubMed

Sundhagen, Henriette Pisani; Almeland, Stian Kreken; Hansson, Emma

2018-01-01

In recent years, emphasis has been put on that medical student should demonstrate pre-practice/pre-registration core procedural skills to ensure patient safety. Nonetheless, the formal teaching and training of basic suturing skills to medical students have received relatively little attention and there is no standard for what should be tested and how. The aim of this study was to develop and validate, using scientific methods, a tool for assessment of medical students' suturing skills, measuring both micro- and macrosurgical qualities. A tool was constructed and content, construct, concurrent validity, and inter-rater, inter-item, inter-test reliability were tested. Three groups were included: students with no training in suturing skills, students who have had training, plastic surgery. The results show promising reliability and validity when assessing novice medical students' suturing skills. Further studies are needed on implementation of the instrument. Moreover, how the instrument can be used to give formative feedback, evaluate if a required standard is met and for curriculum development needs further investigation.Level of Evidence: Not ratable.

An Exploratory Factor Analysis and Construct Validity of the Resident Choice Assessment Scale with Paid Carers of Adults with Intellectual Disabilities and Challenging Behavior in Community Settings

ERIC Educational Resources Information Center

Ratti, Victoria; Vickerstaff, Victoria; Crabtree, Jason; Hassiotis, Angela

2017-01-01

Introduction: The Resident Choice Assessment Scale (RCAS) is used to assess choice availability for adults with intellectual disabilities (ID). The aim of the study was to explore the factor structure, construct validity, and internal consistency of the measure in community settings to further validate this tool. Method: 108 paid carers of adults…

Comparison of Three Cognitive Screening Tools in Older Urban and Regional Aboriginal Australians.

PubMed

Radford, Kylie; Mack, Holly A; Draper, Brian; Chalkley, Simon; Delbaere, Kim; Daylight, Gail; Cumming, Robert G; Bennett, Hayley; Broe, Gerald A

2015-01-01

Validated cognitive screening tools for use in urban and regional Aboriginal populations in Australia are lacking. In a cross-sectional community-based study, 235 participants were assessed on the Mini-Mental State Examination (MMSE), the Rowland Universal Dementia Assessment Scale (RUDAS) and an urban modification of the Kimberley Indigenous Cognitive Assessment (mKICA). Performance on these cognitive screening tools was compared to dementia diagnosis by clinical consensus. All tests were culturally acceptable with good psychometric properties. Receiver operating characteristic curve analyses revealed that the MMSE and mKICA were the most accurate. The MMSE is an effective cognitive screening tool in urban Aboriginal populations. The mKICA is a good alternative when illiteracy, language or cultural considerations deem it appropriate. The RUDAS also has adequate validity in this population. © 2015 S. Karger AG, Basel.

Validation of a Group-Administered Pictorial Dietary Recall with 9- to 11-Year-Old Children

ERIC Educational Resources Information Center

Wallen, Victoria; Cunningham-Sabo, Leslie; Auld, Garry; Romaniello, Cathy

2011-01-01

Objective: Determine validity of Day in the Life Questionnaire-Colorado (DILQ-CO) as a dietary assessment tool for classroom-administered use. Methods: Agreement between DILQ-CO responses and weighed plate waste measured in 125 fourth-grade students in 2 low-income schools. Validity assessed by comparing reported school lunch items and portion…

Development and validation of the Vietnamese primary care assessment tool.

PubMed

Hoa, Nguyen Thi; Tam, Nguyen Minh; Peersman, Wim; Derese, Anselme; Markuns, Jeffrey F

2018-01-01

To adapt the consumer version of the Primary Care Assessment Tool (PCAT) for Vietnam and determine its internal consistency and validity. A quantitative cross sectional study. 56 communes in 3 representative provinces of central Vietnam. Total of 3289 people who used health care services at health facility at least once over the past two years. The Vietnamese adult expanded consumer version of the PCAT (VN PCAT-AE) is an instrument for evaluation of primary care in Vietnam with 70 items comprising six scales representing four core primary care domains, and three additional scales representing three derivative domains. Sixteen other items from the original tool were not included in the final instrument, due to problems with missing values, floor or ceiling effects, and item-total correlations. All the retained scales have a Cronbach's alpha above 0.70 except for the subscale of Family Centeredness. The VN PCAT-AE demonstrates adequate internal consistency and validity to be used as an effective tool for measuring the quality of primary care in Vietnam from the consumer perspective. Additional work in the future to optimize valid measurement in all domains consistent with the original version of the tool may be helpful as the primary care system in Vietnam further develops.

Cognitive screening tools for identification of dementia in illiterate and low-educated older adults, a systematic review and meta-analysis.

PubMed

Paddick, Stella-Maria; Gray, William K; McGuire, Jackie; Richardson, Jenny; Dotchin, Catherine; Walker, Richard W

2017-06-01

The majority of older adults with dementia live in low- and middle-income countries (LMICs). Illiteracy and low educational background are common in older LMIC populations, particularly in rural areas, and cognitive screening tools developed for this setting must reflect this. This study aimed to review published validation studies of cognitive screening tools for dementia in low-literacy settings in order to determine the most appropriate tools for use. A systematic search of major databases was conducted according to PRISMA guidelines. Validation studies of brief cognitive screening tests including illiterate participants or those with elementary education were eligible. Studies were quality assessed using the QUADAS-2 tool. Good or fair quality studies were included in a bivariate random-effects meta-analysis and a hierarchical summary receiver operating characteristic (HSROC) curve constructed. Forty-five eligible studies were quality assessed. A significant proportion utilized a case-control design, resulting in spectrum bias. The area under the ROC (AUROC) curve was 0.937 for community/low prevalence studies, 0.881 for clinic based/higher prevalence studies, and 0.869 for illiterate populations. For the Mini-Mental State Examination (MMSE) (and adaptations), the AUROC curve was 0.853. Numerous tools for assessment of cognitive impairment in low-literacy settings have been developed, and tools developed for use in high-income countries have also been validated in low-literacy settings. Most tools have been inadequately validated, with only MMSE, cognitive abilities screening instrument (CASI), Eurotest, and Fototest having more than one published good or fair quality study in an illiterate or low-literate setting. At present no screening test can be recommended.

The Development and Validation of a Rapid Assessment Tool of Primary Care in China

PubMed Central

Mei, Jie; Liang, Yuan; Shi, LeiYu; Zhao, JingGe; Wang, YuTan; Kuang, Li

2016-01-01

Introduction. With Chinese health care reform increasingly emphasizing the importance of primary care, the need for a tool to evaluate primary care performance and service delivery is clear. This study presents a methodology for a rapid assessment of primary care organizations and service delivery in China. Methods. The study translated and adapted the Primary Care Assessment Tool-Adult Edition (PCAT-AE) into a Chinese version to measure core dimensions of primary care, namely, first contact, continuity, comprehensiveness, and coordination. A cross-sectional survey was conducted to assess the validity and reliability of the Chinese Rapid Primary Care Assessment Tool (CR-PCAT). Eight community health centers in Guangdong province have been selected to participate in the survey. Results. A total of 1465 effective samples were included for data analysis. Eight items were eliminated following principal component analysis and reliability testing. The principal component analysis extracted five multiple-item scales (first contact utilization, first contact accessibility, ongoing care, comprehensiveness, and coordination). The tests of scaling assumptions were basically met. Conclusion. The standard psychometric evaluation indicates that the scales have achieved relatively good reliability and validity. The CR-PCAT provides a rapid and reliable measure of four core dimensions of primary care, which could be applied in various scenarios. PMID:26885509

Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer.

PubMed

Konge, L; Vilmann, P; Clementsen, P; Annema, J T; Ringsted, C

2012-10-01

Fine-needle aspiration (FNA) guided by endoscopic ultrasonography (EUS) is important in mediastinal staging of non-small cell lung cancer (NSCLC). Training standards and implementation strategies of this technique are currently under discussion. The aim of this study was to explore the reliability and validity of a newly developed EUS Assessment Tool (EUSAT) designed to measure competence in EUS - FNA for mediastinal staging of NSCLC. A total of 30 patients with proven or suspected NSCLC underwent EUS - FNA for mediastinal staging by three trainees and three experienced physicians. Their performances were assessed prospectively by three experts in EUS under direct observation and again 2 months later in a blinded fashion using digital video-recordings. Based on the assessments, intra-rater reliability, inter-rater reliability, and construct validity were explored. The intra-rater reliability was good (Cronbach's α = 0.80), but comparison of results based on direct observations and blinded video-recordings indicated a significant bias favoring consultants (P = 0.022). Inter-rater reliability was very good (Cronbach's α = 0.93). However, one rater assessing five procedures or two raters each assessing four procedures were necessary to secure a generalizability coefficient of 0.80. The assessment tool demonstrated construct validity by discriminating between trainees and experienced physicians (P = 0.034). Competency in mediastinal staging of NSCLC using EUS and EUS - FNA can be assessed in a reliable and valid way using the EUSAT assessment tool. Measuring and defining competency and training requirements could improve EUS quality and benefit patient care. © Georg Thieme Verlag KG Stuttgart · New York.

Using the HSE stress indicator tool in a military context.

PubMed

Bridger, Robert S; Dobson, Karen; Davison, Hannah

2016-01-01

An assessment of the Health and Safety Executive's (HSE) stress indicator tool was conducted to determine whether it was suitable for use with Ministry of Defence (MoD) personnel. A total of 1031 respondents from the Army, Navy, Air Force and MoD civilians completed a questionnaire containing the HSE tool and supplementary measures of work-life balance, engagement, deployment, leave taken and hours of work. Six measures of adverse reaction to the stressors were also reported: perceptions of job stress, job stress and health, psychological strain, fatigue after work, work ability and quality of working life. The stressor scales, particularly the 'demands' and 'relationships' scales, were associated with adverse outcomes as was the work-life balance scale. The HSE tool had some validity when used with MoD personnel, but its content was too narrow. The content validity of the tool can be improved for use in a military context with the addition of a 'work-life balance' scale'. The HSE stress tool was tested with a mixed sample of MoD employees. The ‘Demands and “Relationships” scales were associated with adverse outcomes. An additional Work–Life Balance’ scale improved the content validity, demonstrating the importance of assessing the psychometric qualities of scales when used within particular contexts to ensure validity.

Validation of the Virtual MET as an assessment tool for executive functions.

PubMed

Rand, Debbie; Basha-Abu Rukan, Soraya; Weiss, Patrice L Tamar; Katz, Noomi

2009-08-01

The purpose of this study was to establish ecological validity and initial construct validity of a Virtual Multiple Errands Test (VMET) as an assessment tool for executive functions. It was implemented within the Virtual Mall (VMall), a novel functional video-capture virtual shopping environment. The main objectives were (1) to examine the relationships between the performance of three groups of participants in the Multiple Errands Test (MET) carried out in a real shopping mall and their performance in the VMET, (2) to assess the relationships between the MET and VMET of the post-stroke participant's level of executive functioning and independence in instrumental activities of daily living, and (3) to compare the performance of post-stroke participants to those of healthy young and older controls in both the MET and VMET. The study population included three groups; post-stroke participants (n = 9), healthy young participants (n = 20), and healthy older participants (n = 20). The VMET was able to differentiate between two age groups of healthy participants and between healthy and post-stroke participants thus demonstrating that it is sensitive to brain injury and ageing and supports construct validity between known groups. In addition, significant correlations were found between the MET and the VMET for both the post-stroke participants and older healthy participants. This provides initial support for the ecological validity of the VMET as an assessment tool of executive functions. However, further psychometric data on temporal stability are needed, namely test-retest reliability and responsiveness, before it is ready for clinical application. Further research using the VMET as an assessment tool within the VMall with larger groups and in additional populations is also recommended.

Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

PubMed

Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

2016-04-01

History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy Association.

Reliability and validity of play-based assessments of motor and cognitive skills for infants and young children: a systematic review.

PubMed

O'Grady, Michael G; Dusing, Stacey C

2015-01-01

Play is vital for development. Infants and children learn through play. Traditional standardized developmental tests measure whether a child performs individual skills within controlled environments. Play-based assessments can measure skill performance during natural, child-driven play. The purpose of this study was to systematically review reliability, validity, and responsiveness of all play-based assessments that quantify motor and cognitive skills in children from birth to 36 months of age. Studies were identified from a literature search using PubMed, ERIC, CINAHL, and PsycINFO databases and the reference lists of included papers. Included studies investigated reliability, validity, or responsiveness of play-based assessments that measured motor and cognitive skills for children to 36 months of age. Two reviewers independently screened 40 studies for eligibility and inclusion. The reviewers independently extracted reliability, validity, and responsiveness data. They examined measurement properties and methodological quality of the included studies. Four current play-based assessment tools were identified in 8 included studies. Each play-based assessment tool measured motor and cognitive skills in a different way during play. Interrater reliability correlations ranged from .86 to .98 for motor development and from .23 to .90 for cognitive development. Test-retest reliability correlations ranged from .88 to .95 for motor development and from .45 to .91 for cognitive development. Structural validity correlations ranged from .62 to .90 for motor development and from .42 to .93 for cognitive development. One study assessed responsiveness to change in motor development. Most studies had small and poorly described samples. Lack of transparency in data management and statistical analysis was common. Play-based assessments have potential to be reliable and valid tools to assess cognitive and motor skills, but higher-quality research is needed. Psychometric properties should be considered for each play-based assessment before it is used in clinical and research practice. © 2015 American Physical Therapy Association.

Development and validation of a cancer-specific swallowing assessment tool: MASA-C.

PubMed

Carnaby, Giselle D; Crary, Michael A

2014-03-01

We present data from a sample of patients receiving radiotherapy for head/neck cancer to define and measure the validity of a new clinical assessment measure for swallowing. Fifty-eight patients undergoing radiotherapy (±chemotherapy) for head/neck cancer (HNC) supported the development of a physiology-based assessment tool of swallowing (Mann Assessment of Swallowing Ability--Cancer: MASA-C) administered at two time points (baseline and following radiotherapy treatment). The new exam was evaluated for internal consistency of items using Cronbach's alpha. Reliability of measurement was evaluated with intraclass correlation (ICC) and the Kappa statistic between two independent raters. Concurrent validity was established through comparison with the original MASA examination and against the referent standard videofluoroscopic swallowing examination (VFE). Sensitivity, specificity, and likelihood ratios along with 95 % confidence intervals (CIs) were derived for comparison of the two evaluation forms (MASA vs. MASA-C). Accuracy of diagnostic precision was displayed using receiver operator characteristic curves. The new MASA-C tool demonstrated superior validity to the original MASA examination applied to a HNC population. In comparison to the VFE referent exam, the MASA-C revealed strong sensitivity and specificity (Se 83, Sp 96), predictive values (positive predictive value (PPV) 0.95, negative predictive value (NPV) 0.86), and likelihood ratios (21.6). In addition, it demonstrated good reliability (ICC = 0.96) between speech-language pathology raters. The MASA-C is a reliable and valid scale that is sensitive to differences in swallowing performance in HNC patients with and without dysphagia. Future longitudinal evaluation of this tool in larger samples is suggested. The development and refinement of this swallowing assessment tool for use in multidisciplinary HNC teams will facilitate earlier identification of patients with swallowing difficulties and enable more efficient allocation of resources to the management of dysphagia in this population. The MASA-C may also prove useful in future clinical HNC rehabilitation trials with this population.

Assessing the Clinical Skills of Dental Students: A Review of the Literature

ERIC Educational Resources Information Center

Taylor, Carly L.; Grey, Nick; Satterthwaite, Julian D.

2013-01-01

Education, from a student perspective, is largely driven by assessment. An effective assessment tool should be both valid and reliable, yet this is often not achieved. The aim of this literature review is to identify and appraise the evidence base for assessment tools used primarily in evaluating clinical skills of dental students. Methods:…

Structured learning for robotic surgery utilizing a proficiency score: a pilot study.

PubMed

Hung, Andrew J; Bottyan, Thomas; Clifford, Thomas G; Serang, Sarfaraz; Nakhoda, Zein K; Shah, Swar H; Yokoi, Hana; Aron, Monish; Gill, Inderbir S

2017-01-01

We evaluated feasibility and benefit of implementing structured learning in a robotics program. Furthermore, we assessed validity of a proficiency assessment tool for stepwise graduation. Teaching cases included robotic radical prostatectomy and partial nephrectomy. Procedure steps were categorized: basic, intermediate, and advanced. An assessment tool ["proficiency score" (PS)] was developed to evaluate ability to safely and autonomously complete a step. Graduation required a passing PS (PS ≥ 3) on three consecutive attempts. PS and validated global evaluative assessment of robotic skills (GEARS) were evaluated for completed steps. Linear regression was utilized to determine postgraduate year/PS relationship (construct validity). Spearman's rank correlation coefficient measured correlation between PS and GEARS evaluations (concurrent validity). Intraclass correlation (ICC) evaluated PS agreement between evaluator classes. Twenty-one robotic trainees participated within the pilot program, completing a median of 14 (2-69) cases each. Twenty-three study evaluators scored 14 (1-60) cases. Over 4 months, 229/294 (78 %) cases were designated "teaching" cases. Residents completed 91 % of possible evaluations; faculty completed 78 %. Verbal and quantitative feedback received by trainees increased significantly (p = 0.002, p < 0.001, respectively). Average PS increased with PGY (post-graduate year) for basic and intermediate steps (regression slopes: 0.402 (p < 0.0001), 0.323 (p < 0.0001), respectively) (construct validation). Overall, PS correlated highly with GEARS (ρ = 0.81, p < 0.0001) (concurrent validity). ICC was 0.77 (95 % CI 0.61-0.88) for resident evaluations. Structured learning can be implemented in an academic robotic program with high levels of trainee and evaluator participation, encouraging both quantitative and verbal feedback. A proficiency assessment tool developed for step-specific proficiency has construct and concurrent validity.

Reliability and Validity of the Chinese Version of FACIT-AI, a New Tool for Assessing Quality of Life in Patients with Malignant Ascites.

PubMed

Lou, Yanni; Lu, Linghui; Li, Yuan; Liu, Meng; Bredle, Jason M; Jia, Liqun

2015-10-01

The study objective was to determine the reliability and validity of the Chinese version of the Functional Assessment of Chronic Illness Therapy - Ascites Index (FACIT-AI). A forward-backward translation procedure was adopted to develop the Chinese version of the FACIT-AI, which was tested in 69 patients with malignant ascites. Cronbach's α, split-half reliability, and test-retest reliability were used to assess the reliability of the scale. The content validity index was used to assess the content validity, while factor analysis was used for construct validity and correlation analysis was used for criterion validity. The Cronbach's α was 0.772 for the total scale, and the split-half reliability was 0.693. The test-retest correlation was 0.972. The content validity index for the scale was 0.8-1.0. Four factors were extracted by factor analysis, and these contributed 63.51% of the total variance. Item-total correlations ranged from 0.591 to 0.897, and these were correlated with visual analog scale scores (correlation coefficient, 0.889; P<0.01). The Chinese version of the FACIT-AI has good reliability and validity and can be used as a tool to measure quality of life in Chinese patients with malignant ascites.

Testing the validity, reliability and utility of the Self-Administration of Medication (SAM) tool in patients undergoing rehabilitation.

PubMed

Anderson, Jessica; Manias, Elizabeth; Kusljic, Snezana; Finch, Sue

2014-01-01

Determination of patients' ability to self-administer medications in the hospital has largely been determined using the subjective judgment of health professionals. To examine the validity, reliability and utility of the Self-Administration of Medication (SAM) tool as an objective means to determine patients' ability to self-administer in a rehabilitation unit of a public teaching hospital in Melbourne, Australia. To assess validity of the SAM tool, associations were examined between the total SAM tool score and of the patients' competence to self-administer from the perceptions of the tool administrator, patients and nurses. Validity also was determined from a principal component analysis. Pearson correlations were calculated for how SAM scores related to scores obtained from the Functional Independence Measure (FIM) and Barthel Score Index (BSI). To assess the SAM tool's reliability, a Cronbach's alpha coefficient was calculated. Utility of the SAM tool was evidenced by documenting its administration time. One hundred patients participated in this study. The SAM tool had a Cronbach's alpha coefficient of 0.75 and took a mean time of 5.36 min to complete. The capability to self-medicate section of the SAM tool had strong correlations with the FIM (r = 0.485) and BSI (r = 0.472) data, respectively, and the total SAM tool had moderate and strong correlations with the nurses' (r = 0.315) and tool administrator's (r = 0.632) perceptions of patients' ability to self-administer, respectively. Bland-Altman and ROC curve analyses showed poor agreement between the total SAM tool score and the nurses' perceptions. The SAM tool demonstrated acceptable overall internal consistency. It only requires a short time to be completed and is more objective than seeking out health professionals' perceptions. Additional research is needed to further validate this approach to determining patients' ability to self-medicate. Crown Copyright © 2014. Published by Elsevier Inc. All rights reserved.

Assessing Households Preparedness for Earthquakes: An Exploratory Study in the Development of a Valid and Reliable Persian-version Tool.

PubMed

Ardalan, Ali; Sohrabizadeh, Sanaz

2016-02-25

Iran is placed among countries suffering from the highest number of earthquake casualties. Household preparedness, as one component of risk reduction efforts, is often supported in quake-prone areas. In Iran, lack of a valid and reliable household preparedness tool was reported by previous disaster studies. This study is aimed to fill this gap by developing a valid and reliable tool for assessing household preparedness in the event of an earthquake.  This survey was conducted through three phases including literature review and focus group discussions with the participation of eight key informants, validity measurements and reliability measurements. Field investigation was completed with the participation of 450 households within three provinces of Iran. Content validity, construct validity, the use of factor analysis; internal consistency using Cronbach's alpha coefficient, and test-retest reliability were carried out to develop the tool.  Based on the CVIs, ranging from 0.80 to 0.100, and exploratory factor analysis with factor loading of more than 0.5, all items were valid. The amount of Cronbach's alpha (0.7) and test-retest examination by Spearman correlations indicated that the scale was also reliable. The final instrument consisted of six categories and 18 questions including actions at the time of earthquakes, nonstructural safety, structural safety, hazard map, communications, drill, and safety skills.  Using a Persian-version tool that is adjusted to the socio-cultural determinants and native language may result in more trustful information on earthquake preparedness. It is suggested that disaster managers and researchers apply this tool in their future household preparedness projects. Further research is needed to make effective policies and plans for transforming preparedness knowledge into behavior.

Cross-cultural validity of standardized motor development screening and assessment tools: a systematic review.

PubMed

Mendonça, Bianca; Sargent, Barbara; Fetters, Linda

2016-12-01

To investigate whether standardized motor development screening and assessment tools that are used to evaluate motor abilities of children aged 0 to 2 years are valid in cultures other than those in which the normative sample was established. This was a systematic review in which six databases were searched. Studies were selected based on inclusion/exclusion criteria and appraised for evidence level and quality. Study variables were extracted. Twenty-three studies representing six motor development screening and assessment tools in 16 cultural contexts met the inclusion criteria: Alberta Infant Motor Scale (n=7), Ages and Stages Questionnaire, 3rd edition (n=2), Bayley Scales of Infant and Toddler Development, 3rd edition (n=8), Denver Developmental Screening Test, 2nd edition (n=4), Harris Infant Neuromotor Test (n=1), and Peabody Developmental Motor Scales, 2nd edition (n=1). Thirteen studies found significant differences between the cultural context and normative sample. Two studies established reliability and/or validity of standardized motor development assessments in high-risk infants from different cultural contexts. Five studies established new population norms. Eight studies described the cross-cultural adaptation of a standardized motor development assessment. Standardized motor development assessments have limited validity in cultures other than that in which the normative sample was established. Their use can result in under- or over-referral for services. © 2016 Mac Keith Press.

A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

PubMed

Rana, Gurpreet K; Bradley, Doreen R; Hamstra, Stanley J; Ross, Paula T; Schumacher, Robert E; Frohna, John G; Haftel, Hilary M; Lypson, Monica L

2011-01-01

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center. Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies. Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts. The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

Assessing adherence to the evidence base in the management of poststroke dysphagia.

PubMed

Burton, Christopher; Pennington, Lindsay; Roddam, Hazel; Russell, Ian; Russell, Daphne; Krawczyk, Karen; Smith, Hilary A

2006-01-01

To evaluate the reliability and responsiveness to change of an audit tool to assess adherence to evidence of effectiveness in the speech and language therapy (SLT) management of poststroke dysphagia. The tool was used to review SLT practice as part of a randomized study of different education strategies. Medical records were audited before and after delivery of the trial intervention. Seventeen SLT departments in the north-west of England participated in the study. The assessment tool was used to assess the medical records of 753 patients before and 717 patients after delivery of the trial intervention across the 17 departments. A target of 10 records per department per month was sought, using systematic sampling with a random start. Inter- and intra-rater reliability were explored, together with the tool's internal consistency and responsiveness to change. The assessment tool had high face validity, although internal consistency was low (ra = 0.37). Composite scores on the tool were however responsive to differences between SLT departments. Both inter- and intra-rater reliability ranged from 'substantial' to 'near perfect' across all items. The audit tool has high face validity and measurement reliability. The use of a composite adherence score should, however, proceed with caution as internal consistency is low.

Validity and reliability of the infant breastfeeding assessment tool, the mother baby assessment tool, and the LATCH scoring system.

PubMed

Altuntas, Nilgun; Turkyilmaz, Canan; Yildiz, Havva; Kulali, Ferit; Hirfanoglu, Ibrahim; Onal, Esra; Ergenekon, Ebru; Koç, Esin; Atalay, Yıldız

2014-05-01

We aimed to evaluate the validity and reliability of the Infant Breastfeeding Assessment Tool (IBFAT), the Mother Baby Assessment (MBA) Tool, and the LATCH scoring system. Mothers who delivered healthy, full-term infants in the Obstetrics & Gynecology Service of Gazi University, Ankara, Turkey, between December 2013 and January 2014 and their infants were included in the study. Forty-six randomly selected breastfeeding sessions were monitored and scored simultaneously by three researchers (Raters 1, 2, and 3) using LATCH, IBFAT, and the MBA Tool. Researchers put the score sheets in an envelope in order to hide them from each other. The compatibility of the scores given by three researchers was assessed by statistical methods. We found positive and significant correlation coefficients between 0.81 to 0.88 for the total MBA score, between 0.90 to 0.95 for the total IBFAT score, and between 0.85 to 0.91 for the total LATCH score. Correlation coefficients testing these three tools ranged from 0.71 to 0.88, with the minimum value being noted for the correlation between LATCH and IBFAT scores and the maximum value being noted for the correlation between LATCH and MBA scores. We found positive and significant correlations between researchers' scores for 46 observations using the three assessment tools. This study showed that these above-mentioned tools were compatible for the assessment of the efficiency of breastfeeding.

Development of the Music Therapy Assessment Tool for Advanced Huntington's Disease: A Pilot Validation Study.

PubMed

O'Kelly, Julian; Bodak, Rebeka

2016-01-01

Case studies of people with Huntington's disease (HD) report that music therapy provides a range of benefits that may improve quality of life; however, no robust music therapy assessment tools exist for this population. Develop and conduct preliminary psychometric testing of a music therapy assessment tool for patients with advanced HD. First, we established content and face validity of the Music Therapy Assessment Tool for Advanced HD (MATA-HD) through focus groups and field testing. Second, we examined psychometric properties of the resulting MATA-HD in terms of its construct validity, internal consistency, and inter-rater and intra-rater reliability over 10 group music therapy sessions with 19 patients. The resulting MATA-HD included a total of 15 items across six subscales (Arousal/Attention, Physical Presentation, Communication, Musical, Cognition, and Psychological/Behavioral). We found good construct validity (r ≥ 0.7) for Mood, Communication Level, Communication Effectiveness, Choice, Social Behavior, Arousal, and Attention items. Cronbach's α of 0.825 indicated good internal consistency across 11 items with a common focus of engagement in therapy. The inter-rater reliability (IRR) Intra-Class Coefficient (ICC) scores averaged 0.65, and a mean intra-rater ICC reliability of 0.68 was obtained. Further training and retesting provided a mean of IRR ICC of 0.7. Preliminary data indicate that the MATA-HD is a promising tool for measuring patient responses to music therapy interventions across psychological, physical, social, and communication domains of functioning in patients with advanced HD. © the American Music Therapy Association 2016. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Construct validity of the ovine model in endoscopic sinus surgery training.

PubMed

Awad, Zaid; Taghi, Ali; Sethukumar, Priya; Tolley, Neil S

2015-03-01

To demonstrate construct validity of the ovine model as a tool for training in endoscopic sinus surgery (ESS). Prospective, cross-sectional evaluation study. Over 18 consecutive months, trainees and experts were evaluated in their ability to perform a range of tasks (based on previous face validation and descriptive studies conducted by the same group) relating to ESS on the sheep-head model. Anonymized randomized video recordings of the above were assessed by two independent and blinded assessors. A validated assessment tool utilizing a five-point Likert scale was employed. Construct validity was calculated by comparing scores across training levels and experts using mean and interquartile range of global and task-specific scores. Subgroup analysis of the intermediate group ascertained previous experience. Nonparametric descriptive statistics were used, and analysis was carried out using SPSS version 21 (IBM, Armonk, NY). Reliability of the assessment tool was confirmed. The model discriminated well between different levels of expertise in global and task-specific scores. A positive correlation was noted between year in training and both global and task-specific scores (P < .001). Experience of the intermediate group was variable, and the number of ESS procedures performed under supervision had the highest impact on performance. This study describes an alternative model for ESS training and assessment. It is also the first to demonstrate construct validity of the sheep-head model for ESS training. © 2014 The American Laryngological, Rhinological and Otological Society, Inc.

Children and Young People-Mental Health Safety Assessment Tool (CYP-MH SAT) study: Protocol for the development and psychometric evaluation of an assessment tool to identify immediate risk of self-harm and suicide in children and young people (10–19 years) in acute paediatric hospital settings

PubMed Central

Walker, Gemma M; Carter, Tim; Aubeeluck, Aimee; Witchell, Miranda; Coad, Jane

2018-01-01

Introduction Currently, no standardised, evidence-based assessment tool for assessing immediate self-harm and suicide in acute paediatric inpatient settings exists. Aim The aim of this study is to develop and test the psychometric properties of an assessment tool that identifies immediate risk of self-harm and suicide in children and young people (10–19 years) in acute paediatric hospital settings. Methods and analysis Development phase: This phase involved a scoping review of the literature to identify and extract items from previously published suicide and self-harm risk assessment scales. Using a modified electronic Delphi approach, these items will then be rated according to their relevance for assessment of immediate suicide or self-harm risk by expert professionals. Inclusion of items will be determined by 65%–70% consensus between raters. Subsequently, a panel of expert members will convene to determine the face validity, appropriate phrasing, item order and response format for the finalised items. Psychometric testing phase: The finalised items will be tested for validity and reliability through a multicentre, psychometric evaluation. Psychometric testing will be undertaken to determine the following: internal consistency, inter-rater reliability, convergent, divergent validity and concurrent validity. Ethics and dissemination Ethical approval was provided by the National Health Service East Midlands—Derby Research Ethics Committee (17/EM/0347) and full governance clearance received by the Health Research Authority and local participating sites. Findings from this study will be disseminated to professionals and the public via peer-reviewed journal publications, popular social media and conference presentations. PMID:29654046

Review on pen-and-paper-based observational methods for assessing ergonomic risk factors of computer work.

PubMed

Rahman, Mohd Nasrull Abdol; Mohamad, Siti Shafika

2017-01-01

Computer works are associated with Musculoskeletal Disorders (MSDs). There are several methods have been developed to assess computer work risk factor related to MSDs. This review aims to give an overview of current techniques available for pen-and-paper-based observational methods in assessing ergonomic risk factors of computer work. We searched an electronic database for materials from 1992 until 2015. The selected methods were focused on computer work, pen-and-paper observational methods, office risk factors and musculoskeletal disorders. This review was developed to assess the risk factors, reliability and validity of pen-and-paper observational method associated with computer work. Two evaluators independently carried out this review. Seven observational methods used to assess exposure to office risk factor for work-related musculoskeletal disorders were identified. The risk factors involved in current techniques of pen and paper based observational tools were postures, office components, force and repetition. From the seven methods, only five methods had been tested for reliability. They were proven to be reliable and were rated as moderate to good. For the validity testing, from seven methods only four methods were tested and the results are moderate. Many observational tools already exist, but no single tool appears to cover all of the risk factors including working posture, office component, force, repetition and office environment at office workstations and computer work. Although the most important factor in developing tool is proper validation of exposure assessment techniques, the existing observational method did not test reliability and validity. Futhermore, this review could provide the researchers with ways on how to improve the pen-and-paper-based observational method for assessing ergonomic risk factors of computer work.

Commercially available gaming systems as clinical assessment tools to improve value in the orthopaedic setting: a systematic review.

PubMed

Ruff, Jessica; Wang, Tiffany L; Quatman-Yates, Catherine C; Phieffer, Laura S; Quatman, Carmen E

2015-02-01

Commercially available gaming systems (CAGS) such as the Wii Balance Board (WBB) and Microsoft Xbox with Kinect (Xbox Kinect) are increasingly used as balance training and rehabilitation tools. The purpose of this review was to answer the question, "Are commercially available gaming systems valid and reliable instruments for use as clinical diagnostic and functional assessment tools in orthopaedic settings?" and provide a summary of relevant studies, identify their strengths and weaknesses, and generate conclusions regarding general validity/reliability of WBB and Xbox Kinect in orthopaedics. A systematic search was performed using MEDLINE (1996-2013) and Scopus (1996-2013). Inclusion criteria were minimum of 5 subjects, full manuscript provided in English or translated, and studies incorporating investigation of CAG measurement properties. Exclusion criteria included reviews, systematic reviews, summary/clinical commentaries, or case studies; conference proceedings/presentations; cadaveric studies; studies of non-reversible, non-orthopaedic-related musculoskeletal disease; non-human trials; and therapeutic studies not reporting comparative evaluation to already established functional assessment criteria. All studies meeting inclusion and exclusion criteria were appraised for quality by two independent reviewers. Evidence levels (I-V) were assigned to each study based on established methodological criteria. 3 Level II, 7 level III, and 1 Level IV studies met inclusion criteria and provided information related to the use of the WBB and Xbox Kinect as clinical assessment tools in the field of orthopaedics. Studies have used the WBB in a variety of clinical applications, including the measurement of center of pressure (COP), measurement of medial-to-lateral (M/L) or anterior-to-posterior (A/P) symmetry, assessment anatomic landmark positioning, and assessment of fall risk. However, no uniform protocols or outcomes were used to evaluate the quality of the WBB as a clinical assessment tool; therefore a wide range of sensitivities, specificities, accuracies, and validities were reported. Currently it is not possible to make a universal generalization about the clinical utility of CAGS in the field of orthopaedics. However, there is evidence to support using the WBB and the Xbox Kinect as tools to obtain reliable and valid COP measurements. The Wii Fit Game may specifically provide reliable and valid measurements for predicting fall risk. Copyright © 2014 Elsevier Ltd. All rights reserved.

Evaluation of the validity of osteoporosis and fracture risk assessment tools (IOF One Minute Test, SCORE, and FRAX) in postmenopausal Palestinian women.

PubMed

Kharroubi, Akram; Saba, Elias; Ghannam, Ibrahim; Darwish, Hisham

2017-12-01

The need for simple self-assessment tools is necessary to predict women at high risk for developing osteoporosis. In this study, tools like the IOF One Minute Test, Fracture Risk Assessment Tool (FRAX), and Simple Calculated Osteoporosis Risk Estimation (SCORE) were found to be valid for Palestinian women. The threshold for predicting women at risk for each tool was estimated. The purpose of this study is to evaluate the validity of the updated IOF (International Osteoporosis Foundation) One Minute Osteoporosis Risk Assessment Test, FRAX, SCORE as well as age alone to detect the risk of developing osteoporosis in postmenopausal Palestinian women. Three hundred eighty-two women 45 years and older were recruited including 131 women with osteoporosis and 251 controls following bone mineral density (BMD) measurement, 287 completed questionnaires of the different risk assessment tools. Receiver operating characteristic (ROC) curves were evaluated for each tool using bone BMD as the gold standard for osteoporosis. The area under the ROC curve (AUC) was the highest for FRAX calculated with BMD for predicting hip fractures (0.897) followed by FRAX for major fractures (0.826) with cut-off values ˃1.5 and ˃7.8%, respectively. The IOF One Minute Test AUC (0.629) was the lowest compared to other tested tools but with sufficient accuracy for predicting the risk of developing osteoporosis with a cut-off value ˃4 total yes questions out of 18. SCORE test and age alone were also as good predictors of risk for developing osteoporosis. According to the ROC curve for age, women ≥64 years had a higher risk of developing osteoporosis. Higher percentage of women with low BMD (T-score ≤-1.5) or osteoporosis (T-score ≤-2.5) was found among women who were not exposed to the sun, who had menopause before the age of 45 years, or had lower body mass index (BMI) compared to controls. Women who often fall had lower BMI and approximately 27% of the recruited postmenopausal Palestinian women had accidents that caused fractures. Simple self-assessment tools like FRAX without BMD, SCORE, and the IOF One Minute Tests were valid for predicting Palestinian postmenopausal women at high risk of developing osteoporosis.

Going DEEP: guidelines for building simulation-based team assessments.

PubMed

Grand, James A; Pearce, Marina; Rench, Tara A; Chao, Georgia T; Fernandez, Rosemarie; Kozlowski, Steve W J

2013-05-01

Whether for team training, research or evaluation, making effective use of simulation-based technologies requires robust, reliable and accurate assessment tools. Extant literature on simulation-based assessment practices has primarily focused on scenario and instructional design; however, relatively little direct guidance has been provided regarding the challenging decisions and fundamental principles related to assessment development and implementation. The objective of this manuscript is to introduce a generalisable assessment framework supplemented by specific guidance on how to construct and ensure valid and reliable simulation-based team assessment tools. The recommendations reflect best practices in assessment and are designed to empower healthcare educators, professionals and researchers with the knowledge to design and employ valid and reliable simulation-based team assessments. Information and actionable recommendations associated with creating assessments of team processes (non-technical 'teamwork' activities) and performance (demonstration of technical proficiency) are presented which provide direct guidance on how to Distinguish the underlying competencies one aims to assess, Elaborate the measures used to capture team member behaviours during simulation activities, Establish the content validity of these measures and Proceduralise the measurement tools in a way that is systematically aligned with the goals of the simulation activity while maintaining methodological rigour (DEEP). The DEEP framework targets fundamental principles and critical activities that are important for effective assessment, and should benefit healthcare educators, professionals and researchers seeking to design or enhance any simulation-based assessment effort.

Validity and Interrater Reliability of the Visual Quarter-Waste Method for Assessing Food Waste in Middle School and High School Cafeteria Settings.

PubMed

Getts, Katherine M; Quinn, Emilee L; Johnson, Donna B; Otten, Jennifer J

2017-11-01

Measuring food waste (ie, plate waste) in school cafeterias is an important tool to evaluate the effectiveness of school nutrition policies and interventions aimed at increasing consumption of healthier meals. Visual assessment methods are frequently applied in plate waste studies because they are more convenient than weighing. The visual quarter-waste method has become a common tool in studies of school meal waste and consumption, but previous studies of its validity and reliability have used correlation coefficients, which measure association but not necessarily agreement. The aims of this study were to determine, using a statistic measuring interrater agreement, whether the visual quarter-waste method is valid and reliable for assessing food waste in a school cafeteria setting when compared with the gold standard of weighed plate waste. To evaluate validity, researchers used the visual quarter-waste method and weighed food waste from 748 trays at four middle schools and five high schools in one school district in Washington State during May 2014. To assess interrater reliability, researcher pairs independently assessed 59 of the same trays using the visual quarter-waste method. Both validity and reliability were assessed using a weighted κ coefficient. For validity, as compared with the measured weight, 45% of foods assessed using the visual quarter-waste method were in almost perfect agreement, 42% of foods were in substantial agreement, 10% were in moderate agreement, and 3% were in slight agreement. For interrater reliability between pairs of visual assessors, 46% of foods were in perfect agreement, 31% were in almost perfect agreement, 15% were in substantial agreement, and 8% were in moderate agreement. These results suggest that the visual quarter-waste method is a valid and reliable tool for measuring plate waste in school cafeteria settings. Copyright © 2017 Academy of Nutrition and Dietetics. Published by Elsevier Inc. All rights reserved.

Composite Reliability of a Workplace-Based Assessment Toolbox for Postgraduate Medical Education

ERIC Educational Resources Information Center

Moonen-van Loon, J. M. W.; Overeem, K.; Donkers, H. H. L. M.; van der Vleuten, C. P. M.; Driessen, E. W.

2013-01-01

In recent years, postgraduate assessment programmes around the world have embraced workplace-based assessment (WBA) and its related tools. Despite their widespread use, results of studies on the validity and reliability of these tools have been variable. Although in many countries decisions about residents' continuation of training and…

The Language Exposure Assessment Tool: Quantifying Language Exposure in Infants and Children

ERIC Educational Resources Information Center

DeAnda, Stephanie; Bosch, Laura; Poulin-Dubois, Diane; Zesiger, Pascal; Friend, Margaret

2016-01-01

Purpose: The aim of this study was to develop the Language Exposure Assessment Tool (LEAT) and to examine its cross-linguistic validity, reliability, and utility. The LEAT is a computerized interview-style assessment that requests parents to estimate language exposure. The LEAT yields an automatic calculation of relative language exposure and…

Team performance in resuscitation teams: Comparison and critique of two recently developed scoring tools☆

PubMed Central

McKay, Anthony; Walker, Susanna T.; Brett, Stephen J.; Vincent, Charles; Sevdalis, Nick

2012-01-01

Background and aim Following high profile errors resulting in patient harm and attracting negative publicity, the healthcare sector has begun to focus on training non-technical teamworking skills as one way of reducing the rate of adverse events. Within the area of resuscitation, two tools have been developed recently aiming to assess these skills – TEAM and OSCAR. The aims of the study reported here were:1.To determine the inter-rater reliability of the tools in assessing performance within the context of resuscitation.2.To correlate scores of the same resuscitation teams episodes using both tools, thereby determining their concurrent validity within the context of resuscitation.3.To carry out a critique of both tools and establish how best each one may be utilised. Methods The study consisted of two phases – reliability assessment; and content comparison, and correlation. Assessments were made by two resuscitation experts, who watched 24 pre-recorded resuscitation simulations, and independently rated team behaviours using both tools. The tools were critically appraised, and correlation between overall score surrogates was assessed. Results Both OSCAR and TEAM achieved high levels of inter-rater reliability (in the form of adequate intra-class coefficients) and minor significant differences between Wilcoxon tests. Comparison of the scores from both tools demonstrated a high degree of correlation (and hence concurrent validity). Finally, critique of each tool highlighted differences in length and complexity. Conclusion Both OSCAR and TEAM can be used to assess resuscitation teams in a simulated environment, with the tools correlating well with one another. We envisage a role for both tools – with TEAM giving a quick, global assessment of the team, but OSCAR enabling more detailed breakdown of the assessment, facilitating feedback, and identifying areas of weakness for future training. PMID:22561464

Validating a work group climate assessment tool for improving the performance of public health organizations

PubMed Central

Perry, Cary; LeMay, Nancy; Rodway, Greg; Tracy, Allison; Galer, Joan

2005-01-01

Background This article describes the validation of an instrument to measure work group climate in public health organizations in developing countries. The instrument, the Work Group Climate Assessment Tool (WCA), was applied in Brazil, Mozambique, and Guinea to assess the intermediate outcomes of a program to develop leadership for performance improvement. Data were collected from 305 individuals in 42 work groups, who completed a self-administered questionnaire. Methods The WCA was initially validated using Cronbach's alpha reliability coefficient and exploratory factor analysis. This article presents the results of a second validation study to refine the initial analyses to account for nested data, to provide item-level psychometrics, and to establish construct validity. Analyses included eigenvalue decomposition analysis, confirmatory factor analysis, and validity and reliability analyses. Results This study confirmed the validity and reliability of the WCA across work groups with different demographic characteristics (gender, education, management level, and geographical location). The study showed that there is agreement between the theoretical construct of work climate and the items in the WCA tool across different populations. The WCA captures a single perception of climate rather than individual sub-scales of clarity, support, and challenge. Conclusion The WCA is useful for comparing the climates of different work groups, tracking the changes in climate in a single work group over time, or examining differences among individuals' perceptions of their work group climate. Application of the WCA before and after a leadership development process can help work groups hold a discussion about current climate and select a target for improvement. The WCA provides work groups with a tool to take ownership of their own group climate through a process that is simple and objective and that protects individual confidentiality. PMID:16223447

A test of the validity of the motivational interviewing treatment integrity code.

PubMed

Forsberg, Lars; Berman, Anne H; Kallmén, Håkan; Hermansson, Ulric; Helgason, Asgeir R

2008-01-01

To evaluate the Swedish version of the Motivational Interviewing Treatment Code (MITI), MITI coding was applied to tape-recorded counseling sessions. Construct validity was assessed using factor analysis on 120 MITI-coded sessions. Discriminant validity was assessed by comparing MITI coding of motivational interviewing (MI) sessions with information- and advice-giving sessions as well as by comparing MI-trained practitioners with untrained practitioners. A principal-axis factoring analysis yielded some evidence for MITI construct validity. MITI differentiated between practitioners with different levels of MI training as well as between MI practitioners and advice-giving counselors, thus supporting discriminant validity. MITI may be used as a training tool together with supervision to confirm and enhance MI practice in clinical settings. MITI can also serve as a tool for evaluating MI integrity in clinical research.

Development of a self-assessment teamwork tool for use by medical and nursing students.

PubMed

Gordon, Christopher J; Jorm, Christine; Shulruf, Boaz; Weller, Jennifer; Currie, Jane; Lim, Renee; Osomanski, Adam

2016-08-24

Teamwork training is an essential component of health professional student education. A valid and reliable teamwork self-assessment tool could assist students to identify desirable teamwork behaviours with the potential to promote learning about effective teamwork. The aim of this study was to develop and evaluate a self-assessment teamwork tool for health professional students for use in the context of emergency response to a mass casualty. The authors modified a previously published teamwork instrument designed for experienced critical care teams for use with medical and nursing students involved in mass casualty simulations. The 17-item questionnaire was administered to students immediately following the simulations. These scores were used to explore the psychometric properties of the tool, using Exploratory and Confirmatory Factor Analysis. 202 (128 medical and 74 nursing) students completed the self-assessment teamwork tool for students. Exploratory factor analysis revealed 2 factors (5 items - Teamwork coordination and communication; 4 items - Information sharing and support) and these were justified with confirmatory factor analysis. Internal consistency was 0.823 for Teamwork coordination and communication, and 0.812 for Information sharing and support. These data provide evidence to support the validity and reliability of the self-assessment teamwork tool for students This self-assessment tool could be of value to health professional students following team training activities to help them identify the attributes of effective teamwork.

Development and Validation of a Family Meeting Assessment Tool (FMAT).

PubMed

Hagiwara, Yuya; Healy, Jennifer; Lee, Shuko; Ross, Jeanette; Fischer, Dixie; Sanchez-Reilly, Sandra

2018-01-01

A cornerstone procedure in Palliative Medicine is to perform family meetings. Learning how to lead a family meeting is an important skill for physicians and others who care for patients with serious illnesses and their families. There is limited evidence on how to assess best practice behaviors during end-of-life family meetings. Our aim was to develop and validate an observational tool to assess trainees' ability to lead a simulated end-of-life family meeting. Building on evidence from published studies and accrediting agency guidelines, an expert panel at our institution developed the Family Meeting Assessment Tool. All fourth-year medical students (MS4) and eight geriatric and palliative medicine fellows (GPFs) were invited to participate in a Family Meeting Objective Structured Clinical Examination, where each trainee assumed the physician role leading a complex family meeting. Two evaluators observed and rated randomly chosen students' performances using the Family Meeting Assessment Tool during the examination. Inter-rater reliability was measured using percent agreement. Internal consistency was measured using Cronbach α. A total of 141 trainees (MS4 = 133 and GPF = 8) and 26 interdisciplinary evaluators participated in the study. Internal reliability (Cronbach α) of the tool was 0.85. Number of trainees rated by two evaluators was 210 (MS4 = 202 and GPF = 8). Rater agreement was 84%. Composite scores, on average, were significantly higher for fellows than for medical students (P < 0.001). Expert-based content, high inter-rater reliability, good internal consistency, and ability to predict educational level provided initial evidence for construct validity for this novel assessment tool. Copyright © 2017 American Academy of Hospice and Palliative Medicine. All rights reserved.

An Integrative Review of Pediatric Fall Risk Assessment Tools.

PubMed

DiGerolamo, Kimberly; Davis, Katherine Finn

Patient fall prevention begins with accurate risk assessment. However, sustained improvements in prevention and quality of care include use of validated fall risk assessment tools (FRATs). The goal of FRATs is to identify patients at highest risk. Adult FRATs are often borrowed from to create tools for pediatric patients. Though factors associated with pediatric falls in the hospital setting are similar to those in adults, such as mobility, medication use, and cognitive impairment, adult FRATs and the factors associated with them do not adequately assess risk in children. Articles were limited to English language, ages 0-21years, and publish date 2006-2015. The search yielded 22 articles. Ten were excluded as the population was primarily adult or lacked discussion of a FRAT. Critical appraisal and findings were synthesized using the Johns Hopkins Nursing evidence appraisal system. Twelve articles relevant to fall prevention in the pediatric hospital setting that discussed fall risk assessment and use of a FRAT were reviewed. Comparison between and accuracy of FRATs is challenged when different classifications, definitions, risk stratification, and inclusion criteria are used. Though there are several pediatric FRATs published in the literature, none have been found to be reliable and valid across institutions and diverse populations. This integrative review highlights the importance of choosing a FRAT based on an institution's identified risk factors and validating the tool for one's own patient population as well as using the tool in conjunction with nursing clinical judgment to guide interventions. Copyright © 2017 Elsevier Inc. All rights reserved.

Developing a tool for assessing competency in root cause analysis.

PubMed

Gupta, Priyanka; Varkey, Prathibha

2009-01-01

Root cause analysis (RCA) is a tool for identifying the key cause(s) contributing to a sentinel event or near miss. Although training in RCA is gaining popularity in medical education, there is no published literature on valid or reliable methods for assessing competency in the same. A tool for assessing competency in RCA was pilot tested as part of an eight-station Objective Structured Clinical Examination that was conducted at the completion of a three-week quality improvement (QI) curriculum for the Mayo Clinic Preventive Medicine and Endocrinology fellowship programs. As part of the curriculum, fellows completed a QI project to enhance physician communication of the diagnosis and treatment plan at the end of a patient visit. They had a didactic session on RCA, followed by process mapping of the information flow at the project clinic, after which fellows conducted an actual RCA using the Ishikawa fishbone diagram. For the RCA competency assessment, fellows performed an RCA regarding a scenario describing an adverse medication event and provided possible solutions to prevent such errors in the future. All faculty strongly agreed or agreed that they were able to accurately assess competency in RCA using the tool. Interrater reliability for the global competency rating and checklist scoring were 0.96 and 0.85, respectively. Internal consistency (Cronbach's alpha) was 0.76. Six of eight of the fellows found the difficulty level of the test to be optimal. Assessment methods must accompany education programs to ensure that graduates are competent in QI methodologies and are able to apply them effectively in the workplace. The RCA assessment tool was found to be a valid, reliable, feasible, and acceptable method for assessing competency in RCA. Further research is needed to examine its predictive validity and generalizability.

Assessing Procedural Competence: Validity Considerations.

PubMed

Pugh, Debra M; Wood, Timothy J; Boulet, John R

2015-10-01

Simulation-based medical education (SBME) offers opportunities for trainees to learn how to perform procedures and to be assessed in a safe environment. However, SBME research studies often lack robust evidence to support the validity of the interpretation of the results obtained from tools used to assess trainees' skills. The purpose of this paper is to describe how a validity framework can be applied when reporting and interpreting the results of a simulation-based assessment of skills related to performing procedures. The authors discuss various sources of validity evidence because they relate to SBME. A case study is presented.

[Assessing work-related stress: an Italian adaptation of the HSE Management Standards Work-Related Stress Indicator Tool].

PubMed

Marcatto, Francesco; D'Errico, Giuseppe; Di Blas, Lisa; Ferrante, Donatella

2011-01-01

The aim of this paper is to present a preliminary validation of an Italian adaptation of the HSE Management Standards Work-Related Stress Indicator Tool (IT), an instrument for assessing work-related stress at the organizational level, originally developed in Britain by the Health and Safety Executive. A scale that assesses the physical work environment has been added to the original version of the IT. 190 employees of the University of Trieste have been enrolled in the study. A confirmatory analysis showed a satisfactory fit of the eight-factors structure of the instrument. Further psychometric analysis showed adequate internal consistency of the IT scales and good criterion validity, as evidenced by the correlations with self-perception of stress, work satisfaction and motivation. In conclusion, the Indicator Tool proved to be a valid and reliable instrument for the assessment of work-related stress at the organizational level, and it is also compatible with the instructions provided by the Ministry of Labour and Social Policy (Circular letter 18/11/2010).

Tools used for evaluation of Brazilian children's quality of life

PubMed Central

Souza, João Gabriel S.; Pamponet, Marcela Antunes; Souza, Tamirys Caroline S.; Pereira, Alessandra Ribeiro; Souza, Andrey George S.; Martins, Andréa Maria E. de B. L.

2014-01-01

OBJECTIVE: To review the available tools to evaluate children's quality of life validated for Brazilian language and culture. DATA SOURCES: Search of scientific articles in Medline, Lilacs and SciELO databases using the combination of descriptors "quality of life", "child" and "questionnaires" in Portuguese and English. DATA SYNTHESIS: Among the tools designed to assess children's quality of life validated for the Brazilian language and culture, the Auto questionnaire Qualité de Vie Enfant Imagé (AUQEI), the Child Health Questionnaire - Parent Form 50 (CHQ-PF50), the Pediatric Quality of Life Inventory (PedsQL(tm)) version 4.0 and the Kidscreen-52 are highlighted. Some tools do not include all range of ages and some lack domains that are currently considered relevant in the context of childhood, such as bullying. Moreover, due to the cultural diversity of Brazil, it may be necessary to adapt some instruments or to validate other tools. CONCLUSIONS: There are validated instruments to evaluate children's quality of life in Brazil. However, the validation or the adaptation of other international tools have to be considered in order to overcome current deficiencies. PMID:25119761

The Learning Loss Scale as an Assessment Tool: An Empirical Examination of Convergent Validity with Performative Measures

ERIC Educational Resources Information Center

Hooker, John; Denker, Katherine

2014-01-01

Higher education has placed an increasingly greater value on assessment. The Learning Loss Scale may be an appropriate tool to assess learning across disciplines. In this paper, we review the culture of assessment, conceptualizations of cognitive learning, the Learning Loss Scale, and a theoretical explanation, and then we test this measure to…

Validation to Spanish of the Caring Assessment Tool (CAT-V)

PubMed Central

Ayuso, Rosa María Fernández; Velázquez, Juan Manuel Morillo; Ayuso, David Fernández; de la Torre-Montero, Julio César

2017-01-01

Resume Objective: to translate and validate to Spanish the Caring Assessment Scale tool, CAT-V, by Joanne Duffy, within the framework of Jean Watson; as a secondary objective, it is proposed to evaluate its psychometric properties. There are tools designed to measure the patient’s perception of provided cares, including CAT-V, the subject of our interest, in a way that it can be used in Spanish-speaking patients. Methods: to meet the objectives, it was performed sequential translation and retro-translation of the scale to be validated, through a standardized procedure. The final version of that scale was validated in a sample of 349 patients from four public and two private hospitals in Madrid, Spain. Results: The instrument was translated and validated with high internal consistency (Cronbach’s alpha .953). The subsequent factor analysis revealed a three-factor structure, not coincident with the data from the US population. Conclusion: it is considered that the translation of CAT-V is a suitable instrument to be used in the evaluation of patient care in Ibero-american health centers whose language is Spanish. PMID:29069268

Measurement and Research Tools. Symposium 37. [AHRD Conference, 2001].

ERIC Educational Resources Information Center

2001

This symposium on measurement and research tools consists of three presentations. "An Examination of the Multiple Intelligences Developmental Assessment Scales (MIDAS)" (Albert Wiswell et al.) explores MIDAS's psychometric saliency. Findings indicates this instrument represents an incomplete attempt to develop a valid assessment of…

The Arabic Version of the Mayo-Portland Adaptability Inventory 4: A Validation Study

ERIC Educational Resources Information Center

Hamed, Razan; Tariah, Hashem Abu; Malkawi, Somaya; Holm, Margo B.

2012-01-01

The Mayo-Portland Adaptability Inventory 4 (MPAI-4) is a valid and reliable assessment tool to detect clinical impairments in patients with acquired brain injury. The tool is widely used by rehabilitation therapists worldwide, given its good psychometric properties and its availability in several languages. The purpose of this study was to…

An Examination and Validation of an Adapted Youth Experience Scale for University Sport

ERIC Educational Resources Information Center

Rathwell, Scott; Young, Bradley W.

2016-01-01

Limited tools assess positive development through university sport. Such a tool was validated in this investigation using two independent samples of Canadian university athletes. In Study 1, 605 athletes completed 99 survey items drawn from the Youth Experience Scale (YES 2.0), and separate a priori measurement models were evaluated (i.e., 99…

Quasi-experimental study designs series-paper 6: risk of bias assessment.

PubMed

Waddington, Hugh; Aloe, Ariel M; Becker, Betsy Jane; Djimeu, Eric W; Hombrados, Jorge Garcia; Tugwell, Peter; Wells, George; Reeves, Barney

2017-09-01

Rigorous and transparent bias assessment is a core component of high-quality systematic reviews. We assess modifications to existing risk of bias approaches to incorporate rigorous quasi-experimental approaches with selection on unobservables. These are nonrandomized studies using design-based approaches to control for unobservable sources of confounding such as difference studies, instrumental variables, interrupted time series, natural experiments, and regression-discontinuity designs. We review existing risk of bias tools. Drawing on these tools, we present domains of bias and suggest directions for evaluation questions. The review suggests that existing risk of bias tools provide, to different degrees, incomplete transparent criteria to assess the validity of these designs. The paper then presents an approach to evaluating the internal validity of quasi-experiments with selection on unobservables. We conclude that tools for nonrandomized studies of interventions need to be further developed to incorporate evaluation questions for quasi-experiments with selection on unobservables. Copyright © 2017 Elsevier Inc. All rights reserved.

Patient simulation: a literary synthesis of assessment tools in anesthesiology.

PubMed

Edler, Alice A; Fanning, Ruth G; Chen, Michael I; Claure, Rebecca; Almazan, Dondee; Struyk, Brain; Seiden, Samuel C

2009-12-20

High-fidelity patient simulation (HFPS) has been hypothesized as a modality for assessing competency of knowledge and skill in patient simulation, but uniform methods for HFPS performance assessment (PA) have not yet been completely achieved. Anesthesiology as a field founded the HFPS discipline and also leads in its PA. This project reviews the types, quality, and designated purpose of HFPS PA tools in anesthesiology. We used the systematic review method and systematically reviewed anesthesiology literature referenced in PubMed to assess the quality and reliability of available PA tools in HFPS. Of 412 articles identified, 50 met our inclusion criteria. Seventy seven percent of studies have been published since 2000; more recent studies demonstrated higher quality. Investigators reported a variety of test construction and validation methods. The most commonly reported test construction methods included "modified Delphi Techniques" for item selection, reliability measurement using inter-rater agreement, and intra-class correlations between test items or subtests. Modern test theory, in particular generalizability theory, was used in nine (18%) of studies. Test score validity has been addressed in multiple investigations and shown a significant improvement in reporting accuracy. However the assessment of predicative has been low across the majority of studies. Usability and practicality of testing occasions and tools was only anecdotally reported. To more completely comply with the gold standards for PA design, both shared experience of experts and recognition of test construction standards, including reliability and validity measurements, instrument piloting, rater training, and explicit identification of the purpose and proposed use of the assessment tool, are required.

Association Between Malnutrition and Clinical Outcomes in the Intensive Care Unit: A Systematic Review [Formula: see text].

PubMed

Lew, Charles Chin Han; Yandell, Rosalie; Fraser, Robert J L; Chua, Ai Ping; Chong, Mary Foong Fong; Miller, Michelle

2017-07-01

Malnutrition is associated with poor clinical outcomes among hospitalized patients. However, studies linking malnutrition with poor clinical outcomes in the intensive care unit (ICU) often have conflicting findings due in part to the inappropriate diagnosis of malnutrition. We primarily aimed to determine whether malnutrition diagnosed by validated nutrition assessment tools such as the Subjective Global Assessment (SGA) or Mini Nutritional Assessment (MNA) is independently associated with poorer clinical outcomes in the ICU and if the use of nutrition screening tools demonstrate a similar association. PubMed, CINAHL, Scopus, and Cochrane Library were systematically searched for eligible studies. Search terms included were synonyms of malnutrition, nutritional status, screening, assessment, and intensive care unit. Eligible studies were case-control or cohort studies that recruited adults in the ICU; conducted the SGA, MNA, or used nutrition screening tools before or within 48 hours of ICU admission; and reported the prevalence of malnutrition and relevant clinical outcomes including mortality, length of stay (LOS), and incidence of infection (IOI). Twenty of 1168 studies were eligible. The prevalence of malnutrition ranged from 38% to 78%. Malnutrition diagnosed by nutrition assessments was independently associated with increased ICU LOS, ICU readmission, IOI, and the risk of hospital mortality. The SGA clearly had better predictive validity than the MNA. The association between malnutrition risk determined by nutrition screening was less consistent. Malnutrition is independently associated with poorer clinical outcomes in the ICU. Compared with nutrition assessment tools, the predictive validity of nutrition screening tools were less consistent.

Continued Validation of the O-SCORE (Ottawa Surgical Competency Operating Room Evaluation): Use in the Simulated Environment.

PubMed

MacEwan, Matthew J; Dudek, Nancy L; Wood, Timothy J; Gofton, Wade T

2016-01-01

CONSTRUCT: The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) is a 9-item surgical evaluation tool designed to assess technical competence in surgical trainees using behavioral anchors. The initial development of the O-SCORE produced evidence for valid results. Further work is required to determine if the use of a single surgeon or an unblinded rater introduces bias. In addition, the relationship of the O-SCORE to other currently used technical assessment tools should be explored to provide validity evidence related to the relationship to other measures. We have designed this project to provide continued validity evidence for the O-SCORE related to these two issues. Nineteen residents and 2 staff Orthopedic Surgeons from the University of Ottawa volunteered to participate in a 2-part OSCE style station. Participants completed a written questionnaire followed by a videotaped 10-minute simulated open reduction and internal fixation of a midshaft radius fracture. Videos were rated individually by 2 blinded staff orthopedic surgeons using an Objective Structured Assessment of Technical Skills (OSATS) global rating scale, an OSATS checklist, and the O-SCORE in random order. O-SCORE results appeared sensitive to surgical training level even when raters were blinded. In addition, strong agreement between two independent observers using the O-SCORE suggests that the measure captures a performance easily recognized by surgical observers. Ratings on the O-SCORE also were strongly associated with global ratings on the currently most validated technical evaluation tool (OSATS). Collectively, these results suggest that the O-SCORE generates accurate, reproducible, and meaningful results when used in a randomized and blinded fashion, providing continued validity evidence for using this tool to evaluate surgical trainee competence. The O-SCORE was able to differentiate surgical trainee level using blinded raters providing further evidence of validity for the O-SCORE. There was strong agreement between two independent observers using the O-SCORE. Ratings on the O-SCORE also demonstrated equivalence to scores on the most validated technical evaluation tool (OSATS). These results suggest that the O-SCORE demonstrates accurate and reproducible results when used in a randomized and blinded fashion providing continued validity evidence for this tool in the evaluation of surgical competence in the trainees.

Dysphagia in Multiple Sclerosis: Evaluation and Validation of the DYMUS Questionnaire.

PubMed

Alali, Dalal; Ballard, Kirrie; Vucic, Steve; Bogaardt, Hans

2018-06-01

The 10-item Dysphagia in Multiple Sclerosis (DYMUS) questionnaire is a self-administered tool used to identify swallowing problems in adults with MS. The questionnaire was not validated against other existing questionnaires to assess its convergent validity. Moreover, its test-retest reliability was not measured previously. Therefore, the purpose of this study was to assess the factor analysis, internal consistency and test-retest reliability of the DYMUS, as well as its convergent validity against an established and validated questionnaire, the EAT-10. English-speaking adults with MS in New South Wales, Australia who were seen for routine medical check-ups were invited to complete two questionnaires across two phases. One hundred participants completed phase 1, while 55 completed phase 2. Statistical analyses were performed to investigate the psychometric properties of the DYMUS questionnaire. Internal consistency (Cronbach's Alpha) reduced the DYMUS questionnaire from ten to five items. The shortened version of the DYMUS showed high internal consistency (alpha = 0.904). It also showed satisfactory reproducibility, and adequate correlation with the 10-item Eating Assessment Tool (EAT-10). Evaluation of the DYMUS resulted in a shortened version of the questionnaire with five questions related to dysphagia. This shortened version is considered an easy and useful tool in identifying patients with MS-related dysphagia.

Improving the Validity and Reliability of Large Scale Writing Assessment.

ERIC Educational Resources Information Center

Fenton, Ray; Straugh, Tom; Stofflet, Fred; Garrison, Steve

This paper examines the efforts of the Anchorage School District, Alaska, to improve the validity of its writing assessment as a useful tool for the training of teachers and the characterization of the quality of student writing. The paper examines how a number of changes in the process and scoring of the Anchorage Writing Assessment affected the…

Evacuation performance evaluation tool.

PubMed

Farra, Sharon; Miller, Elaine T; Gneuhs, Matthew; Timm, Nathan; Li, Gengxin; Simon, Ashley; Brady, Whittney

2016-01-01

Hospitals conduct evacuation exercises to improve performance during emergency events. An essential aspect in this process is the creation of reliable and valid evaluation tools. The objective of this article is to describe the development and implications of a disaster evacuation performance tool that measures one portion of the very complex process of evacuation. Through the application of the Delphi technique and DeVellis's framework, disaster and neonatal experts provided input in developing this performance evaluation tool. Following development, content validity and reliability of this tool were assessed. Large pediatric hospital and medical center in the Midwest. The tool was pilot tested with an administrative, medical, and nursing leadership group and then implemented with a group of 68 healthcare workers during a disaster exercise of a neonatal intensive care unit (NICU). The tool has demonstrated high content validity with a scale validity index of 0.979 and inter-rater reliability G coefficient (0.984, 95% CI: 0.948-0.9952). The Delphi process based on the conceptual framework of DeVellis yielded a psychometrically sound evacuation performance evaluation tool for a NICU.

What are we assessing when we measure food security? A compendium and review of current metrics.

PubMed

Jones, Andrew D; Ngure, Francis M; Pelto, Gretel; Young, Sera L

2013-09-01

The appropriate measurement of food security is critical for targeting food and economic aid; supporting early famine warning and global monitoring systems; evaluating nutrition, health, and development programs; and informing government policy across many sectors. This important work is complicated by the multiple approaches and tools for assessing food security. In response, we have prepared a compendium and review of food security assessment tools in which we review issues of terminology, measurement, and validation. We begin by describing the evolving definition of food security and use this discussion to frame a review of the current landscape of measurement tools available for assessing food security. We critically assess the purpose/s of these tools, the domains of food security assessed by each, the conceptualizations of food security that underpin each metric, as well as the approaches that have been used to validate these metrics. Specifically, we describe measurement tools that 1) provide national-level estimates of food security, 2) inform global monitoring and early warning systems, 3) assess household food access and acquisition, and 4) measure food consumption and utilization. After describing a number of outstanding measurement challenges that might be addressed in future research, we conclude by offering suggestions to guide the selection of appropriate food security metrics.

What Are We Assessing When We Measure Food Security? A Compendium and Review of Current Metrics12

PubMed Central

Jones, Andrew D.; Ngure, Francis M.; Pelto, Gretel; Young, Sera L.

2013-01-01

The appropriate measurement of food security is critical for targeting food and economic aid; supporting early famine warning and global monitoring systems; evaluating nutrition, health, and development programs; and informing government policy across many sectors. This important work is complicated by the multiple approaches and tools for assessing food security. In response, we have prepared a compendium and review of food security assessment tools in which we review issues of terminology, measurement, and validation. We begin by describing the evolving definition of food security and use this discussion to frame a review of the current landscape of measurement tools available for assessing food security. We critically assess the purpose/s of these tools, the domains of food security assessed by each, the conceptualizations of food security that underpin each metric, as well as the approaches that have been used to validate these metrics. Specifically, we describe measurement tools that 1) provide national-level estimates of food security, 2) inform global monitoring and early warning systems, 3) assess household food access and acquisition, and 4) measure food consumption and utilization. After describing a number of outstanding measurement challenges that might be addressed in future research, we conclude by offering suggestions to guide the selection of appropriate food security metrics. PMID:24038241

Development and psychometric testing of a Clinical Reasoning Evaluation Simulation Tool (CREST) for assessing nursing students' abilities to recognize and respond to clinical deterioration.

PubMed

Liaw, Sok Ying; Rashasegaran, Ahtherai; Wong, Lai Fun; Deneen, Christopher Charles; Cooper, Simon; Levett-Jones, Tracy; Goh, Hongli Sam; Ignacio, Jeanette

2018-03-01

The development of clinical reasoning skills in recognising and responding to clinical deterioration is essential in pre-registration nursing education. Simulation has been increasingly used by educators to develop this skill. To develop and evaluate the psychometric properties of a Clinical Reasoning Evaluation Simulation Tool (CREST) for measuring clinical reasoning skills in recognising and responding to clinical deterioration in a simulated environment. A scale development with psychometric testing and mixed methods study. Nursing students and academic staff were recruited at a university. A three-phase prospective study was conducted. Phase 1 involved the development and content validation of the CREST; Phase 2 included the psychometric testing of the tool with 15 second-year and 15 third-year nursing students who undertook the simulation-based assessment; Phase 3 involved the usability testing of the tool with nine academic staff through a survey questionnaire and focus group discussion. A 10-item CREST was developed based on a model of clinical reasoning. A content validity of 0.93 was obtained from the validation of 15 international experts. The construct validity was supported as the third-year students demonstrated significantly higher (p<0.001) clinical reasoning scores than the second-year students. The concurrent validity was also supported with significant positive correlations between global rating scores and almost all subscale scores, and the total scores. The predictive validity was supported with an existing tool. The internal consistency was high with a Cronbach's alpha of 0.92. A high inter-rater reliability was demonstrated with an intraclass correlation coefficient of 0.88. The usability of the tool was rated positively by the nurse educators but the need to ease the scoring process was highlighted. A valid and reliable tool was developed to measure the effectiveness of simulation in developing clinical reasoning skills for recognising and responding to clinical deterioration. Copyright © 2017. Published by Elsevier Ltd.

Consensus Recommendations on Initiating Prescription Therapies for Opioid‐Induced Constipation

PubMed Central

Argoff, Charles E.; Brennan, Michael J.; Camilleri, Michael; Davies, Andrew; Fudin, Jeffrey; Galluzzi, Katherine E.; Gudin, Jeffrey; Lembo, Anthony; Stanos, Steven P.

2015-01-01

Abstract Objective Aims of this consensus panel were to determine (1) an optimal symptom‐based method for assessing opioid‐induced constipation in clinical practice and (2) a threshold of symptom severity to prompt consideration of prescription therapy. Methods A multidisciplinary panel of 10 experts with extensive knowledge/experience with opioid‐associated adverse events convened to discuss the literature on assessment methods used for opioid‐induced constipation and reach consensus on each objective using the nominal group technique. Results Five validated assessment tools were evaluated: the Patient Assessment of Constipation–Symptoms (PAC‐SYM), Patient Assessment of Constipation–Quality of Life (PAC‐QOL), Stool Symptom Screener (SSS), Bowel Function Index (BFI), and Bowel Function Diary (BF‐Diary). The 3‐item BFI and 4‐item SSS, both clinician administered, are the shortest tools. In published trials, the BFI and 12‐item PAC‐SYM are most commonly used. The 11‐item BF‐Diary is highly relevant in opioid‐induced constipation and was developed and validated in accordance with US Food and Drug Administration guidelines. However, the panel believes that the complex scoring for this tool and the SSS, PAC‐SYM, and 28‐item PAC‐QOL may be unfeasible for clinical practice. The BFI is psychometrically validated and responsive to changes in symptom severity; scores range from 0 to 100, with higher scores indicating greater severity and scores >28.8 points indicating constipation. Conclusions The BFI is a simple assessment tool with a validated threshold of clinically significant constipation. Prescription treatments for opioid‐induced constipation should be considered for patients who have a BFI score of ≥30 points and an inadequate response to first‐line interventions. PMID:26582720

Validation of an early childhood caries risk assessment tool in a low-income Hispanic population.

PubMed

Custodio-Lumsden, Christie L; Wolf, Randi L; Contento, Isobel R; Basch, Charles E; Zybert, Patricia A; Koch, Pamela A; Edelstein, Burton L

2016-03-01

There is a recognized need for valid risk assessment tools for use by both dental and nondental personnel to identify young children at risk for, or with, precavitated stages of early childhood caries (i.e., early stage decalcifications or white spot lesions).The aim of this study is to establish concurrent criterion validity of "MySmileBuddy" (MSB), a novel technology-assisted ECC risk assessment and behavioral intervention tool against four measures of ECC activity: semi-quantitative assays of salivary mutans streptococci levels, visible quantity of dental plaque, visual evidence of enamel decalcifications, and cavitation status (none, ECC, severe ECC). One hundred eight children 2-6 years of age presenting to a pediatric dental clinic were recruited from a predominantly Spanish-speaking, low-income, urban population. All children received a comprehensive oral examination and saliva culture for assessment of ECC indicators. Their caregivers completed the iPad-based MSB assessment in its entirety (15-20 minutes). MSB calculated both diet and comprehensive ECC risk scores. Associations between all variables were determined using ordinal logistic regression. MSB diet risk scores were significantly positively associated with salivary mutans (P < 0.05), and approached significance with visible plaque levels (P < 0.1). MSB comprehensive risk scores were significantly associated with both oral mutans and visible plaque (P < 0.05). Neither was associated with visually evident decalcifications or cavitations. Findings suggest that MSB may have clinical utility as a valid risk assessment tool for identifying children with early precursors of cavitations but does not add value in identifying children with extant lesions. © 2015 American Association of Public Health Dentistry.

Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review.

PubMed

Zendejas, Benjamin; Ruparel, Raaj K; Cook, David A

2016-02-01

The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.

Psychometrics of the Home Safety Self-Assessment Tool (HSSAT) to prevent falls in community-dwelling older adults.

PubMed

Tomita, Machiko R; Saharan, Sumandeep; Rajendran, Sheela; Nochajski, Susan M; Schweitzer, Jo A

2014-01-01

OBJECTIVE. To identify psychometric properties of the Home Safety Self-Assessment Tool (HSSAT) to prevent falls in community-dwelling older adults. METHOD. We tested content validity, test-retest reliability, interrater reliability, construct validity, convergent and discriminant validity, and responsiveness to change. RESULTS. The content validity index was .98, the intraclass correlation coefficient for test-retest reliability was .97, and the interrater reliability was .89. The difference on identified risk factors between the use and nonuse of the HSSAT was significant (p = .005). Convergent validity with the Centers for Disease Control and Prevention Home Safety Checklist was high (r = .65), and discriminant validity with fear of falling was very low (r = .10). The responsiveness to change was moderate (standardized response mean = 0.57). CONCLUSION. The HSSAT is a reliable and valid instrument to identify fall risks in a home environment, and the HSSAT booklet is effective as educational material leading to improvement in home safety. Copyright © 2014 by the American Occupational Therapy Association, Inc.

Validity and inter-observer reliability of subjective hand-arm vibration assessments.

PubMed

Coenen, Pieter; Formanoy, Margriet; Douwes, Marjolein; Bosch, Tim; de Kraker, Heleen

2014-07-01

Exposure to mechanical vibrations at work (e.g., due to handling powered tools) is a potential occupational risk as it may cause upper extremity complaints. However, reliable and valid assessment methods for vibration exposure at work are lacking. Measuring hand-arm vibration objectively is often difficult and expensive, while often used information provided by manufacturers lacks detail. Therefore, a subjective hand-arm vibration assessment method was tested on validity and inter-observer reliability. In an experimental protocol, sixteen tasks handling powered tools were executed by two workers. Hand-arm vibration was assessed subjectively by 16 observers according to the proposed subjective assessment method. As a gold standard reference, hand-arm vibration was measured objectively using a vibration measurement device. Weighted κ's were calculated to assess validity, intra-class-correlation coefficients (ICCs) were calculated to assess inter-observer reliability. Inter-observer reliability of the subjective assessments depicting the agreement among observers can be expressed by an ICC of 0.708 (0.511-0.873). The validity of the subjective assessments as compared to the gold-standard reference can be expressed by a weighted κ of 0.535 (0.285-0.785). Besides, the percentage of exact agreement of the subjective assessment compared to the objective measurement was relatively low (i.e., 52% of all tasks). This study shows that subjectively assessed hand-arm vibrations are fairly reliable among observers and moderately valid. This assessment method is a first attempt to use subjective risk assessments of hand-arm vibration. Although, this assessment method can benefit from some future improvement, it can be of use in future studies and in field-based ergonomic assessments. Copyright © 2014 Elsevier Ltd and The Ergonomics Society. All rights reserved.

The National Clinical Assessment Tool for Medical Students in the Emergency Department (NCAT-EM)

PubMed Central

Jung, Julianna; Franzen, Douglas; Lawson, Luan; Manthey, David; Tews, Matthew; Dubosh, Nicole; Fisher, Jonathan; Haughey, Marianne; House, Joseph B.; Trainor, Arleigh; Wald, David A.; Hiller, Katherine

2018-01-01

Introduction Clinical assessment of medical students in emergency medicine (EM) clerkships is a highly variable process that presents unique challenges and opportunities. Currently, clerkship directors use institution-specific tools with unproven validity and reliability that may or may not address competencies valued most highly in the EM setting. Standardization of assessment practices and development of a common, valid, specialty-specific tool would benefit EM educators and students. Methods A two-day national consensus conference was held in March 2016 in the Clerkship Directors in Emergency Medicine (CDEM) track at the Council of Residency Directors in Emergency Medicine (CORD) Academic Assembly in Nashville, TN. The goal of this conference was to standardize assessment practices and to create a national clinical assessment tool for use in EM clerkships across the country. Conference leaders synthesized the literature, articulated major themes and questions pertinent to clinical assessment of students in EM, clarified the issues, and outlined the consensus-building process prior to consensus-building activities. Results The first day of the conference was dedicated to developing consensus on these key themes in clinical assessment. The second day of the conference was dedicated to discussing and voting on proposed domains to be included in the national clinical assessment tool. A modified Delphi process was initiated after the conference to reconcile questions and items that did not reach an a priori level of consensus. Conclusion The final tool, the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM) is presented here. PMID:29383058

Development and Validation of the Life Sciences Assessment: A Measure of Preschool Children's Conceptions of Basic Life Sciences

ERIC Educational Resources Information Center

Maherally, Uzma Nooreen

2014-01-01

The purpose of this study was to develop and validate a science assessment tool termed the Life Sciences Assessment (LSA) in order to assess preschool children's conceptions of basic life sciences. The hypothesis was that the four sub-constructs, each of which can be measured through a series of questions on the LSA, will make a significant…

Whakawhiti Kōrero, a Method for the Development of a Cultural Assessment Tool, Te Waka Kuaka, in Māori Traumatic Brain Injury.

PubMed

Elder, Hinemoa; Kersten, Paula

2015-01-01

The importance of tools for the measurement of outcomes and needs in traumatic brain injury is well recognised. The development of tools for these injuries in indigenous communities has been limited despite the well-documented disparity of brain injury. The wairua theory of traumatic brain injury (TBI) in Māori proposes that a culturally defined injury occurs in tandem with the physical injury. A cultural response is therefore indicated. This research investigates a Māori method used in the development of cultural needs assessment tool designed to further examine needs associated with the culturally determined injury and in preparation for formal validation. Whakawhiti kōrero is a method used to develop better statements in the development of the assessment tool. Four wānanga (traditional fora) were held including one with whānau (extended family) with experience of traumatic brain injury. The approach was well received. A final version, Te Waka Kuaka, is now ready for validation. Whakawhiti kōrero is an indigenous method used in the development of cultural needs assessment tool in Māori traumatic brain injury. This method is likely to have wider applicability, such as Mental Health and Addictions Services, to ensure robust process of outcome measure and needs assessment development.

Mother-child bonding assessment tools☆

PubMed Central

Perrelli, Jaqueline Galdino Albuquerque; Zambaldi, Carla Fonseca; Cantilino, Amaury; Sougey, Everton Botelho

2014-01-01

Objective: To identify and describe research tools used to evaluate bonding between mother and child up to one year of age, as well as to provide information on reliability and validity measures related to these tools. Data source: Research studies available on PUBMED, LILACS, ScienceDirect, PsycINFO and CINAHL databases with the following descriptors: mother-child relations and mother infant relationship, as well as the expressions validity, reliability and scale. Data synthesis: 23 research studies were selected and fully analyzed. Thirteen evaluation research tools were identified concerning mother and child attachment: seven scales, three questionnaires, two inventories and one observation method. From all tools analyzed, the Prenatal Attachment Inventory presented the higher validity and reliability measures to assess mother and fetus relation during pregnancy. Concerning the puerperal period, better consistency coefficients were found for Maternal Attachment Inventory and Postpartum Bonding Questionnaire. Besides, the last one revealed a higher sensibility to identify amenable and severe disorders in the affective relations between mother and child. Conclusions: The majority of research tools are reliable to study the phenomenon presented, although there are some limitations regarding the construct and criterion related to validity. In addition to this, only two of them are translated into Portuguese and adapted to women and children populations in Brazil, being a decisive gap to scientific production in this area. PMID:25479859

Development and validation of the Vietnamese primary care assessment tool

PubMed Central

2018-01-01

Objective To adapt the consumer version of the Primary Care Assessment Tool (PCAT) for Vietnam and determine its internal consistency and validity. Design A quantitative cross sectional study. Setting 56 communes in 3 representative provinces of central Vietnam. Participants Total of 3289 people who used health care services at health facility at least once over the past two years. Results The Vietnamese adult expanded consumer version of the PCAT (VN PCAT-AE) is an instrument for evaluation of primary care in Vietnam with 70 items comprising six scales representing four core primary care domains, and three additional scales representing three derivative domains. Sixteen other items from the original tool were not included in the final instrument, due to problems with missing values, floor or ceiling effects, and item-total correlations. All the retained scales have a Cronbach’s alpha above 0.70 except for the subscale of Family Centeredness. Conclusions The VN PCAT-AE demonstrates adequate internal consistency and validity to be used as an effective tool for measuring the quality of primary care in Vietnam from the consumer perspective. Additional work in the future to optimize valid measurement in all domains consistent with the original version of the tool may be helpful as the primary care system in Vietnam further develops. PMID:29324851

Assessing behavioural changes in ALS: cross-validation of ALS-specific measures.

PubMed

Pinto-Grau, Marta; Costello, Emmet; O'Connor, Sarah; Elamin, Marwa; Burke, Tom; Heverin, Mark; Pender, Niall; Hardiman, Orla

2017-07-01

The Beaumont Behavioural Inventory (BBI) is a behavioural proxy report for the assessment of behavioural changes in ALS. This tool has been validated against the FrSBe, a non-ALS-specific behavioural assessment, and further comparison of the BBI against a disease-specific tool was considered. This study cross-validates the BBI against the ALS-FTD-Q. Sixty ALS patients, 8% also meeting criteria for FTD, were recruited. All patients were evaluated using the BBI and the ALS-FTD-Q, completed by a carer. Correlational analysis was performed to assess construct validity. Precision, sensitivity, specificity, and overall accuracy of the BBI when compared to the ALS-FTD-Q, were obtained. The mean score of the whole sample on the BBI was 11.45 ± 13.06. ALS-FTD patients scored significantly higher than non-demented ALS patients (31.6 ± 14.64, 9.62 ± 11.38; p < 0.0001). A significant large positive correlation between the BBI and the ALS-FTD-Q was observed (r = 0.807, p < 0.0001), and no significant correlations between the BBI and other clinical/demographic characteristics indicate good convergent and discriminant validity, respectively. 72% of overall concordance was observed. Precision, sensitivity, and specificity for the classification of severely impaired patients were adequate. However, lower concordance in the classification of mild behavioural changes was observed, with higher sensitivity using the BBI, most likely secondary to BBI items which endorsed behavioural aspects not measured by the ALS-FTD-Q. Good construct validity has been further confirmed when the BBI is compared to an ALS-specific tool. Furthermore, the BBI is a more comprehensive behavioural assessment for ALS, as it measures the whole behavioural spectrum in this condition.

Development of the CarMen-Q Questionnaire for mental workload assessment.

PubMed

Rubio-Valdehita, Susana; López-Núñez, María I; López-Higes, Ramón; Díaz-Ramiro, Eva M

2017-11-01

Mental workload has emerged as one of the most important occupational risk factors present in most psychological and physical diseases caused by work. In view of the lack of specific tools to assess mental workload, the objective of this research was to assess the construct validity and reliability of a new questionnaire for mental workload assessment (CarMen-Q). The sample was composed of 884 workers from several professional sectors, between 18 and 65 years old, 53.4% men and 46.6% women. To evaluate the validity based on relationships with other measures, the NASA-TLX scale was also administered. Confirmatory factor analysis showed an internal structure made up of four dimensions: cognitive, temporal and emotional demands and performance requirement. The results show satisfactory evidence of validity based on relationships with NASA-TLX and good reliability. The questionnaire has good psychometric properties and can be an easy, brief, useful tool for mental workload diagnosis and prevention.

Development and Validity Testing of the Worksite Health Index: An Assessment Tool to Help and Improve Korean Employees' Health-Related Outcome.

PubMed

Yun, Young Ho; Sim, Jin Ah; Lim, Ye Jin; Lim, Cheol Il; Kang, Sung-Choon; Kang, Joon-Ho; Park, Jun Dong; Noh, Dong Young

2016-06-01

The objective of this study was to develop the Worksite Health Index (WHI) and validate its psychometric properties. The development of the WHI questionnaire included item generation, item construction, and field testing. To assess the instrument's reliability and validity, we recruited 30 different Korean worksites. We developed the WHI questionnaire of 136 items categorized into five domains, namely Governance and Infrastructure, Need Assessment and Planning, Health Prevention and Promotion Program, Occupational Safety, and Monitoring and Feedback. All WHI domains demonstrated a high reliability with good internal consistency. The total WHI scores differentiated worksite groups effectively according to firm size. Each domain was associated significantly with employees' health status, absence, and financial outcome. The WHI can assess comprehensive worksite health programs. This tool is publicly available for addressing the growing need for worksite health programs.

Nutrition screening tools: an analysis of the evidence.

PubMed

Skipper, Annalynn; Ferguson, Maree; Thompson, Kyle; Castellanos, Victoria H; Porcari, Judy

2012-05-01

In response to questions about tools for nutrition screening, an evidence analysis project was developed to identify the most valid and reliable nutrition screening tools for use in acute care and hospital-based ambulatory care settings. An oversight group defined nutrition screening and literature search criteria. A trained analyst conducted structured searches of the literature for studies of nutrition screening tools according to predetermined criteria. Eleven nutrition screening tools designed to detect undernutrition in patients in acute care and hospital-based ambulatory care were identified. Trained analysts evaluated articles for quality using criteria specified by the American Dietetic Association's Evidence Analysis Library. Members of the oversight group assigned quality grades to the tools based on the quality of the supporting evidence, including reliability and validity data. One tool, the NRS-2002, received a grade I, and 4 tools-the Simple Two-Part Tool, the Mini-Nutritional Assessment-Short Form (MNA-SF), the Malnutrition Screening Tool (MST), and Malnutrition Universal Screening Tool (MUST)-received a grade II. The MST was the only tool shown to be both valid and reliable for identifying undernutrition in the settings studied. Thus, validated nutrition screening tools that are simple and easy to use are available for application in acute care and hospital-based ambulatory care settings.

External Validation of a Tool Predicting 7-Year Risk of Developing Cardiovascular Disease, Type 2 Diabetes or Chronic Kidney Disease.

PubMed

Rauh, Simone P; Rutters, Femke; van der Heijden, Amber A W A; Luimes, Thomas; Alssema, Marjan; Heymans, Martijn W; Magliano, Dianna J; Shaw, Jonathan E; Beulens, Joline W; Dekker, Jacqueline M

2018-02-01

Chronic cardiometabolic diseases, including cardiovascular disease (CVD), type 2 diabetes (T2D) and chronic kidney disease (CKD), share many modifiable risk factors and can be prevented using combined prevention programs. Valid risk prediction tools are needed to accurately identify individuals at risk. We aimed to validate a previously developed non-invasive risk prediction tool for predicting the combined 7-year-risk for chronic cardiometabolic diseases. The previously developed tool is stratified for sex and contains the predictors age, BMI, waist circumference, use of antihypertensives, smoking, family history of myocardial infarction/stroke, and family history of diabetes. This tool was externally validated, evaluating model performance using area under the receiver operating characteristic curve (AUC)-assessing discrimination-and Hosmer-Lemeshow goodness-of-fit (HL) statistics-assessing calibration. The intercept was recalibrated to improve calibration performance. The risk prediction tool was validated in 3544 participants from the Australian Diabetes, Obesity and Lifestyle Study (AusDiab). Discrimination was acceptable, with an AUC of 0.78 (95% CI 0.75-0.81) in men and 0.78 (95% CI 0.74-0.81) in women. Calibration was poor (HL statistic: p < 0.001), but improved considerably after intercept recalibration. Examination of individual outcomes showed that in men, AUC was highest for CKD (0.85 [95% CI 0.78-0.91]) and lowest for T2D (0.69 [95% CI 0.65-0.74]). In women, AUC was highest for CVD (0.88 [95% CI 0.83-0.94)]) and lowest for T2D (0.71 [95% CI 0.66-0.75]). Validation of our previously developed tool showed robust discriminative performance across populations. Model recalibration is recommended to account for different disease rates. Our risk prediction tool can be useful in large-scale prevention programs for identifying those in need of further risk profiling because of their increased risk for chronic cardiometabolic diseases.

Validation of the Practice Environment Scale to the Brazilian culture.

PubMed

Gasparino, Renata C; Guirardello, Edinêis de B

2017-07-01

To validate the Brazilian version of the Practice Environment Scale. The Practice Environment Scale is a tool that evaluates the presence of characteristics that are favourable for professional nursing practice because a better work environment contributes to positive results for patients, professionals and institutions. Methodological study including 209 nurses. Validity was assessed via a confirmatory factor analysis using structural equation modelling, in which the correlations between the instrument and the following variables were tested: burnout, job satisfaction, safety climate, perception of quality of care and intention to leave the job. Subgroups were compared and the reliability was assessed using Cronbach's alpha and the composite reliability. Factor analysis resulted in exclusion of seven items. Significant correlations were obtained between the subscales and all variables in the study. The reliability was considered acceptable. The Brazilian version of the Practice Environment Scale is a valid and reliable tool used to assess the characteristics that promote professional nursing practice. Use of this tool in Brazilian culture should allow managers to implement changes that contribute to the achievement of better results, in addition to identifying and comparing the environments of health institutions. © 2017 John Wiley & Sons Ltd.

Development and Validation of a Social Capital Questionnaire for Adolescent Students (SCQ-AS)

PubMed Central

Paiva, Paula Cristina Pelli; de Paiva, Haroldo Neves; de Oliveira Filho, Paulo Messias; Lamounier, Joel Alves; Ferreira, Efigênia Ferreira e; Ferreira, Raquel Conceição; Kawachi, Ichiro; Zarzar, Patrícia Maria

2014-01-01

Objectives Social capital has been studied due to its contextual influence on health. However, no specific assessment tool has been developed and validated for the measurement of social capital among 12-year-old adolescent students. The aim of the present study was to develop and validate a quick, simple assessment tool to measure social capital among adolescent students. Methods A questionnaire was developed based on a review of relevant literature. For such, searches were made of the Scientific Electronic Library Online, Latin American and Caribbean Health Sciences, The Cochrane Library, ISI Web of Knowledge, International Database for Medical Literature and PubMed Central bibliographical databases from September 2011 to January 2014 for papers addressing assessment tools for the evaluation of social capital. Focus groups were also formed by adolescent students as well as health, educational and social professionals. The final assessment tool was administered to a convenience sample from two public schools (79 students) and one private school (22 students), comprising a final sample of 101 students. Reliability and internal consistency were evaluated using the Kappa coefficient and Cronbach's alpha coefficient, respectively. Content validity was determined by expert consensus as well as exploratory and confirmatory factor analysis. Results The final version of the questionnaire was made up of 12 items. The total scale demonstrated very good internal consistency (Cronbach's alpha: 0.71). Reproducibility was also very good, as the Kappa coefficient was higher than 0.72 for the majority of items (range: 0.63 to 0.97). Factor analysis grouped the 12 items into four subscales: School Social Cohesion, School Friendships, Neighborhood Social Cohesion and Trust (school and neighborhood). Conclusions The present findings indicate the validity and reliability of the Social Capital Questionnaire for Adolescent Students. PMID:25093409

Consequential Validity of an Assistive Technology Supplement for the School Function Assessment

ERIC Educational Resources Information Center

Silverman, Michelle Kaye; Smith, Roger O.

2006-01-01

Educators and therapists implement assistive technology to maximize educational outcomes of students with disabilities. However, few measure the outcomes of interventions because of a lack of valid measurement tools. This study investigated whether an assistive technology supplement for the School Function Assessment demonstrates an important…

Validity aspects of the patient feedback questionnaire on consultation skills (PFC), a promising learning instrument in medical education.

PubMed

Reinders, Marcel E; Blankenstein, Annette H; Knol, Dirk L; de Vet, Henrica C W; van Marwijk, Harm W J

2009-08-01

A focus on the communicator competency is considered to be an important requirement to help physicians to acquire consultation skills. A feedback questionnaire, in which patients assess consultation skills might be a useful learning tool. An existing questionnaire on patient perception of patient-centeredness (PPPC) was adapted to cover the 'communicator' items in the competency profile. We assessed the face and content validity, the construct validity and the internal consistency of this new patient feedback on consultation skills (PFC) questionnaire. We assessed the face validity of the PFC by interviewing patients and general practice trainees (GPTs) during the developmental process. The content validity was determined by experts (n=10). First-year GPTs (23) collected 222 PFCs, from which the data were used to assess the construct validity (factor analysis), internal consistency, response rates and ceiling effects. The PFC adequately covers the corresponding 'communicator' competency (face and content validity). Factor analysis showed a one-dimensional construct. The internal consistency was high (Cronbach's alpha 0.89). For the single items, the response rate varied from 89.2% to 100%; the maximum score (ceiling effect) varied from 45.5% to 89.2%. The PFC appears to be a valid, internally consistent instrument. The PFC may be a valuable learning tool with which GPTs, other physicians and medical students can acquire feedback from patients regarding their consultation skills.

Validation of the Middlesex Elderly Assessment of Mental State (MEAMS) as a cognitive screening test in patients with acquired brain injury in Turkey.

PubMed

Kutlay, Sehim; Kuçukdeveci, Ayse A; Elhan, Atilla H; Yavuzer, Gunes; Tennant, Alan

2007-02-28

Assessment of cognitive impairment with a valid cognitive screening tool is essential in neurorehabilitation. The aim of this study was to test the reliability and validity of the Turkish-adapted version of the Middlesex Elderly Assessment of Mental State (MEAMS) among acquired brain injury patients in Turkey. Some 155 patients with acquired brain injury admitted for rehabilitation were assessed by the adapted version of MEAMS at admission and discharge. Reliability was tested by internal consistency, intra-class correlation coefficient (ICC) and person separation index; internal construct validity by Rasch analysis; external construct validity by associations with physical and cognitive disability (FIM); and responsiveness by Effect Size. Reliability was found to be good with Cronbach's alpha of 0.82 at both admission and discharge; and likewise an ICC of 0.80. Person separation index was 0.813. Internal construct validity was good by fit of the data to the Rasch model (mean item fit -0.178; SD 1.019). Items were substantially free of differential item functioning. External construct validity was confirmed by expected associations with physical and cognitive disability. Effect size was 0.42 compared with 0.22 for cognitive FIM. The reliability and validity of the Turkish version of MEAMS as a cognitive impairment screening tool in acquired brain injury has been demonstrated.

Psychological testing and psychological assessment. A review of evidence and issues.

PubMed

Meyer, G J; Finn, S E; Eyde, L D; Kay, G G; Moreland, K L; Dies, R R; Eisman, E J; Kubiszyn, T W; Reed, G M

2001-02-01

This article summarizes evidence and issues associated with psychological assessment. Data from more than 125 meta-analyses on test validity and 800 samples examining multimethod assessment suggest 4 general conclusions: (a) Psychological test validity is strong and compelling, (b) psychological test validity is comparable to medical test validity, (c) distinct assessment methods provide unique sources of information, and (d) clinicians who rely exclusively on interviews are prone to incomplete understandings. Following principles for optimal nomothetic research, the authors suggest that a multimethod assessment battery provides a structured means for skilled clinicians to maximize the validity of individualized assessments. Future investigations should move beyond an examination of test scales to focus more on the role of psychologists who use tests as helpful tools to furnish patients and referral sources with professional consultation.

Assessing patient capacity to consent to treatment: an integrative review of instruments and tools.

PubMed

Lamont, Scott; Jeon, Yun-Hee; Chiarella, Mary

2013-09-01

To provide a narrative synthesis of research findings on instruments or tools designed to aid assessment of patient capacity to consent to treatment. Capacity assessment is of significant priority within health care as a finding of incapacity is a vehicle for the removal of many of an individual's fundamental rights. Despite there being many instruments and tools available to aid health professionals in the assessment of patient capacity, there are no standardised guidelines from professional bodies that inform the assessment of mental capacity. Integrative review. Primary studies of instruments or tools concerning assessment of patient capacity to consent to treatment, published in English in peer-reviewed journals between January 2005-December 2010, were included in the review. Review papers of capacity assessment instruments were included for years including and prior to 2006. Nineteen instruments were found which assess patient capacity to consent. Key themes were identified in terms of capacity domains assessed, psychometric properties, instrument implementation, patient populations studied and instrument versus clinician judgement. Despite a plethora of capacity assessment instruments and tools available, only a small number of instruments were found to have demonstrated both reliability and validity. Further research is required to improve the validity of existing capacity assessment instruments. Increased attention to patient rights and autonomy arguably places a considerable burden on healthcare professionals to facilitate capacity assessments across a continuum of health care. Despite a plethora of capacity assessment instruments and tools being available to healthcare professionals, a comprehensive assessment requires time and is often difficult in the acute care setting. A strictly formulaic approach to the assessment of capacity is unlikely to capture specific individual nuances; therefore, capacity assessment instruments should support, but not replace, experienced clinical judgement. © 2013 John Wiley & Sons Ltd.

Training and Assessment of Hysteroscopic Skills: A Systematic Review.

PubMed

Savran, Mona Meral; Sørensen, Stine Maya Dreier; Konge, Lars; Tolsgaard, Martin G; Bjerrum, Flemming

2016-01-01

The aim of this systematic review was to identify studies on hysteroscopic training and assessment. PubMed, Excerpta Medica, the Cochrane Library, and Web of Science were searched in January 2015. Manual screening of references and citation tracking were also performed. Studies on hysteroscopic educational interventions were selected without restrictions on study design, populations, language, or publication year. A qualitative data synthesis including the setting, study participants, training model, training characteristics, hysteroscopic skills, assessment parameters, and study outcomes was performed by 2 authors working independently. Effect sizes were calculated when possible. Overall, 2 raters independently evaluated sources of validity evidence supporting the outcomes of the hysteroscopy assessment tools. A total of 25 studies on hysteroscopy training were identified, of which 23 were performed in simulated settings. Overall, 10 studies used virtual-reality simulators and reported effect sizes for technical skills ranging from 0.31 to 2.65; 12 used inanimate models and reported effect sizes for technical skills ranging from 0.35 to 3.19. One study involved live animal models; 2 studies were performed in clinical settings. The validity evidence supporting the assessment tools used was low. Consensus between the 2 raters on the reported validity evidence was high (94%). This systematic review demonstrated large variations in the effect of different tools for hysteroscopy training. The validity evidence supporting the assessment of hysteroscopic skills was limited. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

The Myotonometer: Not a Valid Measurement Tool for Active Hamstring Musculotendinous Stiffness.

PubMed

Pamukoff, Derek N; Bell, Sarah E; Ryan, Eric D; Blackburn, J Troy

2016-05-01

Hamstring musculotendinous stiffness (MTS) is associated with lower-extremity injury risk (ie, hamstring strain, anterior cruciate ligament injury) and is commonly assessed using the damped oscillatory technique. However, despite a preponderance of studies that measure MTS reliably in laboratory settings, there are no valid clinical measurement tools. A valid clinical measurement technique is needed to assess MTS and permit identification of individuals at heightened risk of injury and track rehabilitation progress. To determine the validity and reliability of the Myotonometer for measuring active hamstring MTS. Descriptive laboratory study. Laboratory. 33 healthy participants (15 men, age 21.33 ± 2.94 y, height 172.03 ± 16.36 cm, mass 74.21 ± 16.36 kg). Hamstring MTS was assessed using the damped oscillatory technique and the Myotonometer. Intraclass correlations were used to determine the intrasession, intersession, and interrater reliability of the Myotonometer. Criterion validity was assessed via Pearson product-moment correlation between MTS measures obtained from the Myotonometer and from the damped oscillatory technique. The Myotonometer demonstrated good intrasession (ICC3,1 = .807) and interrater reliability (ICC2,k = .830) and moderate intersession reliability (ICC2,k = .693). However, it did not provide a valid measurement of MTS compared with the damped oscillatory technique (r = .346, P = .061). The Myotonometer does not provide a valid measure of active hamstring MTS. Although the Myotonometer does not measure active MTS, it possesses good reliability and portability and could be used clinically to measure tissue compliance, muscle tone, or spasticity associated with multiple musculoskeletal disorders. Future research should focus on portable and clinically applicable tools to measure active hamstring MTS in efforts to prevent and monitor injuries.

Development and feasibility of the misuse, abuse, and diversion drug event reporting system (MADDERS®).

PubMed

Treister, Roi; Trudeau, Jeremiah J; Van Inwegen, Richard; Jones, Judith K; Katz, Nathaniel P

2016-12-01

Inappropriate use of analgesic drugs has become increasingly pervasive over the past decade. Currently, drug abuse potential is primarily assessed post-marketing; no validated tools are available to assess this potential in phase II and III clinical trials. This paper describes the development and feasibility testing of a Misuse, Abuse, and Diversion Drug Event Reporting System (MADDERS), which aims to identify potentially abuse-related events and classify them according to a recently developed classification scheme, allowing the quantification of these events in clinical trials. The system was initially conceived and designed with input from experts and patients, followed by field-testing to assess its feasibility and content validity in both completed and ongoing clinical trials. The results suggest that MADDERS is a feasible system with initial validity. It showed higher rates of the triggering events in subjects taking medications with known abuse potential than in patients taking medications without abuse potential. Additionally, experts agreed on the classification of most abuse-related events in MADDERS. MADDERS is a new systematic approach to collect information on potentially abuse-related events in clinical trials and classify them. The system has demonstrated feasibility for implementation. Additional research is ongoing to further evaluate its validity. Currently, there are no validated tools to assess drug abuse potential during clinical trials. Because of its ease of implementation, its systematic approach, and its preliminary validation results, MADDERS could provide such a tool for clinical trials. (Am J Addict 2016;25:641-651). © 2016 American Academy of Addiction Psychiatry.

Tools for assessing fall risk in the elderly: a systematic review and meta-analysis.

PubMed

Park, Seong-Hi

2018-01-01

The prevention of falls among the elderly is arguably one of the most important public health issues in today's aging society. The aim of this study was to assess which tools best predict the risk of falls in the elderly. Electronic searches were performed using Medline, EMBASE, the Cochrane Library, CINAHL, etc., using the following keywords: "fall risk assessment", "elderly fall screening", and "elderly mobility scale". The QUADAS-2 was applied to assess the internal validity of the diagnostic studies. Selected studies were meta-analyzed with MetaDisc 1.4. A total of 33 studies were eligible out of the 2,321 studies retrieved from selected databases. Twenty-six assessment tools for fall risk were used in the selected articles, and they tended to vary based on the setting. The fall risk assessment tools currently used for the elderly did not show sufficiently high predictive validity for differentiating high and low fall risks. The Berg Balance scale and Mobility Interaction Fall chart showed stable and high specificity, while the Downton Fall Risk Index, Hendrich II Fall Risk Model, St. Thomas's Risk Assessment Tool in Falling elderly inpatients, Timed Up and Go test, and Tinetti Balance scale showed the opposite results. We concluded that rather than a single measure, two assessment tools used together would better evaluate the characteristics of falls by the elderly that can occur due to a multitude of factors and maximize the advantages of each for predicting the occurrence of falls.

Tools for Embedded Computing Systems Software

NASA Technical Reports Server (NTRS)

1978-01-01

A workshop was held to assess the state of tools for embedded systems software and to determine directions for tool development. A synopsis of the talk and the key figures of each workshop presentation, together with chairmen summaries, are presented. The presentations covered four major areas: (1) tools and the software environment (development and testing); (2) tools and software requirements, design, and specification; (3) tools and language processors; and (4) tools and verification and validation (analysis and testing). The utility and contribution of existing tools and research results for the development and testing of embedded computing systems software are described and assessed.

Measurement properties of tools measuring mental health knowledge: a systematic review.

PubMed

Wei, Yifeng; McGrath, Patrick J; Hayden, Jill; Kutcher, Stan

2016-08-23

Mental health literacy has received great attention recently to improve mental health knowledge, decrease stigma and enhance help-seeking behaviors. We conducted a systematic review to critically appraise the qualities of studies evaluating the measurement properties of mental health knowledge tools and the quality of included measurement properties. We searched PubMed, PsycINFO, EMBASE, CINAHL, the Cochrane Library, and ERIC for studies addressing psychometrics of mental health knowledge tools and published in English. We applied the COSMIN checklist to assess the methodological quality of each study as "excellent", "good", "fair", or "indeterminate". We ranked the level of evidence of the overall quality of each measurement property across studies as "strong", "moderate", "limited", "conflicting", or "unknown". We identified 16 mental health knowledge tools in 17 studies, addressing reliability, validity, responsiveness or measurement errors. The methodological quality of included studies ranged from "poor" to "excellent" including 6 studies addressing the content validity, internal consistency or structural validity demonstrating "excellent" quality. We found strong evidence of the content validity or internal consistency of 6 tools; moderate evidence of the internal consistency, the content validity or the reliability of 8 tools; and limited evidence of the reliability, the structural validity, the criterion validity, or the construct validity of 12 tools. Both the methodological qualities of included studies and the overall evidence of measurement properties are mixed. Based on the current evidence, we recommend that researchers consider using tools with measurement properties of strong or moderate evidence that also reached the threshold for positive ratings according to COSMIN checklist.

Development and Application of the CAT-RPM Report for Strengths-Based Case Management of At-Risk Youth in Schools

ERIC Educational Resources Information Center

Bower, J. M.; Carroll, A.; Ashman, A.

2015-01-01

The Contextualised Assessment Tool for Risk and Protection Management (CAT-RPM) has been established as a valid and reliable tool for differentiating groups across age, sex and behaviour and assisting young people to find their strengths [Bower, J., A. Carroll, and A. Ashman. 2014. "The Development and Validation of the Contextualised…

Evaluating a Bedside Tool for Neuroanatomical Localization with Extended-Matching Questions

ERIC Educational Resources Information Center

Tan, Kevin; Chin, Han Xin; Yau, Christine W. L.; Lim, Erle C. H.; Samarasekera, Dujeepa; Ponnamperuma, Gominda; Tan, Nigel C. K.

2018-01-01

Neuroanatomical localization (NL) is a key skill in neurology, but learners often have difficulty with it. This study aims to evaluate a concise NL tool (NLT) developed to help teach and learn NL. To evaluate the NLT, an extended-matching questions (EMQ) test to assess NL was designed and validated. The EMQ was validated with fourth-year medical…

Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions.

PubMed

Barton, K L; Wrieden, W L; Anderson, A S

2011-12-01

Food skills programmes are widely used as a means to improve confidence in food preparation, the use of basic food skills and food selections amongst low income communities. However, the impact of such interventions are rarely evaluated as a result of a lack of validated assessment tools appropriate for use within this target group. A two-page questionnaire utilising a closed-question format was designed based on key domains known to be influenced by cooking skills programmes. Content validity was assessed by a panel of public health experts and face validity by individuals, typical of those who may attend cooking skills classes. Internal and repeat reliability were assessed with groups of adults attending community-based classes. The feasibility of using the tool in community settings was also assessed. The draft questionnaire was amended as appropriate subsequent to content and face validity testing. Cronbach's alpha for confidence and knowledge sections was 0.86 and 0.84, respectively, indicating good internal consistency. Spearman correlation coefficients for repeat reliability testing between time 1 and time 2 for each item were in the range 0.46-0.91 (all significant at P < 0.001), indicating that the questionnaire elicited stable responses for repeated use. Feasibility testing highlighted the need for detailed instructions for course tutors on how to distribute and check questionnaires for completion. This tool provides a standardised method of evaluating cooking skills interventions that could be utilised in the development and evaluation of multicentre cooking skills interventions. © 2011 The Authors. Journal of Human Nutrition and Dietetics © 2011 The British Dietetic Association Ltd.

The Predictive Validity of the Assessment of Basic Learning Abilities versus Parents' Predictions with Children with Autism

ERIC Educational Resources Information Center

Murphy, Colleen; Martin, Garry L.; Yu, C. T.

2014-01-01

The Assessment of Basic Learning Abilities (ABLA) is an empirically validated clinical tool for assessing the learning ability of persons with intellectual disabilities and children with autism. An ABLA tester uses standardized prompting and reinforcement procedures to attempt to teach, individually, each of six tasks, called levels, to a testee,…

Development and Validation of a Parent Report Measure for Assessing Social-Emotional Competencies of Children and Adolescents

ERIC Educational Resources Information Center

Merrell, Kenneth W.; Felver-Gant, Josh C.; Tom, Karalyn M.

2011-01-01

Based on the premises that strength-based assessment of children and adolescents is an important emerging area, and that additional tools for this purpose are needed, this study details development and validation efforts on a new strength-based assessment: the Social-Emotional Assets and Resilience Scale, parent form (SEARS-P). Following careful…

Tensions Between Validity and Outcomes: Teacher Assessment of Written Work of Recently Arrived Immigrant ESL Students

ERIC Educational Resources Information Center

Arkoudis, Sophie; O'Loughlin, Kieran

2004-01-01

This article reports on a collaborative study involving ESL teachers in an Australian English Language Centre as they work through some of their concerns about reliability and validity in their assessment practices. The focus of this article is on how teachers work with the Curriculum Standards Framework (CSF) as an assessment tool. The discussion…

[Spanish validation of the MacArthur Competence Assessment Tool for Treatment interview to assess patients competence to consent treatment].

PubMed

Alvarez Marrodán, Ignacio; Baón Pérez, Beatriz; Navío Acosta, Mercedes; López-Antón, Raul; Lobo Escolar, Elena; Ventura Faci, Tirso

2014-09-09

To validate the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) Spanish version, which assesses the mental capacity of patients to consent treatment, by examining 4 areas (Understanding, Appreciation, Reasoning and Expressing a choice). 160 subjects (80 Internal Medicine inpatients, 40 Psychiatric inpatients and 40 healthy controls). MacCAT-T, Mini-Mental Status Examination (MMSE). Feasibility study, reliability and validity calculations (against to gold standard of clinical expert). Mean duration of the MacCAT-T interview was 18min. Inter-rater reliability: Intraclass correlation coefficient for Understanding=0.98, Appreciation=0.97, Reasoning=0.98, Expressing a choice=0.91. Internal consistency (Cronbach's alpha): Understanding=0.87, for Appreciation=0.76, for Reasoning=0.86. Patients considered to be incapable (gold standard) scored lower in all the MacCAT-T areas. Poor performance on the MacCAT-T was related to cognitive impairment assessed by MMSE. Spanish version of the MacCAT-T is feasible, reliable, and valid for assessing the capacity of patients to consent treatment. Copyright © 2013 Elsevier España, S.L. All rights reserved.

Validation of the first peoples cultural capability measurement tool with undergraduate health students: A descriptive cohort study.

PubMed

West, Roianne; Mills, Kyly; Rowland, Dale; Creedy, Debra K

2018-05-01

Health professional graduates require the capacity to work safely, both clinically and culturally, when delivering care to Indigenous peoples worldwide. In the Australian context, the Aboriginal and Torres Strait Islander Health Curriculum Framework (The Framework) provides guidance for health professional programs to integrate, teach and assess Aboriginal and Torres Strait Islander peoples' (First Peoples) health content. There is, however, a lack of validated tools that measure the development of students' cultural capabilities. To validate the Cultural Capability Measurement Tool with a cohort of health professional students. A descriptive cohort design was used. All students (N = 753) enrolled in a discrete First Peoples Health course at an Australian university were invited to complete the Cultural Capability Measurement Tool. The tool was tested for reliability, content and construct validity using confirmatory factor analysis; and concurrent validity using and the Cultural Understanding Self-Assessment Tool. A sample of 418 (73% response rate) was recruited. Most participants were enrolled in the Bachelor of Nursing program (n = 369, 82%). The Cultural Capability Measurement Tool had a Cronbach's alpha coefficient of 0.86. A five-factor solution was confirmed which reflected the cultural capability domains and accounted for 51% of the variance. Scores correlated with students' cultural understanding (r = 0.28, p < 0.001). Successful implementation of The Framework requires instruments to measure changes in students' cultural capabilities. Measuring nursing students' cultural capabilities can inform their development, identify areas of strengths and deficits for educators, and will ultimately contribute to the development of a culturally safe nursing workforce. Copyright © 2018 Elsevier Ltd. All rights reserved.

Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings.

PubMed

Bergström, Anna; Skeen, Sarah; Duc, Duong M; Blandon, Elmer Zelaya; Estabrooks, Carole; Gustavsson, Petter; Hoa, Dinh Thi Phuong; Källestål, Carina; Målqvist, Mats; Nga, Nguyen Thu; Persson, Lars-Åke; Pervin, Jesmin; Peterson, Stefan; Rahman, Anisur; Selling, Katarina; Squires, Janet E; Tomlinson, Mark; Waiswa, Peter; Wallin, Lars

2015-08-15

The gap between what is known and what is practiced results in health service users not benefitting from advances in healthcare, and in unnecessary costs. A supportive context is considered a key element for successful implementation of evidence-based practices (EBP). There were no tools available for the systematic mapping of aspects of organizational context influencing the implementation of EBPs in low- and middle-income countries (LMICs). Thus, this project aimed to develop and psychometrically validate a tool for this purpose. The development of the Context Assessment for Community Health (COACH) tool was premised on the context dimension in the Promoting Action on Research Implementation in Health Services framework, and is a derivative product of the Alberta Context Tool. Its development was undertaken in Bangladesh, Vietnam, Uganda, South Africa and Nicaragua in six phases: (1) defining dimensions and draft tool development, (2) content validity amongst in-country expert panels, (3) content validity amongst international experts, (4) response process validity, (5) translation and (6) evaluation of psychometric properties amongst 690 health workers in the five countries. The tool was validated for use amongst physicians, nurse/midwives and community health workers. The six phases of development resulted in a good fit between the theoretical dimensions of the COACH tool and its psychometric properties. The tool has 49 items measuring eight aspects of context: Resources, Community engagement, Commitment to work, Informal payment, Leadership, Work culture, Monitoring services for action and Sources of knowledge. Aspects of organizational context that were identified as influencing the implementation of EBPs in high-income settings were also found to be relevant in LMICs. However, there were additional aspects of context of relevance in LMICs specifically Resources, Community engagement, Commitment to work and Informal payment. Use of the COACH tool will allow for systematic description of the local healthcare context prior implementing healthcare interventions to allow for tailoring implementation strategies or as part of the evaluation of implementing healthcare interventions and thus allow for deeper insights into the process of implementing EBPs in LMICs.

Ready, Set, Change! Development and usability testing of an online readiness for change decision support tool for healthcare organizations.

PubMed

Timmings, Caitlyn; Khan, Sobia; Moore, Julia E; Marquez, Christine; Pyka, Kasha; Straus, Sharon E

2016-02-24

To address challenges related to selecting a valid, reliable, and appropriate readiness assessment measure in practice, we developed an online decision support tool to aid frontline implementers in healthcare settings in this process. The focus of this paper is to describe a multi-step, end-user driven approach to developing this tool for use during the planning stages of implementation. A multi-phase, end-user driven approach was used to develop and test the usability of a readiness decision support tool. First, readiness assessment measures that are valid, reliable, and appropriate for healthcare settings were identified from a systematic review. Second, a mapping exercise was performed to categorize individual items of included measures according to key readiness constructs from an existing framework. Third, a modified Delphi process was used to collect stakeholder ratings of the included measures on domains of feasibility, relevance, and likelihood to recommend. Fourth, two versions of a decision support tool prototype were developed and evaluated for usability. Nine valid and reliable readiness assessment measures were included in the decision support tool. The mapping exercise revealed that of the nine measures, most measures (78 %) focused on assessing readiness for change at the organizational versus the individual level, and that four measures (44 %) represented all constructs of organizational readiness. During the modified Delphi process, stakeholders rated most measures as feasible and relevant for use in practice, and reported that they would be likely to recommend use of most measures. Using data from the mapping exercise and stakeholder panel, an algorithm was developed to link users to a measure based on characteristics of their organizational setting and their readiness for change assessment priorities. Usability testing yielded recommendations that were used to refine the Ready, Set, Change! decision support tool . Ready, Set, Change! decision support tool is an implementation support that is designed to facilitate the routine incorporation of a readiness assessment as an early step in implementation. Use of this tool in practice may offer time and resource-saving implications for implementation.

Elbow-specific clinical rating systems: extent of established validity, reliability, and responsiveness.

PubMed

The, Bertram; Reininga, Inge H F; El Moumni, Mostafa; Eygendaal, Denise

2013-10-01

The modern standard of evaluating treatment results includes the use of rating systems. Elbow-specific rating systems are frequently used in studies aiming at elbow-specific pathology. However, proper validation studies seem to be relatively sparse. In addition, these scoring systems might not always be used for appropriate populations of interest. Both of these issues might give rise to invalid conclusions being reported in the literature. Our aim was to investigate the extent to which the available elbow-specific outcome measurement tools have been validated and the quality of the validation itself. We also aimed to provide characteristics of the populations used for validation of these scales to enable clinicians to use them appropriately. A literature search identified 17 studies of 12 different elbow-specific scoring systems. These were assessed for validity, reliability, and responsiveness characteristics. The quality of these assessments was rated according to the Consensus Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist criteria, a standardized and validated tool developed specifically for this purpose. Currently, the only elbow-specific rating system that is validated using high-quality methodology is the Oxford Elbow Score, a patient-administered outcome measure tool that has been validated on heterogeneous study populations. Other rating systems still have to be proven in the future to be as good as the Oxford Elbow Score for clinical or research purposes. Additional validation studies are needed. Copyright © 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. Published by Mosby, Inc. All rights reserved.

Statistical methodology: II. Reliability and validity assessment in study design, Part B.

PubMed

Karras, D J

1997-02-01

Validity measures the correspondence between a test and other purported measures of the same or similar qualities. When a reference standard exists, a criterion-based validity coefficient can be calculated. If no such standard is available, the concepts of content and construct validity may be used, but quantitative analysis may not be possible. The Pearson and Spearman tests of correlation are often used to assess the correspondence between tests, but do not account for measurement biases and may yield misleading results. Techniques that measure interest differences may be more meaningful in validity assessment, and the kappa statistic is useful for analyzing categorical variables. Questionnaires often can be designed to allow quantitative assessment of reliability and validity, although this may be difficult. Inclusion of homogeneous questions is necessary to assess reliability. Analysis is enhanced by using Likert scales or similar techniques that yield ordinal data. Validity assessment of questionnaires requires careful definition of the scope of the test and comparison with previously validated tools.

The FORGE AHEAD clinical readiness consultation tool: a validated tool to assess clinical readiness for chronic disease care mobilization in Canada's First Nations.

PubMed

Hayward, Mariam Naqshbandi; Mequanint, Selam; Paquette-Warren, Jann; Bailie, Ross; Chirila, Alexandra; Dyck, Roland; Green, Michael; Hanley, Anthony; Tompkins, Jordan; Harris, Stewart

2017-03-23

Given the astounding rates of diabetes and related complications, and the barriers to providing care present in Indigenous communities in Canada, intervention strategies that take into account contextual factors such as readiness to mobilize are needed to maximize improvements and increase the likelihood of success and sustainment. As part of the national FORGE AHEAD Program, we sought to develop, test and validate a clinical readiness consultation tool aimed at assessing the readiness of clinical teams working on-reserve in First Nations communities to participate in quality improvement (QI) to enhance diabetes care in Canada. A literature review was conducted to identify existing readiness tools. The ABCD - SAT was adapted using a consensus approach that emphasized a community-based participatory approach and prioritized the knowledge and wisdom held by community members. The tool was piloted with a group of 16 people from 7 provinces and 11 partnering communities to assess language use, clarity, relevance, format, and ease of completion using examples. Internal reliability analysis and convergence validity were conducted with data from 53 clinical team members from 11 First Nations communities (3-5 per community) who have participated in the FORGE AHEAD program. The 27-page Clinical Readiness Consultation Tool (CRCT) consists of five main components, 21 sub-components, and 74 items that are aligned with the Expanded Chronic Care Model. Five-point Likert scale feedback from the pilot ranged from 3.25 to 4.5. Length of the tool was reported as a drawback but respondents noted that all the items were needed to provide a comprehensive picture of the healthcare system. Results for internal consistency showed that all sub-components except for two were within acceptable ranges (0.77-0.93). The Team Structure and Function sub-component scale had a moderately significant positive correlation with the validated Team Climate Inventory, r = 0.45, p < 0.05. The testing and validation of the FORGE AHEAD CRCT demonstrated that the tool is acceptable, valid and reliable. The CRCT has been successfully used to support the implementation of the FORGE AHEAD Program and the health services changes that partnering First Nations communities have designed and undertaken to improve diabetes care. Current ClinicalTrial.gov protocol ID NCT02234973 . Date of Registration: July 30, 2014.

Assessing teamwork performance in obstetrics: A systematic search and review of validated tools.

PubMed

Fransen, Annemarie F; de Boer, Liza; Kienhorst, Dieneke; Truijens, Sophie E; van Runnard Heimel, Pieter J; Oei, S Guid

2017-09-01

Teamwork performance is an essential component for the clinical efficiency of multi-professional teams in obstetric care. As patient safety is related to teamwork performance, it has become an important learning goal in simulation-based education. In order to improve teamwork performance, reliable assessment tools are required. These can be used to provide feedback during training courses, or to compare learning effects between different types of training courses. The aim of the current study is to (1) identify the available assessment tools to evaluate obstetric teamwork performance in a simulated environment, and (2) evaluate their psychometric properties in order to identify the most valuable tool(s) to use. We performed a systematic search in PubMed, MEDLINE, and EMBASE to identify articles describing assessment tools for the evaluation of obstetric teamwork performance in a simulated environment. In order to evaluate the quality of the identified assessment tools the standards and grading rules have been applied as recommended by the Accreditation Council for Graduate Medical Education (ACGME) Committee on Educational Outcomes. The included studies were also assessed according to the Oxford Centre for Evidence Based Medicine (OCEBM) levels of evidence. This search resulted in the inclusion of five articles describing the following six tools: Clinical Teamwork Scale, Human Factors Rating Scale, Global Rating Scale, Assessment of Obstetric Team Performance, Global Assessment of Obstetric Team Performance, and the Teamwork Measurement Tool. Based on the ACGME guidelines we assigned a Class 3, level C of evidence, to all tools. Regarding the OCEBM levels of evidence, a level 3b was assigned to two studies and a level 4 to four studies. The Clinical Teamwork Scale demonstrated the most comprehensive validation, and the Teamwork Measurement Tool demonstrated promising results, however it is recommended to further investigate its reliability. Copyright © 2017. Published by Elsevier B.V.

Assessing college-level learning difficulties and "at riskness" for learning disabilities and ADHD: development and validation of the learning difficulties assessment.

PubMed

Kane, Steven T; Walker, John H; Schmidt, George R

2011-01-01

This article describes the development and validation of the Learning Difficulties Assessment (LDA), a normed and web-based survey that assesses perceived difficulties with reading, writing, spelling, mathematics, listening, concentration, memory, organizational skills, sense of control, and anxiety in college students. The LDA is designed to (a) map individual learning strengths and weaknesses, (b) provide users with a comparative sense of their academic skills, (c) integrate research in user-interface design to assist those with reading and learning challenges, and (d) identify individuals who may be at risk for learning disabilities and attention-deficit/hyperactivity disorder (ADHD) and who should thus be further assessed. Data from a large-scale 5-year study describing the instrument's validity as a screening tool for learning disabilities and ADHD are presented. This article also describes unique characteristics of the LDA including its user-interface design, normative characteristics, and use as a no-cost screening tool for identifying college students at risk for learning disorders and ADHD.

Initial Validation of the Usage Rating Profile-Assessment for Use within German Language Schools

ERIC Educational Resources Information Center

Briesch, Amy M.; Casale, Gino; Grosche, Michael; Volpe, Robert J.; Hennemann, Thomas

2017-01-01

Modern attempts to explain why some assessment tools are readily adopted by school-based personnel whereas others are not have focused on the concept of usability. Usability encompasses not only the degree to which consumers find an assessment tool to be acceptable, but also the degree to which it is well-understood, believed to be feasible,…

A unified approach to validation, reliability, and education study design for surgical technical skills training.

PubMed

Sweet, Robert M; Hananel, David; Lawrenz, Frances

2010-02-01

To present modern educational psychology theory and apply these concepts to validity and reliability of surgical skills training and assessment. In a series of cross-disciplinary meetings, we applied a unified approach of behavioral science principles and theory to medical technical skills education given the recent advances in the theories in the field of behavioral psychology and statistics. While validation of the individual simulation tools is important, it is only one piece of a multimodal curriculum that in and of itself deserves examination and study. We propose concurrent validation throughout the design of simulation-based curriculum rather than once it is complete. We embrace the concept that validity and curriculum development are interdependent, ongoing processes that are never truly complete. Individual predictive, construct, content, and face validity aspects should not be considered separately but as interdependent and complementary toward an end application. Such an approach could help guide our acceptance and appropriate application of these exciting new training and assessment tools for technical skills training in medicine.

Self-management in chronic conditions: partners in health scale instrument validation.

PubMed

Peñarrieta-de Córdova, Isabel; Barrios, Flores Florabel; Gutierrez-Gomes, Tranquilina; Piñonez-Martinez, Ma del Socorro; Quintero-Valle, Luz Maria; Castañeda-Hidalgo, Hortensia

2014-03-01

This article describes a study that aimed to validate the Self-care in Chronic Conditions Partners in Health Scale instrument in the Mexican population. The instrument has been validated in Australia for use as a screening tool by primary healthcare professionals to assess the self-care skills and abilities of people with a chronic illness. Validation was conducted using baseline data for 552 people with diabetes, hypertension and cancer aged 18 or older who were users of healthcare centres in Tampico, Tamaulipas, Mexico. Results show high reliability and validity of the instrument and three themes were identified: knowledge, adherence, and dealing with and managing side effects. The findings suggest the scale is useful as a generic self-rated clinical tool for assessing self-management in a range of chronic conditions, and provides an outcome measure for comparing populations and change in patient self-management knowledge and behaviour. The authors recommend validating the scale in other Latin-American settings with more research into the effect of gender on self- management.

Elaboration and Validation of the Medication Prescription Safety Checklist 1

PubMed Central

Pires, Aline de Oliveira Meireles; Ferreira, Maria Beatriz Guimarães; do Nascimento, Kleiton Gonçalves; Felix, Márcia Marques dos Santos; Pires, Patrícia da Silva; Barbosa, Maria Helena

2017-01-01

ABSTRACT Objective: to elaborate and validate a checklist to identify compliance with the recommendations for the structure of medication prescriptions, based on the Protocol of the Ministry of Health and the Brazilian Health Surveillance Agency. Method: methodological research, conducted through the validation and reliability analysis process, using a sample of 27 electronic prescriptions. Results: the analyses confirmed the content validity and reliability of the tool. The content validity, obtained by expert assessment, was considered satisfactory as it covered items that represent the compliance with the recommendations regarding the structure of the medication prescriptions. The reliability, assessed through interrater agreement, was excellent (ICC=1.00) and showed perfect agreement (K=1.00). Conclusion: the Medication Prescription Safety Checklist showed to be a valid and reliable tool for the group studied. We hope that this study can contribute to the prevention of adverse events, as well as to the improvement of care quality and safety in medication use. PMID:28793128

A web-based team-oriented medical error communication assessment tool: development, preliminary reliability, validity, and user ratings.

PubMed

Kim, Sara; Brock, Doug; Prouty, Carolyn D; Odegard, Peggy Soule; Shannon, Sarah E; Robins, Lynne; Boggs, Jim G; Clark, Fiona J; Gallagher, Thomas

2011-01-01

Multiple-choice exams are not well suited for assessing communication skills. Standardized patient assessments are costly and patient and peer assessments are often biased. Web-based assessment using video content offers the possibility of reliable, valid, and cost-efficient means for measuring complex communication skills, including interprofessional communication. We report development of the Web-based Team-Oriented Medical Error Communication Assessment Tool, which uses videotaped cases for assessing skills in error disclosure and team communication. Steps in development included (a) defining communication behaviors, (b) creating scenarios, (c) developing scripts, (d) filming video with professional actors, and (e) writing assessment questions targeting team communication during planning and error disclosure. Using valid data from 78 participants in the intervention group, coefficient alpha estimates of internal consistency were calculated based on the Likert-scale questions and ranged from α=.79 to α=.89 for each set of 7 Likert-type discussion/planning items and from α=.70 to α=.86 for each set of 8 Likert-type disclosure items. The preliminary test-retest Pearson correlation based on the scores of the intervention group was r=.59 for discussion/planning and r=.25 for error disclosure sections, respectively. Content validity was established through reliance on empirically driven published principles of effective disclosure as well as integration of expert views across all aspects of the development process. In addition, data from 122 medicine and surgical physicians and nurses showed high ratings for video quality (4.3 of 5.0), acting (4.3), and case content (4.5). Web assessment of communication skills appears promising. Physicians and nurses across specialties respond favorably to the tool.

Validation of the Italian version of the Stanford Presenteeism Scale in nurses.

PubMed

Cicolini, Giancarlo; Della Pelle, Carlo; Cerratti, Francesca; Franza, Marcello; Flacco, Maria E

2016-07-01

To ascertain the validity and reliability of the Italian version of the Stanford Presenteeism Scale (SPS-6). Presenteeism has been associated with a work productivity reduction, a lower quality of work and an increased risk of developing health disorders. It is particularly high among nurses and needs valid tools to be assessed. A validation study was carried out from July to September 2014. A three-section tool, made of a demographic form, the Stanford Presenteeism Scale (SPS-6) and the Perceived Stress Scale (PSS-10) was administered to a sample of nurses, enrolled in three Italian hospitals. Cronbach's α for the entire sample (229 nurses) was found to be 0.72. A significant negative correlation between SPS and perceived stress scores evidenced the external validity. The factor analysis showed a two-component solution, accounting for 71.2% of the variance. The confirmatory factor analysis showed an adequate fit. The Italian SPS-6 is a valid and reliable tool for workplace surveys. Since the validity and reliability of SPS-6 has been confirmed for the Italian version, we have now a valid tool that can measure the levels of presenteeism among Italian nurses. © 2016 John Wiley & Sons Ltd.

Validity and reliability of the Turkish version of the pressure ulcer prevention knowledge assessment instrument.

PubMed

Tulek, Zeliha; Polat, Cansu; Ozkan, Ilknur; Theofanidis, Dimitris; Togrol, Rifat Erdem

2016-11-01

Sound knowledge of pressure ulcers is important to enable good prevention. There are limited instruments assessing pressure ulcer knowledge. The Pressure Ulcer Prevention Knowledge Assessment Instrument is among the scales of which psychometric properties have been studied rigorously and reflects the latest evidence. This study aimed to evaluate the validity and reliability of the Turkish version of the Pressure Ulcer Prevention Knowledge Assessment Instrument (PUPKAI-T), an instrument that assesses knowledge of pressure ulcer prevention by using multiple-choice questions. Linguistic validity was verified through front-to-back translation. Psychometric properties of the instrument were studied on a sample of 150 nurses working in a tertiary hospital in Istanbul, Turkey. The content validity index of the translated instrument was 0.94, intra-class correlation coefficients were between 0.37 and 0.80, item difficulty indices were between 0.21 and 0.88, discrimination indices were 0.20-0.78, and the Kuder Richardson for the internal consistency was 0.803. The PUPKAI-T was found to be a valid and reliable tool to evaluate nurses' knowledge on pressure ulcer prevention. The PUPKAI-T may be a useful tool for determining educational needs of nurses on pressure ulcer prevention. Copyright © 2016 Tissue Viability Society. Published by Elsevier Ltd. All rights reserved.

Developing and testing instruments for improving cooperation and patient's participation in mental health care.

PubMed

Latvala, E; Saranto, K; Pekkala, E

2004-10-01

The main purpose of the project was to develop computerized instruments that could be used by nurses and patients to assess their cooperation and mutual contributions to care. This paper presents a part of the project: the reliability and validity testing phase of a process of instrument development. To test the validity and reliability of the instruments, data were collected with questionnaires from nurses (n = 146) and patients (n = 286). The validity evaluated as construct validity and the reliability evaluated as internal consistency of the instruments were quite good. Construct validity was tested by factor analysis, and internal consistency was tested by Cronbach's alpha coefficient, which varied from 0.69 to 0.79. The instruments, which consisted of a software application that can be operated in a www environment, were meant to be used as tools in the psychiatric nursing context for assessing the cooperation between the nurses and patients and the patient's participation in his/her care. Furthermore, the computer programme can be used as a tool for developing and assessing the patient orientation in nursing.

Assessing safety climate in acute hospital settings: a systematic review of the adequacy of the psychometric properties of survey measurement tools.

PubMed

Alsalem, Gheed; Bowie, Paul; Morrison, Jillian

2018-05-10

The perceived importance of safety culture in improving patient safety and its impact on patient outcomes has led to a growing interest in the assessment of safety climate in healthcare organizations; however, the rigour with which safety climate tools were developed and psychometrically tested was shown to be variable. This paper aims to identify and review questionnaire studies designed to measure safety climate in acute hospital settings, in order to assess the adequacy of reported psychometric properties of identified tools. A systematic review of published empirical literature was undertaken to examine sample characteristics and instrument details including safety climate dimensions, origin and theoretical basis, and extent of psychometric evaluation (content validity, criterion validity, construct validity and internal reliability). Five questionnaire tools, designed for general evaluation of safety climate in acute hospital settings, were included. Detailed inspection revealed ambiguity around concepts of safety culture and climate, safety climate dimensions and the methodological rigour associated with the design of these measures. Standard reporting of the psychometric properties of developed questionnaires was variable, although evidence of an improving trend in the quality of the reported psychometric properties of studies was noted. Evidence of the theoretical underpinnings of climate tools was limited, while a lack of clarity in the relationship between safety culture and patient outcome measures still exists. Evidence of the adequacy of the psychometric development of safety climate questionnaire tools is still limited. Research is necessary to resolve the controversies in the definitions and dimensions of safety culture and climate in healthcare and identify related inconsistencies. More importance should be given to the appropriate validation of safety climate questionnaires before extending their usage in healthcare contexts different from those in which they were originally developed. Mixed methods research to understand why psychometric assessment and measurement reporting practices can be inadequate and lacking in a theoretical basis is also necessary.

Validation of Assessment Vignettes and Scoring Rubric of Multicultural and International Competency in Faculty Teaching

ERIC Educational Resources Information Center

Henderson, Sheila J.; Horton, Ruth A.; Saito, Paul K.; Shorter-Gooden, Kumea

2016-01-01

The purpose of this research was to develop a new tool for assessing multicultural and international competency in faculty teaching through vignette scenarios of university classroom critical incidents--across disciplines of clinical and forensics psychology, business, and education. Construct and content validity of the initial draft vignettes…

Exploring the Validity of a Second Language Intercultural Pragmatics Assessment Tool

ERIC Educational Resources Information Center

Timpe-Laughlin, Veronika; Choi, Ikkyu

2017-01-01

Pragmatics has been a key component of language competence frameworks. While the majority of second/foreign language (L2) pragmatics tests have targeted productive skills, the assessment of receptive pragmatic skills remains a developing field. This study explores validation evidence for a test of receptive L2 pragmatic ability called the American…

Establishing the Validity of the Task-Based English Speaking Test (TBEST) for International Teaching Assistants

ERIC Educational Resources Information Center

Witt, Autumn Song

2010-01-01

This dissertation follows an oral language assessment tool from initial design and implementation to validity analysis. The specialized variables of this study are the population: international teaching assistants and the purpose: spoken assessment as a hiring prerequisite. However, the process can easily be applied to other populations and…

Validation of a 4-item Negative Symptom Assessment (NSA-4): a short, practical clinical tool for the assessment of negative symptoms in schizophrenia.

PubMed

Alphs, Larry; Morlock, Robert; Coon, Cheryl; Cazorla, Pilar; Szegedi, Armin; Panagides, John

2011-06-01

The 16-item Negative Symptom Assessment (NSA-16) scale is a validated tool for evaluating negative symptoms of schizophrenia. The psychometric properties and predictive power of a four-item version (NSA-4) were compared with the NSA-16. Baseline data from 561 patients with predominant negative symptoms of schizophrenia who participated in two identically designed clinical trials were evaluated. Ordered logistic regression analysis of ratings using NSA-4 and NSA-16 were compared with ratings using several other standard tools to determine predictive validity and construct validity. Internal consistency and test--retest reliability were also analyzed. NSA-16 and NSA-4 scores were both predictive of scores on the NSA global rating (odds ratio = 0.83-0.86) and the Clinical Global Impressions--Severity scale (odds ratio = 0.91-0.93). NSA-16 and NSA-4 showed high correlation with each other (Pearson r = 0.85), similar high correlation with other measures of negative symptoms (demonstrating convergent validity), and lesser correlations with measures of other forms of psychopathology (demonstrating divergent validity). NSA-16 and NSA-4 both showed acceptable internal consistency (Cronbach α, 0.85 and 0.64, respectively) and test--retest reliability (intraclass correlation coefficient, 0.87 and 0.82). This study demonstrates that NSA-4 offers accuracy comparable to the NSA-16 in rating negative symptoms in patients with schizophrenia. Copyright © 2011 John Wiley & Sons, Ltd.

Development of the TeamOBS-PPH - targeting clinical performance in postpartum hemorrhage.

PubMed

Brogaard, Lise; Hvidman, Lone; Hinshaw, Kim; Kierkegaard, Ole; Manser, Tanja; Musaeus, Peter; Arafeh, Julie; Daniels, Kay I; Judy, Amy E; Uldbjerg, Niels

2018-06-01

This study aimed to develop a valid and reliable TeamOBS-PPH tool for assessing clinical performance in the management of postpartum hemorrhage (PPH). The tool was evaluated using video-recordings of teams managing PPH in both real-life and simulated settings. A Delphi panel consisting of 12 obstetricians from the UK, Norway, Sweden, Iceland, and Denmark achieved consensus on (i) the elements to include in the assessment tool, (ii) the weighting of each element, and (iii) the final tool. The validity and reliability were evaluated according to Cook and Beckman. (Level 1) Four raters scored four video-recordings of in situ simulations of PPH. (Level 2) Two raters scored 85 video-recordings of real-life teams managing patients with PPH ≥1000 mL in two Danish hospitals. (Level 3) Two raters scored 15 video-recordings of in situ simulations of PPH from a US hospital. The tool was designed with scores from 0 to 100. (Level 1) Teams of novices had a median score of 54 (95% CI 48-60), whereas experienced teams had a median score of 75 (95% CI 71-79; p < 0.001). (Level 2) The intra-rater [intra-class correlation (ICC) = 0.96] and inter-rater (ICC = 0.83) agreements for real-life PPH were strong. The tool was applicable in all cases: atony, retained placenta, and lacerations. (Level 3) The tool was easily adapted to in situ simulation settings in the USA (ICC = 0.86). The TeamOBS-PPH tool appears to be valid and reliable for assessing clinical performance in real-life and simulated settings. The tool will be shared as the free TeamOBS App. © 2018 Nordic Federation of Societies of Obstetrics and Gynecology.

A Protocol for Advanced Psychometric Assessment of Surveys

PubMed Central

Squires, Janet E.; Hayduk, Leslie; Hutchinson, Alison M.; Cranley, Lisa A.; Gierl, Mark; Cummings, Greta G.; Norton, Peter G.; Estabrooks, Carole A.

2013-01-01

Background and Purpose. In this paper, we present a protocol for advanced psychometric assessments of surveys based on the Standards for Educational and Psychological Testing. We use the Alberta Context Tool (ACT) as an exemplar survey to which this protocol can be applied. Methods. Data mapping, acceptability, reliability, and validity are addressed. Acceptability is assessed with missing data frequencies and the time required to complete the survey. Reliability is assessed with internal consistency coefficients and information functions. A unitary approach to validity consisting of accumulating evidence based on instrument content, response processes, internal structure, and relations to other variables is taken. We also address assessing performance of survey data when aggregated to higher levels (e.g., nursing unit). Discussion. In this paper we present a protocol for advanced psychometric assessment of survey data using the Alberta Context Tool (ACT) as an exemplar survey; application of the protocol to the ACT survey is underway. Psychometric assessment of any survey is essential to obtaining reliable and valid research findings. This protocol can be adapted for use with any nursing survey. PMID:23401759

Development and evaluation of a brief screener to estimate fast-food and beverage consumption among adolescents.

PubMed

Nelson, Melissa C; Lytle, Leslie A

2009-04-01

Sweetened beverage and fast-food intake have been identified as important targets for obesity prevention. However, there are few brief dietary assessment tools available to evaluate these behaviors among adolescents. The objective of this research was to examine reliability and validity of a 22-item dietary screener assessing adolescent consumption of specific energy-containing and non-energy-containing beverages (nine items) and fast food (13 items). The screener was administered to adolescents (ages 11 to 18 years) recruited from the Minneapolis/St Paul, MN, metro region. One sample of adolescents completed test-retest reliability of the screener (n=33, primarily white adolescents). Another adolescent sample completed the screener along with three 24-hour dietary recalls to assess criterion validity (n=59 white adolescents). Test-retest assessments were completed approximately 7 to 14 days apart, and agreement between the two administrations of the screener was substantial, with most items yielding Spearman correlations and kappa statistics that were >0.60. When compared to the gold standard dietary recall data, findings indicate that the validity of the screener items assessing adolescents' intake of regular soda, sports drinks, milk, and water was fair. However, the differential assessment periods captured by the two methods (ie, 1 month for the screener vs 3 days for the recalls) posed challenges in analysis and made it impossible to assess the validity of some screener items. Overall while these screener items largely represent reliable measures with fair validity, our findings highlight the challenges inherent in the validation of brief dietary assessment tools.

Tools for observational gait analysis in patients with stroke: a systematic review.

PubMed

Ferrarello, Francesco; Bianchi, Valeria Anna Maria; Baccini, Marco; Rubbieri, Gaia; Mossello, Enrico; Cavallini, Maria Chiara; Marchionni, Niccolò; Di Bari, Mauro

2013-12-01

Stroke severely affects walking ability, and assessment of gait kinematics is important in defining diagnosis, planning treatment, and evaluating interventions in stroke rehabilitation. Although observational gait analysis is the most common approach to evaluate gait kinematics, tools useful for this purpose have received little attention in the scientific literature and have not been thoroughly reviewed. The aims of this systematic review were to identify tools proposed to conduct observational gait analysis in adults with a stroke, to summarize evidence concerning their quality, and to assess their implementation in rehabilitation research and clinical practice. An extensive search was performed of original articles reporting on visual/observational tools developed to investigate gait kinematics in adults with a stroke. Two reviewers independently selected studies, extracted data, assessed quality of the included studies, and scored the metric properties and clinical utility of each tool. Rigor in reporting metric properties and dissemination of the tools also was evaluated. Five tools were identified, not all of which had been tested adequately for their metric properties. Evaluation of content validity was partially satisfactory. Reliability was poorly investigated in all but one tool. Concurrent validity and sensitivity to change were shown for 3 and 2 tools, respectively. Overall, adequate levels of quality were rarely reached. The dissemination of the tools was poor. Based on critical appraisal, the Gait Assessment and Intervention Tool shows a good level of quality, and its use in stroke rehabilitation is recommended. Rigorous studies are needed for the other tools in order to establish their usefulness.

The development and validation of a meta-tool for quality appraisal of public health evidence: Meta Quality Appraisal Tool (MetaQAT).

PubMed

Rosella, L; Bowman, C; Pach, B; Morgan, S; Fitzpatrick, T; Goel, V

2016-07-01

Most quality appraisal tools were developed for clinical medicine and tend to be study-specific with a strong emphasis on risk of bias. In order to be more relevant to public health, an appropriate quality appraisal tool needs to be less reliant on the evidence hierarchy and consider practice applicability. Given the broad range of study designs used in public health, the objective of this study was to develop and validate a meta-tool that combines public health-focused principles of appraisal coupled with a set of design-specific companion tools. Several design methods were used to develop and validate the tool including literature review, synthesis, and validation with a reference standard. A search of critical appraisal tools relevant to public health was conducted; core concepts were collated. The resulting framework was piloted during three feedback sessions with public health practitioners. Following subsequent revisions, the final meta-tool, the Meta Quality Appraisal Tool (MetaQAT), was then validated through a content analysis of appraisals conducted by two groups of experienced public health researchers (MetaQAT vs generic appraisal form). The MetaQAT framework consists of four domains: relevancy, reliability, validity, and applicability. In addition, a companion tool was assembled from existing critical appraisal tools to provide study design-specific guidance on validity appraisal. Content analysis showed similar methodological and generalizability concerns were raised by both groups; however, the MetaQAT appraisers commented more extensively on applicability to public health practice. Critical appraisal tools designed for clinical medicine have limitations for use in the context of public health. The meta-tool structure of the MetaQAT allows for rigorous appraisal, while allowing users to simultaneously appraise the multitude of study designs relevant to public health research and assess non-standard domains, such as applicability. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Translating and validating a Training Needs Assessment tool into Greek

PubMed Central

Markaki, Adelais; Antonakis, Nikos; Hicks, Carolyn M; Lionis, Christos

2007-01-01

Background The translation and cultural adaptation of widely accepted, psychometrically tested tools is regarded as an essential component of effective human resource management in the primary care arena. The Training Needs Assessment (TNA) is a widely used, valid instrument, designed to measure professional development needs of health care professionals, especially in primary health care. This study aims to describe the translation, adaptation and validation of the TNA questionnaire into Greek language and discuss possibilities of its use in primary care settings. Methods A modified version of the English self-administered questionnaire consisting of 30 items was used. Internationally recommended methodology, mandating forward translation, backward translation, reconciliation and pretesting steps, was followed. Tool validation included assessing item internal consistency, using the alpha coefficient of Cronbach. Reproducibility (test – retest reliability) was measured by the kappa correlation coefficient. Criterion validity was calculated for selected parts of the questionnaire by correlating respondents' research experience with relevant research item scores. An exploratory factor analysis highlighted how the items group together, using a Varimax (oblique) rotation and subsequent Cronbach's alpha assessment. Results The psychometric properties of the Greek version of the TNA questionnaire for nursing staff employed in primary care were good. Internal consistency of the instrument was very good, Cronbach's alpha was found to be 0.985 (p < 0.001) and Kappa coefficient for reproducibility was found to be 0.928 (p < 0.0001). Significant positive correlations were found between respondents' current performance levels on each of the research items and amount of research involvement, indicating good criterion validity in the areas tested. Factor analysis revealed seven factors with eigenvalues of > 1.0, KMO (Kaiser-Meyer-Olkin) measure of sampling adequacy = 0.680 and Bartlett's test of sphericity, p < 0.001. Conclusion The translated and adapted Greek version is comparable with the original English instrument in terms of validity and reliability and it is suitable to assess professional development needs of nursing staff in Greek primary care settings. PMID:17474989

Computational assessment of hemodynamics-based diagnostic tools using a database of virtual subjects: Application to three case studies.

PubMed

Willemet, Marie; Vennin, Samuel; Alastruey, Jordi

2016-12-08

Many physiological indexes and algorithms based on pulse wave analysis have been suggested in order to better assess cardiovascular function. Because these tools are often computed from in-vivo hemodynamic measurements, their validation is time-consuming, challenging, and biased by measurement errors. Recently, a new methodology has been suggested to assess theoretically these computed tools: a database of virtual subjects generated using numerical 1D-0D modeling of arterial hemodynamics. The generated set of simulations encloses a wide selection of healthy cases that could be encountered in a clinical study. We applied this new methodology to three different case studies that demonstrate the potential of our new tool, and illustrated each of them with a clinically relevant example: (i) we assessed the accuracy of indexes estimating pulse wave velocity; (ii) we validated and refined an algorithm that computes central blood pressure; and (iii) we investigated theoretical mechanisms behind the augmentation index. Our database of virtual subjects is a new tool to assist the clinician: it provides insight into the physical mechanisms underlying the correlations observed in clinical practice. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

The development of the PARENTS: a tool for parents to assess residents' non-technical skills in pediatric emergency departments.

PubMed

Moreau, Katherine A; Eady, Kaylee; Tang, Kenneth; Jabbour, Mona; Frank, Jason R; Campbell, Meaghan; Hamstra, Stanley J

2017-11-14

Parents can assess residents' non-technical skills (NTS) in pediatric emergency departments (EDs). There are no assessment tools, with validity evidence, for parental use in pediatric EDs. The purpose of this study was to develop the Parents' Assessment of Residents Enacting Non-Technical Skills (PARENTS) educational assessment tool and collect three sources of validity evidence (i.e., content, response process, internal structure) for it. We established content evidence for the PARENTS through interviews with physician-educators and residents, focus groups with parents, a literature review, and a modified nominal group technique with experts. We collected response process evidence through cognitive interviews with parents. To examine the internal structure evidence, we administered the PARENTS and performed exploratory factor analysis. Initially, a 20-item PARENTS was developed. Cognitive interviews led to the removal of one closed-ended item, the addition of resident photographs, and wording/formatting changes. Thirty-seven residents and 434 parents participated in the administration of the resulting 19-item PARENTS. Following factor analysis, a one-factor model prevailed. The study presents initial validity evidence for the PARENTS. It also highlights strategies for potentially: (a) involving parents in the assessment of residents, (b) improving the assessment of NTS in pediatric EDs, and (c) capturing parents' perspectives to improve the preparation of future physicians.

Validation of a General and Sport Nutrition Knowledge Questionnaire in Adolescents and Young Adults: GeSNK.

PubMed

Calella, Patrizia; Iacullo, Vittorio Maria; Valerio, Giuliana

2017-04-29

Good knowledge of nutrition is widely thought to be an important aspect to maintaining a balanced and healthy diet. The aim of this study was to develop and validate a new reliable tool to measure the general and the sport nutrition knowledge (GeSNK) in people who used to practice sports at different levels. The development of (GeSNK) was carried out in six phases as follows: (1) item development and selection by a panel of experts; (2) pilot study in order to assess item difficulty and item discrimination; (3) measurement of the internal consistency; (4) reliability assessment with a 2-week test-retest analysis; (5) concurrent validity was tested by administering the questionnaire along with other two similar tools; (6) construct validity by administering the questionnaire to three groups of young adults with different general nutrition and sport nutrition knowledge. The final questionnaire, consisted of 62 items of the original 183 questions. It is a consistent, valid, and suitable instrument that can be applied over time, making it a promising tool to look at the relationship between nutrition knowledge, demographic characteristics, and dietary behavior in adolescents and young adults.

Nursing Minimum Data Sets for documenting nutritional care for adults in primary healthcare: a scoping review.

PubMed

Håkonsen, Sasja Jul; Pedersen, Preben Ulrich; Bjerrum, Merete; Bygholm, Ann; Peters, Micah D J

2018-01-01

To identify all published nutritional screening instruments that have been validated in the adult population in primary healthcare settings and to report on their psychometric validity. Within health care, there is an urgent need for the systematic collection of nursing care data in order to make visible what nurses do and to facilitate comparison, quality assurance, management, research and funding of nursing care. To be effective, nursing records should accurately and comprehensively document all required information to support safe and high quality care of patients. However, this process of documentation has been criticized from many perspectives as being highly inadequate. A Nursing Minimum Data Set within the nutritional area in primary health care could therefore be beneficial in order to support nurses in their daily documentation and observation of patients. The review considered studies that included adults aged over 18 years of any gender, culture, diagnosis and ethnicity, as well as nutritional experts, patients and their relatives. The concepts of interest were: the nature and content of any nutritional screening tools validated (regardless of the type of validation) in the adult population in primary healthcare; and the views and opinions of eligible participants regarding the appropriateness of nutritional assessment were the concept of interest. Studies included must have been conducted in primary healthcare settings, both within home care and nursing home facilities. This scoping review used a two-step approach as a preliminary step to the subsequent development of a Nursing Minimum Data Set within the nutritional area in primary healthcare: i) a systematic literature search of existing nutritional screening tools validated in primary health care; and ii) a systematic literature search on nutritional experts opinions on the assessment of nutritional nursing care of adults in primary healthcare as well as the views of patients and their relatives. Multiple databases (PubMed, CINAHL, Embase, Scopus, Swemed+, MedNar, CDC, MEDION, Health Technology Assessment Database, TRIP database, NTIS, ProQuest Dissertations and Theses, Google Scholar, Current Contents) were searched from their inception to September 2016. The results from the studies were extracted using pre-developed extraction tools to all three questions, and have been presented narratively and by using figures to support the text. Twenty-nine nutritional screening tools that were validated within a primary care setting, and two documents on consensus statements regarding expert opinion were identified. No studies on the patients or relatives views were identified. The nutritional screening instruments have solely been validated in an over-55 population. Construct validity was the type of validation most frequently used in the validation process covering a total of 25 of the 29 tools. Two studies were identified in relation to the third review question. These two documents are both consensus statement documents developed by experts within the geriatric and nutritional care field. Overall, experts find it appropriate to: i) conduct a comprehensive geriatric assessment, ii) use a validated nutritional screening instrument, and iii) conduct a history and clinical diagnosis, physical examination and dietary assessment when assessing primarily the elderly's nutritional status in primary health care.

Validation of a Videoconferenced Speaking Test

ERIC Educational Resources Information Center

Kim, Jungtae; Craig, Daniel A.

2012-01-01

Videoconferencing offers new opportunities for language testers to assess speaking ability in low-stakes diagnostic tests. To be considered a trusted testing tool in language testing, a test should be examined employing appropriate validation processes [Chapelle, C.A., Jamieson, J., & Hegelheimer, V. (2003). "Validation of a web-based ESL…

Pain assessment in elderly with dementia: Brazilian validation of the PACSLAC scale

PubMed Central

Thé, Karol Bezerra; Gazoni, Fernanda Martins; Cherpak, Guilherme Liausu; Lorenzet, Isabel Clasen; dos Santos, Luciana Alves; Nardes, Edlene Maria; dos Santos, Fânia Cristina

2016-01-01

ABSTRACT Objective To validate the Pain Assessment Checklist for Seniors with Limited Ability to Communicate – Portuguese in demented elderly and to analyze its measurement properties. Methods We evaluated 50 elderly with dementia, residing in a nursing home and with limited communication ability, when exposed to potentially painful situations. The tool was applied at two different moments. First, two interviewers applied it simultaneously, and the intensity of pain was asked based on the caregiver’s opinion. After 14 days, with no analgesic intervention, one of the interviewers applied it again. Results The sample comprised more females, aged over 80 years, with dementia due to Alzheimer, presenting musculoskeletal pain of moderate to severe intensity. The psychometric properties of the tool demonstrated appropriate internal consistency (Cronbach’s alpha coefficient of 0.827). The scale had excellent reproducibility, according to the intraclass correlation coefficient, and the tool has been duly validated. Conclusion The Pain Assessment Checklist for Seniors with Limited Ability to Communicate – Portuguese had adequate measuring properties for use with elderly presenting limited communication. PMID:27462888

Assessing reliability and validity of the GroPromo audit tool for evaluation of grocery store marketing and promotional environments.

PubMed

Kerr, Jacqueline; Sallis, James F; Bromby, Erica; Glanz, Karen

2012-01-01

To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Trained observers used the GroPromo instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity, customers' receipts were coded for percentage of food purchases in each of the categories. Of the 22 categories tested, 21 demonstrated moderate to high interrater reliability (intraclass correlation ≥ 0.61). When more unhealthy items were placed in prominent locations, a higher percentage of money was spent on less-healthy items, and a lower percentage of food dollars were spent on fruits and vegetables. The prominence of locations was more important than the number of locations. The GroPromo tool can be used to assess promotional practices in stores. Data may help advocates campaign for more healthy food items in key promotional locations. Copyright © 2012 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

PubMed

Shea, Beverley J; Hamel, Candyce; Wells, George A; Bouter, Lex M; Kristjansson, Elizabeth; Grimshaw, Jeremy; Henry, David A; Boers, Maarten

2009-10-01

Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR). We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times. The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete. AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.

Children and Young People-Mental Health Safety Assessment Tool (CYP-MH SAT) study: Protocol for the development and psychometric evaluation of an assessment tool to identify immediate risk of self-harm and suicide in children and young people (10-19 years) in acute paediatric hospital settings.

PubMed

Manning, Joseph C; Walker, Gemma M; Carter, Tim; Aubeeluck, Aimee; Witchell, Miranda; Coad, Jane

2018-04-12

Currently, no standardised, evidence-based assessment tool for assessing immediate self-harm and suicide in acute paediatric inpatient settings exists. The aim of this study is to develop and test the psychometric properties of an assessment tool that identifies immediate risk of self-harm and suicide in children and young people (10-19 years) in acute paediatric hospital settings. Development phase: This phase involved a scoping review of the literature to identify and extract items from previously published suicide and self-harm risk assessment scales. Using a modified electronic Delphi approach, these items will then be rated according to their relevance for assessment of immediate suicide or self-harm risk by expert professionals. Inclusion of items will be determined by 65%-70% consensus between raters. Subsequently, a panel of expert members will convene to determine the face validity, appropriate phrasing, item order and response format for the finalised items.Psychometric testing phase: The finalised items will be tested for validity and reliability through a multicentre, psychometric evaluation. Psychometric testing will be undertaken to determine the following: internal consistency, inter-rater reliability, convergent, divergent validity and concurrent validity. Ethical approval was provided by the National Health Service East Midlands-Derby Research Ethics Committee (17/EM/0347) and full governance clearance received by the Health Research Authority and local participating sites. Findings from this study will be disseminated to professionals and the public via peer-reviewed journal publications, popular social media and conference presentations. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Understanding diagnosis and management of dementia and guideline implementation in general practice: a qualitative study using the theoretical domains framework.

PubMed

Murphy, Kerry; O'Connor, Denise A; Browning, Colette J; French, Simon D; Michie, Susan; Francis, Jill J; Russell, Grant M; Workman, Barbara; Flicker, Leon; Eccles, Martin P; Green, Sally E

2014-03-03

Dementia is a growing problem, causing substantial burden for patients, their families, and society. General practitioners (GPs) play an important role in diagnosing and managing dementia; however, there are gaps between recommended and current practice. The aim of this study was to explore GPs' reported practice in diagnosing and managing dementia and to describe, in theoretical terms, the proposed explanations for practice that was and was not consistent with evidence-based guidelines. Semi-structured interviews were conducted with GPs in Victoria, Australia. The Theoretical Domains Framework (TDF) guided data collection and analysis. Interviews explored the factors hindering and enabling achievement of 13 recommended behaviours. Data were analysed using content and thematic analysis. This paper presents an in-depth description of the factors influencing two behaviours, assessing co-morbid depression using a validated tool, and conducting a formal cognitive assessment using a validated scale. A total of 30 GPs were interviewed. Most GPs reported that they did not assess for co-morbid depression using a validated tool as per recommended guidance. Barriers included the belief that depression can be adequately assessed using general clinical indicators and that validated tools provide little additional information (theoretical domain of 'Beliefs about consequences'); discomfort in using validated tools ('Emotion'), possibly due to limited training and confidence ('Skills'; 'Beliefs about capabilities'); limited awareness of the need for, and forgetting to conduct, a depression assessment ('Knowledge'; 'Memory, attention and decision processes'). Most reported practising in a manner consistent with the recommendation that a formal cognitive assessment using a validated scale be undertaken. Key factors enabling this were having an awareness of the need to conduct a cognitive assessment ('Knowledge'); possessing the necessary skills and confidence ('Skills'; 'Beliefs about capabilities'); and having adequate time and resources ('Environmental context and resources'). This is the first study to our knowledge to use a theoretical approach to investigate the barriers and enablers to guideline-recommended diagnosis and management of dementia in general practice. It has identified key factors likely to explain GPs' uptake of the guidelines. The results have informed the design of an intervention aimed at supporting practice change in line with dementia guidelines, which is currently being evaluated in a cluster randomised trial.

An Evaluation of the Predictive Validity of Confidence Ratings in Identifying Functional Behavioral Assessment Hypothesis Statements

ERIC Educational Resources Information Center

Borgmeier, Chris; Horner, Robert H.

2006-01-01

Faced with limited resources, schools require tools that increase the accuracy and efficiency of functional behavioral assessment. Yarbrough and Carr (2000) provided evidence that informant confidence ratings of the likelihood of problem behavior in specific situations offered a promising tool for predicting the accuracy of function-based…

An Evaluation Framework and Instrument for Evaluating e-Assessment Tools

ERIC Educational Resources Information Center

Singh, Upasana Gitanjali; de Villiers, Mary Ruth

2017-01-01

e-Assessment, in the form of tools and systems that deliver and administer multiple choice questions (MCQs), is used increasingly, raising the need for evaluation and validation of such systems. This research uses literature and a series of six empirical action research studies to develop an evaluation framework of categories and criteria called…

Prevalence of Pediatric Asthma Risk in Santo Domingo, Dominican Republic.

PubMed

Sun, Wendy W; Gupta, Lipi; Andreae, Andrew E; Romutis, Kristin; Borda, Allison M; Sabu, Priya; McKenna, Sean; Ryan, Mark

Early detection and treatment of pediatric asthma could reduce morbidity and lessen burden on society. Currently there is no known research on the prevalence of pediatric asthma in the Dominican Republic (DR) and no known asthma risk assessment tool for one-time encounters in a fast-paced clinic. To pilot a streamlined version of previously validated screening tools to estimate the prevalence of pediatric asthma risk in Santo Domingo Norte, DR. A combined asthma questionnaire and clinical assessment tool was developed and administered to patients aged 2-12 years. We found that 25.7% of the 74 study participants were categorized as probable asthma, 21.6% were at high risk for asthma, 14.9% elevated risk, and 37.8% not at risk. If the prevalence of 25.7% is representative of the DR as a whole, the DR would have one of the highest national rates in Latin America. The study assessment tool was convenient to use, but tool validation is needed. Copyright © 2016 Icahn School of Medicine at Mount Sinai. Published by Elsevier Inc. All rights reserved.

SurvExpress: an online biomarker validation tool and database for cancer gene expression data using survival analysis.

PubMed

Aguirre-Gamboa, Raul; Gomez-Rueda, Hugo; Martínez-Ledesma, Emmanuel; Martínez-Torteya, Antonio; Chacolla-Huaringa, Rafael; Rodriguez-Barrientos, Alberto; Tamez-Peña, José G; Treviño, Victor

2013-01-01

Validation of multi-gene biomarkers for clinical outcomes is one of the most important issues for cancer prognosis. An important source of information for virtual validation is the high number of available cancer datasets. Nevertheless, assessing the prognostic performance of a gene expression signature along datasets is a difficult task for Biologists and Physicians and also time-consuming for Statisticians and Bioinformaticians. Therefore, to facilitate performance comparisons and validations of survival biomarkers for cancer outcomes, we developed SurvExpress, a cancer-wide gene expression database with clinical outcomes and a web-based tool that provides survival analysis and risk assessment of cancer datasets. The main input of SurvExpress is only the biomarker gene list. We generated a cancer database collecting more than 20,000 samples and 130 datasets with censored clinical information covering tumors over 20 tissues. We implemented a web interface to perform biomarker validation and comparisons in this database, where a multivariate survival analysis can be accomplished in about one minute. We show the utility and simplicity of SurvExpress in two biomarker applications for breast and lung cancer. Compared to other tools, SurvExpress is the largest, most versatile, and quickest free tool available. SurvExpress web can be accessed in http://bioinformatica.mty.itesm.mx/SurvExpress (a tutorial is included). The website was implemented in JSP, JavaScript, MySQL, and R.

SurvExpress: An Online Biomarker Validation Tool and Database for Cancer Gene Expression Data Using Survival Analysis

PubMed Central

Aguirre-Gamboa, Raul; Gomez-Rueda, Hugo; Martínez-Ledesma, Emmanuel; Martínez-Torteya, Antonio; Chacolla-Huaringa, Rafael; Rodriguez-Barrientos, Alberto; Tamez-Peña, José G.; Treviño, Victor

2013-01-01

Validation of multi-gene biomarkers for clinical outcomes is one of the most important issues for cancer prognosis. An important source of information for virtual validation is the high number of available cancer datasets. Nevertheless, assessing the prognostic performance of a gene expression signature along datasets is a difficult task for Biologists and Physicians and also time-consuming for Statisticians and Bioinformaticians. Therefore, to facilitate performance comparisons and validations of survival biomarkers for cancer outcomes, we developed SurvExpress, a cancer-wide gene expression database with clinical outcomes and a web-based tool that provides survival analysis and risk assessment of cancer datasets. The main input of SurvExpress is only the biomarker gene list. We generated a cancer database collecting more than 20,000 samples and 130 datasets with censored clinical information covering tumors over 20 tissues. We implemented a web interface to perform biomarker validation and comparisons in this database, where a multivariate survival analysis can be accomplished in about one minute. We show the utility and simplicity of SurvExpress in two biomarker applications for breast and lung cancer. Compared to other tools, SurvExpress is the largest, most versatile, and quickest free tool available. SurvExpress web can be accessed in http://bioinformatica.mty.itesm.mx/SurvExpress (a tutorial is included). The website was implemented in JSP, JavaScript, MySQL, and R. PMID:24066126

Validation of Social Cognition Rating Tools in Indian Setting (SOCRATIS): A new test-battery to assess social cognition.

PubMed

Mehta, Urvakhsh M; Thirthalli, Jagadisha; Naveen Kumar, C; Mahadevaiah, Mahesh; Rao, Kiran; Subbakrishna, Doddaballapura K; Gangadhar, Bangalore N; Keshavan, Matcheri S

2011-09-01

Social cognition is a cognitive domain that is under substantial cultural influence. There are no culturally appropriate standardized tools in India to comprehensively test social cognition. This study describes validation of tools for three social cognition constructs: theory of mind, social perception and attributional bias. Theory of mind tests included adaptations of, (a) two first order tasks [Sally-Anne and Smarties task], (b) two second order tasks [Ice cream van and Missing cookies story], (c) two metaphor-irony tasks and (d) the faux pas recognition test. Internal, Personal, and Situational Attributions Questionnaire (IPSAQ) and Social Cue Recognition Test were adapted to assess attributional bias and social perception, respectively. These tests were first modified to suit the Indian cultural context without changing the constructs to be tested. A panel of experts then rated the tests on likert scales as to (1) whether the modified tasks tested the same construct as in the original and (2) whether they were culturally appropriate. The modified tests were then administered to groups of actively symptomatic and remitted schizophrenia patients as well as healthy comparison subjects. All tests of the Social Cognition Rating Tools in Indian Setting had good content validity and known groups validity. In addition, the social cure recognition test in Indian setting had good internal consistency and concurrent validity. Copyright © 2011 Elsevier B.V. All rights reserved.

Construct validity and inter-rater reliability of the Dutch activity measure for post-acute care "6-clicks" basic mobility form to assess the mobility of hospitalized patients.

PubMed

Geelen, Sven Jacobus Gertruda; Valkenet, Karin; Veenhof, Cindy

2018-05-12

To evaluate the construct validity and the inter-rater reliability of the Dutch Activity Measure for Post-Acute Care "6-clicks" Basic Mobility short form measuring the patient's mobility in Dutch hospital care. First, the "6-clicks" was translated by using a forward-backward translation protocol. Next, 64 patients were assessed by the physiotherapist to determine the validity while being admitted to the Internal Medicine wards of a university medical center. Six hypotheses were tested regarding the construct "mobility" which showed that: Better "6-clicks" scores were related to less restrictive pre-admission living situations (p = 0.011), less restrictive discharge locations (p = 0.001), more independence in activities of daily living (p = 0.001) and less physiotherapy visits (p < 0.001). A correlation was found between the "6-clicks" and length of stay (r= -0.408, p = 0.001), but not between the "6-clicks" and age (r= -0.180, p = 0.528). To determine the inter-rater reliability, an additional 50 patients were assessed by pairs of physiotherapists who independently scored the patients. Intraclass Correlation Coefficients of 0.920 (95%CI: 0.828-0.964) were found. The Kappa Coefficients for the individual items ranged from 0.649 (walking stairs) to 0.841 (sit-to-stand). The Dutch "6-clicks" shows a good construct validity and moderate-to-excellent inter-rater reliability when used to assess the mobility of hospitalized patients. Implications for Rehabilitation Even though various measurement tools have been developed, it appears the majority of physiotherapists working in a hospital currently do not use these tools as a standard part of their care. The Activity Measure for Post-Acute Care "6-clicks" Basic Mobility is the only tool which is designed to be short, easy to use within usual care and has been validated in the entire hospital population. This study shows that the Dutch version of the Activity Measure for Post-Acute Care "6-clicks" Basic Mobility form is a valid, easy to use, quick tool to assess the basic mobility of Dutch hospitalized patients.

Reliability and validity of the workplace harassment questionnaire for Korean finance and service workers.

PubMed

Lee, Myeongjun; Kim, Hyunjung; Shin, Donghee; Lee, Sangyun

2016-01-01

Harassment means systemic and repeated unethical acts. Research on workplace harassment have been conducted widely and the NAQ-R has been widely used for the researches. But this tool, however the limitations in revealing differended in sub-factors depending on the culture and in reflecting that unique characteristics of the Koren society. So, The workplace harassment questionnaire for Korean finace and service workers has been developed to assess the level of personal harassment at work. This study aims to develop a tool to assess the level of personal harassment at work and to test its validity and reliability while examining specific characteristics of workplace harassment against finance and service workers in Korea. The framework of survey was established based on literature review, focused-group interview for the Korean finance and service workers. To verify its reliability, Cronbach's alpha coefficient was calculated; and to verify its validity, items and factors of the tool were analyzed. The correlation matrix analysis was examined to verify the tool's convergent validity and discriminant validity. Structural validity was verified by checking statistical significance in relation to the BDI-K. Cronbach's alpha coefficient of this survey was 0.93, which indicates a quite high level of reliability. To verify the appropriateness of this survey tool, its construct validity was examined through factor analysis. As a result of the factor analysis, 3 factors were extracted, explaining 56.5 % of the total variance. The loading values and communalities of the 20 items were 0.85 to 0.48 and 0.71 to 0.46. The convergent validity and discriminant validity were analyzed and rate of item discriminant validity was 100 %. Finally, for the concurrent validity, We examined the relationship between the WHI-KFSW and pschosocial stress by examining the correlation with the BDI-K. The results of chi-square test and multiple logistic analysis indicated that the correlation with the BDI-K was satatisctically significant. Workplace harassment in actual workplaces were investigated based on interviews, and the statistical analysis contributed to systematizing the types of actual workplace harassment. By statistical method, we developed the questionare, 20 items of 3 categories.

Daily functioning profile of children with attention deficit hyperactive disorder: A pilot study using an ecological assessment.

PubMed

Rosenblum, Sara; Frisch, Carmit; Deutsh-Castel, Tsofia; Josman, Naomi

2015-01-01

Children with attention-deficit hyperactivity disorder (ADHD) often present with activities of daily living (ADL) performance deficits. This study aimed to compare the performance characteristics of children with ADHD to those of controls based on the Do-Eat assessment tool, and to establish the tool's validity. Participants were 23 children with ADHD and 24 matched controls, aged 6-9 years. In addition to the Do-Eat, the Children Activity Scale-Parent (ChAS-P) and the Behavioral Rating Inventory of Executive Function (BRIEF) were used to measure sensorimotor abilities and executive function (EF). Significant differences were found in the Do-Eat scores between children with ADHD and controls. Significant moderate correlations were found between the Do-Eat sensorimotor scores, the ChAS-P and the BRIEF scores in the ADHD group. Significant correlations were found between performance on the Do-Eat and the ChAS-P questionnaire scores, verifying the tool's ecological validity. A single discriminant function described primarily by four Do-Eat variables, correctly classified 95.5% of the study participants into their respective study groups, establishing the tool's predictive validity within this population. These preliminary findings indicate that the Do-Eat may serve as a reliable and valid tool that provides insight into the daily functioning characteristics of children with ADHD. However, further research on larger samples is indicated.

The validity and reliability of the four square step test in different adult populations: a systematic review.

PubMed

Moore, Martha; Barker, Karen

2017-09-11

The four square step test (FSST) was first validated in healthy older adults to provide a measure of dynamic standing balance and mobility. The FSST has since been used in a variety of patient populations. The purpose of this systematic review is to determine the validity and reliability of the FSST in these different adult patient populations. The literature search was conducted to highlight all the studies that measured validity and reliability of the FSST. Six electronic databases were searched including AMED, CINAHL, MEDLINE, PEDro, Web of Science and Google Scholar. Grey literature was also searched for any documents relevant to the review. Two independent reviewers carried out study selection and quality assessment. The methodological quality was assessed using the QUADAS-2 tool, which is a validated tool for the quality assessment of diagnostic accuracy studies, and the COSMIN four-point checklist, which contains standards for evaluating reliability studies on the measurement properties of health instruments. Fifteen studies were reviewed studying community-dwelling older adults, Parkinson's disease, Huntington's disease, multiple sclerosis, vestibular disorders, post stroke, post unilateral transtibial amputation, knee pain and hip osteoarthritis. Three of the studies were of moderate methodological quality scoring low in risk of bias and applicability for all domains in the QUADAS-2 tool. Three studies scored "fair" on the COSMIN four-point checklist for the reliability components. The concurrent validity of the FSST was measured in nine of the studies with moderate to strong correlations being found. Excellent Intraclass Correlation Coefficients were found between physiotherapists carrying out the tests (ICC = .99) with good to excellent test-retest reliability shown in nine of the studies (ICC = .73-.98). The FSST may be an effective and valid tool for measuring dynamic balance and a participants' falls risk. It has been shown to have strong correlations with other measures of balance and mobility with good reliability shown in a number of populations. However, the quality of the papers reviewed was variable with key factors, such as sample size and test set up, needing to be addressed before the tool can be confidently used in these specified populations.

Developing the Polish Educational Needs Assessment Tool (Pol-ENAT) in rheumatoid arthritis and systemic sclerosis: a cross-cultural validation study using Rasch analysis.

PubMed

Sierakowska, Matylda; Sierakowski, Stanisław; Sierakowska, Justyna; Horton, Mike; Ndosi, Mwidimi

2015-03-01

To undertake cross-cultural adaptation and validation of the educational needs assessment tool (ENAT) for use with people with rheumatoid arthritis (RA) and systemic sclerosis (SSc) in Poland. The study involved two main phases: (1) cross-cultural adaptation of the ENAT from English into Polish and (2) Cross-cultural validation of Polish Educational Needs Assessment Tool (Pol-ENAT). The first phase followed an established process of cross-cultural adaptation of self-report measures. The second phase involved completion of the Pol-ENAT by patients and subjecting the data to Rasch analysis to assess the construct validity, unidimensionality, internal consistency and cross-cultural invariance. An adequate conceptual equivalence was achieved following the adaptation process. The dataset for validation comprised a total of 278 patients, 237 (85.3 %) of which were female. In each disease group (145, RA and 133, SSc), the 7 domains of the Pol-ENAT were found to fit the Rasch model, X (2)(df) = 16.953(14), p = 0.259 and 8.132(14), p = 0.882 for RA and SSc, respectively. Internal consistency of the Pol-ENAT was high (patient separation index = 0.85 and 0.89 for SSc and RA, respectively), and unidimensionality was confirmed. Cross-cultural differential item functioning (DIF) was detected in some subscales, and DIF-adjusted conversion tables were calibrated to enable cross-cultural comparison of data between Poland and the UK. Using a standard process in cross-cultural adaptation, conceptual equivalence was achieved between the original (UK) ENAT and the adapted Pol-ENAT. Fit to the Rasch model, confirmed that the construct validity, unidimensionality and internal consistency of the ENAT have been preserved.

Reliability and validity assessment of gastrointestinal dystemperaments questionnaire: a novel scale in Persian traditional medicine

PubMed Central

Hoseinzadeh, Hamidreza; Taghipour, Ali; Yousefi, Mahdi

2018-01-01

Background Development of a questionnaire based on the resources of Persian traditional medicine seems necessary. One of the problems faced by practitioners of traditional medicine is the different opinions regarding the diagnosis of general temperament or temperament of member. One of the reasons is the lack of validity tools, and it has led to difficulties in training the student of traditional medicine and the treatment of patients. The differences in the detection methods, have given rise to several treatment methods. Objective The present study aimed to develop a questionnaire and standard software for diagnosis of gastrointestinal dystemperaments. Methods The present research is a tool developing study which included 8 stages of developing the items, determining the statements based on items, assessing the face validity, assessing the content validity, assessing the reliability, rating the items, developing a software for calculation of the total score of the questionnaire named GDS v.1.1, and evaluating the concurrent validity using statistical tests including Cronbach’s alpha coefficient, Cohen’s kappa coefficient. Results Based on the results, 112 notes including 62 symptoms were extracted from resources, and 58 items were obtained from in-person interview sessions with a panel of experts. A statement was selected for each item and, after merging a number of statements, a total of 49 statements were finally obtained. By calculating the score of statement impact and determining the content validity, respectively, 6 and 10 other items were removed from the list of statements. Standardized Cronbach’s alpha for this questionnaire was obtained 0.795 and its concurrent validity was equal to 0.8. Conclusion A quantitative tool was developed for diagnosis and examination of gastrointestinal dystemperaments. The developed questionnaire is adequately reliable and valid for this purpose. In addition, the software can be used for clinical diagnosis. PMID:29629060

[Quality of the pharmacotherapeutic recommendations for the integrated care procedures in Andalusia].

PubMed

Corte, Rosa María Muñoz; Estepa, Raúl García; Ramos, Bernardo Santos; Paloma, Francisco Javier Bautista

2009-01-01

To evaluate the quality of the pharmacotherapeutic recommendations included in the Integrated Care Procedures (PAIs regarding its initials in Spanish) of the Andalusian Ministry of Health, published up to March 2008, through the design and validation of a tool. The assessment tool was designed based on similar instruments, specifically the AGREE. Other criteria included were taken from various literature sources or were devised by ourselves. The tool was validated prior to being used. After applying it to all the PAIs, we examined the degree of compliance with these pharmacotherapeutic criteria, both as a whole and by PAIs subgroups. The developed tool is a questionnaire of 20 items, divided into 4 sections. The first section consists of the essential criteria, and the rest make reference to more specific, non essential criteria: definition of the level of evidence, thoroughness of information and definition of indicators. It was found that 4 of the 60 PAIs do not contain any type of therapeutic recommendation. No PAI fulfils all the items listed in the tool, however, 70 % of them fulfil the essential quality criteria established. There is a great variability in the content of pharmacotherapeutic recommendations for each PAI. Once the validity of the tool has been proved, it could be used to assess the quality of the therapeutic recommendations in clinical practice guidelines.

The development and validation of the clinicians' awareness towards cognitive errors (CATChES) in clinical decision making questionnaire tool.

PubMed

Chew, Keng Sheng; Kueh, Yee Cheng; Abdul Aziz, Adlihafizi

2017-03-21

Despite their importance on diagnostic accuracy, there is a paucity of literature on questionnaire tools to assess clinicians' awareness toward cognitive errors. A validation study was conducted to develop a questionnaire tool to evaluate the Clinician's Awareness Towards Cognitive Errors (CATChES) in clinical decision making. This questionnaire is divided into two parts. Part A is to evaluate the clinicians' awareness towards cognitive errors in clinical decision making while Part B is to evaluate their perception towards specific cognitive errors. Content validation for both parts was first determined followed by construct validation for Part A. Construct validation for Part B was not determined as the responses were set in a dichotomous format. For content validation, all items in both Part A and Part B were rated as "excellent" in terms of their relevance in clinical settings. For construct validation using exploratory factor analysis (EFA) for Part A, a two-factor model with total variance extraction of 60% was determined. Two items were deleted. Then, the EFA was repeated showing that all factor loadings are above the cut-off value of >0.5. The Cronbach's alpha for both factors are above 0.6. The CATChES questionnaire tool is a valid questionnaire tool aimed to evaluate the awareness among clinicians toward cognitive errors in clinical decision making.

Validation of a method for assessing resident physicians' quality improvement proposals.

PubMed

Leenstra, James L; Beckman, Thomas J; Reed, Darcy A; Mundell, William C; Thomas, Kris G; Krajicek, Bryan J; Cha, Stephen S; Kolars, Joseph C; McDonald, Furman S

2007-09-01

Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. Seven raters used the instrument to assess 45 resident QI proposals. Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

Linguistic validation of the Alberta Context Tool and two measures of research use, for German residential long term care.

PubMed

Hoben, Matthias; Bär, Marion; Mahler, Cornelia; Berger, Sarah; Squires, Janet E; Estabrooks, Carole A; Kruse, Andreas; Behrens, Johann

2014-01-31

To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks' Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity-an initial stage of validity, necessary before more advanced validity assessment. We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants' answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants. Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups. Cognitive debriefing is essential to detect and respond to problematic instrument items, particularly when translating instruments for heterogeneous, less well educated provider groups such as HCAs. Cognitive debriefing is an important step in research tool development and a vital component of establishing response process validity evidence. Publishing cognitive debriefing results helps researchers to determine potentially critical elements of the translated tools and assists with interpreting scores.

Nutritional status and nutrition risk screening in hospitalized children in New Zealand.

PubMed

Moeeni, Vesal; Walls, Tony; Day, Andrew S

2013-09-01

Children requiring hospitalization are at risk of malnutrition. This study aimed to define the nutritional status of paediatric inpatients in comparison with healthy children and to compare and contrast the feasibility and validity of three nutritional risk screening (NRS) tools in the hospitalized children. A total of 162 children admitted to Christchurch Hospital were assessed along with a similar group of healthy children. Their nutritional state was assessed and classified using standard criteria. The NRS tools were applied, and patients were classified into low-, medium- and high-risk groups. The feasibility and validity of the tools were assessed. Under-nutrition was more frequent in the inpatient group (9.9% vs. 3.7%; p = 0.04), whereas both groups had similar rates of overweight/obesity. NRS tools were able to identify between 81% and 100% of the malnourished patients in the medium- to high-risk groups. Undernourished patients had longer hospital stay than well-nourished patients. Hospitalized children have higher rates of under-nutrition than healthy children in NZ. The three NRS tools were able to identify children at nutritional risk with differing utility. In this setting, STRONGkids was the most reliable tool. ©2013 Foundation Acta Paediatrica. Published by John Wiley & Sons Ltd.

Development and evaluation of a patient-centred measurement tool for surgeons' non-technical skills.

PubMed

Yule, J; Hill, K; Yule, S

2018-06-01

Non-technical skills are essential for safe and effective surgery. Several tools to assess surgeons' non-technical skills from the clinician's perspective have been developed. However, a reliable measurement tool using a patient-centred approach does not currently exist. The aim of this study was to translate the existing Non-Technical Skills for Surgeons (NOTSS) tool into a patient-centred evaluation tool. Data were gathered from four cohorts of patients using an iterative four-stage mixed-methods research design. Exploratory and confirmatory factor analyses were performed to establish the psychometric properties of the tool, focusing on validity, reliability, usability and parsimony. Some 534 patients were recruited to the study. A total of 24 patient-centred non-technical skill items were developed in stage 1, and reduced to nine items in stage 2 using exploratory factor analysis. In stage 3, confirmatory factor analysis demonstrated that these nine items each loaded on to one of three factors, with excellent internal consistency: decision-making, leadership, and communication and teamwork. In stage 4, validity testing established that the new tool was independent of physician empathy and predictive of surgical quality. Surgical leadership emerged as the most dominant skill that patients could recognize and evaluate. A novel nine-item assessment tool has been developed. The Patients' Evaluation of Non-Technical Skills (PENTS) tool allows valid and reliable measurement of surgeons' non-technical skills from the patient perspective. © 2018 BJS Society Ltd Published by John Wiley & Sons Ltd.

Development and validation of risk models and molecular diagnostics to permit personalized management of cancer.

PubMed

Pu, Xia; Ye, Yuanqing; Wu, Xifeng

2014-01-01

Despite the advances made in cancer management over the past few decades, improvements in cancer diagnosis and prognosis are still poor, highlighting the need for individualized strategies. Toward this goal, risk prediction models and molecular diagnostic tools have been developed, tailoring each step of risk assessment from diagnosis to treatment and clinical outcomes based on the individual's clinical, epidemiological, and molecular profiles. These approaches hold increasing promise for delivering a new paradigm to maximize the efficiency of cancer surveillance and efficacy of treatment. However, they require stringent study design, methodology development, comprehensive assessment of biomarkers and risk factors, and extensive validation to ensure their overall usefulness for clinical translation. In the current study, the authors conducted a systematic review using breast cancer as an example and provide general guidelines for risk prediction models and molecular diagnostic tools, including development, assessment, and validation. © 2013 American Cancer Society.

The Desired Concept Maps and Goal Setting for Assessing Professionalism in Medicine.

PubMed

Guraya, Salman Y; Guraya, Shaista S; Mahabbat, Nehal Anam; Fallatah, Khulood Yahya; Al-Ahmadi, Bashaer Ahmad; Alalawi, Hadeel Hadi

2016-05-01

Due to the multi-dimensional characteristics of professionalism, no single assessment modality has shown to reliably assess professionalism. This review aims to describe some of the popular assessment tools that are being used to assess professionalism with a view to formulate a framework of assessment of professionalism in medicine. In December 2015, the online research databases of MEDLINE, the Educational Resources Information Center (ERIC), Elton Bryson Stephens Company (EBSCO), SCOPUS, OVID and PsychINFO were searched for full-text English language articles published during 2000 to 2015. MeSH terms "professionalism" AND "duty" AND "assessment" OR "professionalism behavioural" AND "professionalism-cognitive" were used. The research articles that assessed professionalism across medical fields along with other areas of competencies were included. A final list of 35 articles were selected for this review. Several assessment tools are available for assessing professionalism that includes, but not limited to, mini clinical evaluation exercise, standardised direct observation of procedural skills, professionalism mini-evaluation exercise, multi-source feedback and 360 degree evaluation, and case based discussions. Because professionalism is a complex construct, it is less likely that a single assessment strategy will adequately measure it. Since every single assessment tool has its own weaknesses, triangulation involving multiple tools can compensate the shortcomings associated with any single approach. Assessment of professionalism necessitates a combination of modalities at individual, interpersonal, societal, and institutional levels and should be accompanied by feedback and motivational reflection that will, in turn, lead to behaviour and identity formation. The assessment of professionalism in medicine should meet the criteria of validity, reliability, feasibility and acceptability. Educators are urged to enhance the depth and quality of assessment instruments in the existing medical curricula for ensuring validity and reliability of assessment tools for professionalism.

Developing a Scale for Innovation Management at Schools: A Study of Validity and Reliability

ERIC Educational Resources Information Center

Bulbul, Tuncer

2012-01-01

The purpose of this study is to develop a valid and reliable assessment tool for use in determining the competency beliefs of school administrators about innovation management. The scale applied to a study group of 216 school administrators, after work Centered on assessing intelligibility and specialized opinion. Exploratory and confirmatory…

The Professionalism Assessment of Clinical Teachers (PACT): The Reliability and Validity of a Novel Tool to Evaluate Professional and Clinical Teaching Behaviors

ERIC Educational Resources Information Center

Young, Meredith E.; Cruess, Sylvia R.; Cruess, Richard L.; Steinert, Yvonne

2014-01-01

Physicians function as clinicians, teachers, and role models within the clinical environment. Negative learning environments have been shown to be due to many factors, including the presence of unprofessional behaviors among clinical teachers. Reliable and valid assessments of clinical teacher performance, including professional behaviors, may…

Developing and Verifying a Business-Creativity Assessment Tool: A Nationwide Study in Taiwan

ERIC Educational Resources Information Center

Cheng, Kai-Wen; Chen, Yu-Fen

2010-01-01

This study was designed to assess business creativity of students in Taiwan. A questionnaire was developed and administered to students in business-related departments of 16 vocational schools during November 2006. Of the 1,420 administered questionnaires, 1,052 were valid, resulting in a valid response rate of 70.48%. Exploratory factor analysis…

Objective assessment of technique in laparoscopic colorectal surgery: what are the existing tools?

PubMed

Foster, J D; Francis, N K

2015-01-01

Assessment can improve the effectiveness of surgical training and enable valid judgments of competence. Laparoscopic colon resection surgery is now taught within surgical residency programs, and assessment tools are increasingly used to stimulate formative feedback and enhance learning. Formal assessment of technical performance in laparoscopic colon resection has been successfully applied at the specialist level in the English "LAPCO" National Training Program. Objective assessment tools need to be developed for training and assessment in laparoscopic rectal cancer resection surgery. Simulation may have a future role in assessment and accreditation in laparoscopic colorectal surgery; however, existing virtual reality models are not ready to be used for assessment of this advanced surgery.

Validating workplace performance assessments in health sciences students: a case study from speech pathology.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Allison; McAllister, Lindy

2013-01-01

Valid assessment of health science students' ability to perform in the real world of workplace practice is critical for promoting quality learning and ultimately certifying students as fit to enter the world of professional practice. Current practice in performance assessment in the health sciences field has been hampered by multiple issues regarding assessment content and process. Evidence for the validity of scores derived from assessment tools are usually evaluated against traditional validity categories with reliability evidence privileged over validity, resulting in the paradoxical effect of compromising the assessment validity and learning processes the assessments seek to promote. Furthermore, the dominant statistical approaches used to validate scores from these assessments fall under the umbrella of classical test theory approaches. This paper reports on the successful national development and validation of measures derived from an assessment of Australian speech pathology students' performance in the workplace. Validation of these measures considered each of Messick's interrelated validity evidence categories and included using evidence generated through Rasch analyses to support score interpretation and related action. This research demonstrated that it is possible to develop an assessment of real, complex, work based performance of speech pathology students, that generates valid measures without compromising the learning processes the assessment seeks to promote. The process described provides a model for other health professional education programs to trial.

The Readiness to Change for Bariatric Surgery Assessment Tool: Validity, Factor Structure, and Reliability.

PubMed

Carlson, Susan M S

2017-11-01

Currently, there is no guideline or standard of practice for performing the psychiatric/psychological evaluation that is a requirement for approval for bariatric surgery. The Readiness to Change for Bariatric Surgery Assessment Tool (RCB-SAT) establishes a means for psychiatric evaluators to objectively assess the patient's cognition, beliefs, and motivation around the bariatric diet and lifestyle changes. Development of a clinical decision-making tool for assessing readiness to change in bariatric patients will be useful regarding The Strategic Plan for NIH Obesity Research. The strategic plan outlines 6 overarching themes, with the last 3 centering around creation of such a clinical decision-making tool to assess a bariatric patient's readiness to change: evaluate promising strategies for obesity prevention and treatment in realworld settings and diverse populations, harness technology and tools to advance obesity research and improve health care delivery, and facilitate integration of research results into community programs and medical practice (National Institutes of Health, 2011). The pilot tool was administered to 153 potential bariatric patients, with 61 patients completing the survey a second time. Face and content validity of the items were established through an expert review process. Principle axis factoring by means of varimax rotation with Kaiser normalization identified 15 items loading on 3 factors associated with Prochaska and DiClemente's transtheoretical model of health behavior change: precontemplation, contemplation, and action (DiClemente & Prochaska, 1998). Test-retest reliability was also established for the tool. The proposed RCB-SAT demonstrates potential for assessing a patient's readiness to change regarding the bariatric diet and lifestyle.

NASA Electronic Parts and Packaging Program

NASA Technical Reports Server (NTRS)

Kayali, Sammy

2000-01-01

NEPP program objectives are to: (1) Access the reliability of newly available electronic parts and packaging technologies for usage on NASA projects through validations, assessments, and characterizations, and the development of test methods/tools; (2)Expedite infusion paths for advanced (emerging) electronic parts and packaging technologies by evaluations of readiness for manufacturability and project usage consideration; (3) Provide NASA projects with technology selection, application, and validation guidelines for electronic parts and packaging hardware and processes; nd (4) Retain and disseminate electronic parts and packaging quality assurance, reliability validations, tools, and availability information to the NASA community.

The Chinese version of monitoring and evaluation system strengthening tool for human immunodeficiency virus (HIV) capacity building: Development and evaluation.

PubMed

Zhao, Ran; Chen, Ren; Zhang, Bing; Ma, Ying; Qin, Xia; Hu, Zhi

2015-08-01

Monitoring and evaluation (M&E) for human immunodeficiency virus (HIV) capacity building has become a significant step for HIV prevention and control. The M&E system strengthening tool published by the United Nations Joint Programme on HIV/AIDS (UNAIDS) was intended to be the most authoritative assessment tool internationally. Facing the fact that the M&E system in China did not function at an optimum level, we considered taking the international standards for reference. By linguistic validating and different stages' discussions and revisions, we came up with the Chinese version of the capacity diagnosis tool with at least 12 components and tested its validity and reliability. The tool turned out to have a sufficiently linguistic validation and proved to be a scientific and feasible instrument which was suitable for China's national conditions.

Validation and Cultural Adaptation of the Arabic Version of the Eating Assessment Tool (EAT-10).

PubMed

Farahat, Mohamed; Mesallam, Tamer A

2015-01-01

The Eating Assessment Tool (EAT-10) is a 10-item self-administered questionnaire. It is a noninvasive tool to measure patients' perception of their swallowing problems. The purposes of the present study were to develop an Arabic version of the EAT-10 and to evaluate its validity, consistency, and reliability in the Arabic-speaking population with oropharyngeal dysphagia. This was a prospective study carried out at the Communication and Swallowing Disorders Unit, King Saud University, Riyadh, Saudi Arabia. The Arabic EAT-10 was administered to 138 patients with oropharyngeal dysphagia and 83 control subjects. Internal consistency and test-retest reliability were evaluated. Content and clinical validity were studied, and the EAT-10 results were compared across patients and control groups. The Arabic EAT-10 showed excellent internal consistency (Cronbach's α = 0.92). Also, good test-retest reliability was found for the total scores of the Arabic EAT-10 (intraclass correlation = 0.73). There was a significant difference in Arabic EAT-10 scores between the oropharyngeal dysphagia group and the control group (p < 0.001). This study demonstrated that the Arabic EAT-10 is a valid tool that can be used for screening of dysphagia-related problems in an Arabic-speaking population. © 2016 S. Karger AG, Basel.

Reliability and Validity of the Korean Cancer Pain Assessment Tool (KCPAT)

PubMed Central

Kim, Jeong A; Lee, Juneyoung; Park, Jeanno; Lee, Myung Ah; Yeom, Chang Hwan; Jang, Se Kwon; Yoon, Duck Mi; Kim, Jun Suk

2005-01-01

The Korean Cancer Pain Assessment Tool (KCPAT), which was developed in 2003, consists of questions concerning the location of pain, the nature of pain, the present pain intensity, the symptoms associated with the pain, and psychosocial/spiritual pain assessments. This study was carried out to evaluate the reliability and validity of the KCPAT. A stratified, proportional-quota, clustered, systematic sampling procedure was used. The study population (903 cancer patients) was 1% of the target population (90,252 cancer patients). A total of 314 (34.8%) questionnaires were collected. The results showed that the average pain score (5 point on Likert scale) according to the cancer type and the at-present average pain score (VAS, 0-10) were correlated (r=0.56, p<0.0001), and showed moderate agreement (kappa=0.364). The mean satisfaction score was 3.8 (1-5). The average time to complete the questionnaire was 8.9 min. In conclusion, the KCPAT is a reliable and valid instrument for assessing cancer pain in Koreans. PMID:16224166

Validation of an explanatory tool for data-fused displays for high-technology future aircraft

NASA Astrophysics Data System (ADS)

Fletcher, Georgina C. L.; Shanks, Craig R.; Selcon, Stephen J.

1996-05-01

As the number of sensor and data sources in the military cockpit increases, pilots will suffer high levels of workload which could result in reduced performance and the loss of situational awareness. A DRA research program has been investigating the use of data-fused displays in decision support and has developed and laboratory-tested an explanatory tool for displaying information in air combat scenarios. The tool has been designed to provide pictorial explanations of data that maintain situational awareness by involving the pilot in the hostile aircraft threat assessment task. This paper reports a study carried out to validate the success of the explanatory tool in a realistic flight simulation facility. Aircrew were asked to perform a threat assessment task, either with or without the explanatory tool providing information in the form of missile launch success zone envelopes, while concurrently flying a waypoint course within set flight parameters. The results showed that there was a significant improvement (p less than 0.01) in threat assessment accuracy of 30% when using the explanatory tool. This threat assessment performance advantage was achieved without a trade-off with flying task performance. Situational awareness measures showed no general differences between the explanatory and control conditions, but significant learning effects suggested that the explanatory tool makes the task initially more intuitive and hence less demanding on the pilots' attentional resources. The paper concludes that DRA's data-fused explanatory tool is successful at improving threat assessment accuracy in a realistic simulated flying environment, and briefly discusses the requirements for further research in the area.

Measuring critical thinking in pre-registration midwifery students: A multi-method approach.

PubMed

Carter, Amanda G; Creedy, Debra K; Sidebotham, Mary

2018-02-01

Test the concurrent validity of three newly developed tools (student self-rating, preceptor rating, and reflective writing) that aim to measure critical thinking in midwifery practice. A descriptive matched cohort design was used. Australian research intensive university offering a three year Bachelor of Midwifery programme. Fifty-five undergraduate midwifery students. Students assessed their ability to apply critical thinking in midwifery practice using a 25-item tool and a 5-item subscale in Motivated Strategies for Learning Questionnaire. Clinical preceptors completed a 24-item tool assessing the students' application of critical thinking in practice. Reflective writing by students was assessed by midwifery academics using a 15-item tool. Internal reliability, and concurrent validity were assessed. Correlations, t-tests, multiple regression and confidence levels were calculated for the three scales and associations with student characteristics. The three scales achieved good internal reliability with a Cronbach's alpha coefficient between 0.93 and 0.97. Matched total scores for the three critical thinking scales were moderately correlated; student/preceptor (r=0.36, p<0.01); student/reflective writing (r=0.38, p<0.01); preceptor/reflective writing (r=0.30, p<0.05). All critical thinking mean scores were higher for students with a previous degree, but only significant for reflective writing (t (53)=-2.35, p=0.023). Preceptor ratings were predictive of GPA (beta=0.50, p<0.001, CI=0.10 to 0.30). Students' self-rating scores were predictive of year level (beta=0.32, p<0.05, CI=0.00 to 0.03). The student, preceptor, and reflective writing tools were found to be reliable and valid measures of critical thinking. The three tools can be used individually or in combination to provide students with various sources of feedback to improve their practice. The tools allow formative measurement of critical thinking over time. Further testing of the tools with larger, diverse samples is recommended. Crown Copyright © 2017. Published by Elsevier Ltd. All rights reserved.

The Desired Concept Maps and Goal Setting for Assessing Professionalism in Medicine

PubMed Central

Guraya, Shaista S.; Mahabbat, Nehal Anam; Fallatah, Khulood Yahya; Al-Ahmadi, Bashaer Ahmad; Alalawi, Hadeel Hadi

2016-01-01

Due to the multi-dimensional characteristics of professionalism, no single assessment modality has shown to reliably assess professionalism. This review aims to describe some of the popular assessment tools that are being used to assess professionalism with a view to formulate a framework of assessment of professionalism in medicine. In December 2015, the online research databases of MEDLINE, the Educational Resources Information Center (ERIC), Elton Bryson Stephens Company (EBSCO), SCOPUS, OVID and PsychINFO were searched for full-text English language articles published during 2000 to 2015. MeSH terms “professionalism” AND “duty” AND “assessment” OR “professionalism behavioural” AND “professionalism–cognitive” were used. The research articles that assessed professionalism across medical fields along with other areas of competencies were included. A final list of 35 articles were selected for this review. Several assessment tools are available for assessing professionalism that includes, but not limited to, mini clinical evaluation exercise, standardised direct observation of procedural skills, professionalism mini-evaluation exercise, multi-source feedback and 360 degree evaluation, and case based discussions. Because professionalism is a complex construct, it is less likely that a single assessment strategy will adequately measure it. Since every single assessment tool has its own weaknesses, triangulation involving multiple tools can compensate the shortcomings associated with any single approach. Assessment of professionalism necessitates a combination of modalities at individual, interpersonal, societal, and institutional levels and should be accompanied by feedback and motivational reflection that will, in turn, lead to behaviour and identity formation. The assessment of professionalism in medicine should meet the criteria of validity, reliability, feasibility and acceptability. Educators are urged to enhance the depth and quality of assessment instruments in the existing medical curricula for ensuring validity and reliability of assessment tools for professionalism. PMID:27437247

Validity and reliability of the Turkish version of the Optimality Index-US (OI-US) to assess maternity care outcomes.

PubMed

Yucel, Cigdem; Taskin, Lale; Low, Lisa Kane

2015-12-01

Although obstetrical interventions are used commonly in Turkey, there is no standardized evidence-based assessment tool to evaluate maternity care outcomes. The Optimality Index-US (OI-US) is an evidence-based tool that was developed for the purpose of measuring aggregate perinatal care processes and outcomes against an optimal or best possible standard. This index has been validated and used in Netherlands, USA and UK until now. The objective of this study was to adapt the OI-US to assess maternity care outcomes in Turkey. Translation and back translation were used to develop the Optimality Index-Turkey (OI-TR) version. To evaluate the content validity of the OI-TR, an expert panel group (n=10) reviewed the items and evidence-based quality of the OI-TR for application in Turkey. Following the content validity process, the OI-TR was used to assess 150 healthy and 150 high-risk pregnant women who gave birth at a high volume, urban maternity hospital in Turkey. The scores between the two groups were compared to assess the discriminant validity of the OI-TR. The percentage of agreement between two raters and the Kappa statistic were calculated to evaluate the reliability. Content validity was established for the OI-TR by an expert group. Discriminant validity was confirmed by comparing the OI scores of healthy pregnant women (mean OI score=77.65%) and those of high-risk pregnant women (mean OI score=78.60%). The percentage of agreement between the two raters was 96.19, and inter-rater agreement was provided for each item in the OI-TR. OI-TR is a valid and reliable tool that can be used to assess maternity care outcomes in Turkey. The results of this study indicate that although the risk statuses of the women differed, the type of care they received was essentially the same, as measured by the OI-TR. Care was not individualised based on risk and for a majority of items was inconsistent with evidence based practice, which is not optimal. Use of the OI-TR will help to provide a standardized way to assess maternity care process and outcomes of maternity care in Turkey which can inform future research aimed at improving maternity care outcomes. Copyright © 2015 Elsevier Ltd. All rights reserved.

Diagnostic Tools for Performance Evaluation of Innovative In-Situ Remediation Technologies at Chlorinated Solvent-Contaminated Sites

DTIC Science & Technology

2011-07-01

to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT...these innovative methods with conventional diagnostic tools that are currently used for assessing bioremediation performance. 132 Rula Deeb (510) 596...conventional diagnostic tools that are currently used for assessing bioremediation performance. DEMONSTRATION RESULTS 3-D multi-level systems

Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.

PubMed

Koehler, Ryan J; Nicandri, Gregg T

2013-12-04

Examination of arthroscopic skill requires evaluation tools that are valid and reliable with clear criteria for passing. The Arthroscopic Surgery Skill Evaluation Tool was developed as a video-based assessment of technical skill with criteria for passing established by a panel of experts. The purpose of this study was to test the validity and reliability of the Arthroscopic Surgery Skill Evaluation Tool as a pass-fail examination of arthroscopic skill. Twenty-eight residents and two sports medicine faculty members were recorded performing diagnostic knee arthroscopy on a left and right cadaveric specimen in our arthroscopic skills laboratory. Procedure videos were evaluated with use of the Arthroscopic Surgery Skill Evaluation Tool by two raters blind to subject identity. Subjects were considered to pass the Arthroscopic Surgery Skill Evaluation Tool when they attained scores of ≥ 3 on all eight assessment domains. The raters agreed on a pass-fail rating for fifty-five of sixty videos rated with an interclass correlation coefficient value of 0.83. Ten of thirty participants were assigned passing scores by both raters for both diagnostic arthroscopies performed in the laboratory. Receiver operating characteristic analysis demonstrated that logging more than eighty arthroscopic cases or performing more than thirty-five arthroscopic knee cases was predictive of attaining a passing Arthroscopic Surgery Skill Evaluation Tool score on both procedures performed in the laboratory. The Arthroscopic Surgery Skill Evaluation Tool is valid and reliable as a pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory. This study demonstrates that the Arthroscopic Surgery Skill Evaluation Tool may be a useful tool for pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory. Further study is necessary to determine whether the Arthroscopic Surgery Skill Evaluation Tool can be used for the assessment of multiple arthroscopic procedures and whether it can be used to evaluate arthroscopic procedures performed in the operating room.

CROSS-CULTURAL ADAPTATION AND VALIDATION OF THE KOREAN VERSION OF THE CUMBERLAND ANKLE INSTABILITY TOOL.

PubMed

Ko, Jupil; Rosen, Adam B; Brown, Cathleen N

2015-12-01

The Cumberland Ankle Instability Tool (CAIT) is a valid and reliable patient reported outcome used to assess the presence and severity of chronic ankle instability (CAI). The CAIT has been cross-culturally adapted into other languages for use in non-English speaking populations. However, there are no valid questionnaires to assess CAI in individuals who speak Korean. The purpose of this study was to translate, cross-culturally adapt, and validate the CAIT, for use in a Korean-speaking population with CAI. Cross-cultural reliability study. The CAIT was cross-culturally adapted into Korean according to accepted guidelines and renamed the Cumberland Ankle Instability Tool-Korean (CAIT-K). Twenty-three participants (12 males, 11 females) who were bilingual in English and Korean were recruited and completed the original and adapted versions to assess agreement between versions. An additional 168 national level Korean athletes (106 male, 62 females; age = 20.3 ± 1.1 yrs), who participated in ≥ 90 minutes of physical activity per week, completed the final version of the CAIT-K twice within 14 days. Their completed questionnaires were assessed for internal consistency, test-retest reliability, criterion validity, and construct validity. For bilingual participants, intra-class correlation coefficients (ICC2,1) between the CAIT and the CAIT-K for test-retest reliability were 0.95 (SEM=1.83) and 0.96 (SEM=1.50) in right and left limbs, respectively. The Cronbach's alpha coefficients were 0.92 and 0.90 for the CAIT-K in right and left limbs, respectively. For native Korean speakers, the CAIT-K had high internal consistency (Cronbach's α=0.89) and intra-class correlation coefficient (ICC2,1 = 0.94, SEM=1.72), correlation with the physical component score (rho=0.70, p = 0.001) of the Short-Form Health Survey (SF-36), and the Kaiser-Meyer-Olkin score was 0.87. The original CAIT was translated, cross-culturally adapted, and validated from English to Korean. The CAIT-K appears to be valid and reliable and could be useful in assessing the Korean speaking population with CAI.

Learning to Write: Progress-Monitoring Tools for Beginning and at-Risk Writers

ERIC Educational Resources Information Center

Ritchey, Kristen D.

2006-01-01

Teachers now have a wide range of tools to help assess the beginning reading performance of kindergarten and first-grade children. However, validated procedures for assessing the beginning writing skills of kindergarten and first-grade children are less widely available. Learning to write, like learning to read, is a complex task. The ability to…

The Assessment of Afterschool Program Practices Tool (APT): Findings from the APT Validation Study

ERIC Educational Resources Information Center

Tracy, Allison; Surr, Wendy; Richer, Amanda

2012-01-01

The Assessment of Afterschool Program Practices Tool ("APT"), developed by the National Institute of Out-of-School Time (NIOST), is an observational instrument designed to measure the aspects of afterschool program quality that research suggests contribute to the 21st century skills, attitudes, and behaviors youth need to be successful…

A new comprehension and communication tool: a valuable resource for internationally educated occupational therapists.

PubMed

Nguyen, Tram; Baptiste, Sue; Jung, Bonny; Wilkins, Seanne

2014-06-01

The need was identified for a way to assess internationally educated occupational therapists’ skills in understanding and communicating professional terminology used in occupational therapy practice. The project aim was to develop and validate such a resource. A scenario-based assessment was developed using a three-phase process for tool development. The development process involved completion of a literature scan of professional terminology used in occupational therapy practice; selection of terms and concepts commonly used in occupational therapy practice; and, creation of practice-based scenarios illustrating key concepts complete with rating rubrics. An advisory group provided oversight, and a sample of internationally educated occupational therapists completed pilot and validity testing. The initial findings showed the assessment to be easy to complete and sensitive to testing understanding of the defined terms. The final outcome is an assessment tool that has broad application for occupational therapists wishing to enter professional practice in a new country. © 2013 Occupational Therapy Australia.

Identifying high quality medical education websites in Otolaryngology: a guide for medical students and residents.

PubMed

Yang, Nathan; Hosseini, Sarah; Mascarella, Marco A; Young, Meredith; Posel, Nancy; Fung, Kevin; Nguyen, Lily H P

2017-05-25

Learners often utilize online resources to supplement formalized curricula, and to appropriately support learning, these resources should be of high quality. Thus, the objectives of this study are to develop and provide validity evidence supporting an assessment tool designed to assess the quality of educational websites in Otolaryngology- Head & Neck Surgery (ORL-HNS), and identify those that could support effective web-based learning. METHODS: After a literature review, the Modified Education in Otolaryngology Website (MEOW) assessment tool was designed by a panel of experts based on a previously validated website assessment tool. A search strategy using a Google-based search engine was used subsequently to identify websites. Those that were free of charge and in English were included. Websites were coded for whether their content targeted medical students or residents. Using the MEOW assessment tool, two independent raters scored the websites. Inter-rater and intra-rater reliability were evaluated, and scores were compared to recommendations from a content expert. The MEOW assessment tool included a total of 20 items divided in 8 categories related to authorship, frequency of revision, content accuracy, interactivity, visual presentation, navigability, speed and recommended hyperlinks. A total of 43 out of 334 websites identified by the search met inclusion criteria. The scores generated by our tool appeared to differentiate higher quality websites from lower quality ones: websites that the expert "would recommend" scored 38.4 (out of 56; CI [34.4-42.4]) and "would not recommend" 27.0 (CI [23.2-30.9]). Inter-rater and intra-rater intraclass correlation coefficient were greater than 0.7. Using the MEOW assessment tool, high quality ORL-HNS educational websites were identified.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study.

PubMed

Schnall, Rebecca; Cho, Hwayoung; Liu, Jianfang

2018-01-05

Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. ©Rebecca Schnall, Hwayoung Cho, Jianfang Liu. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 05.01.2018.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study

PubMed Central

Cho, Hwayoung; Liu, Jianfang

2018-01-01

Background Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. Objective The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. Methods A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. Results The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. Conclusions The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. PMID:29305343

Evaluation of hand sensibility: a review.

PubMed

Novak, C B

2001-01-01

Many assessment devices and measures have been described to evaluate sensibility, with little consensus on the optimal measurement tool. The purpose of this paper is to review the assessment methods and devices used in the evaluation of hand sensibility. Consideration is given to the characteristics of each measurement tool, the information necessary for complete patient evaluation, and the battery of valid and reliable measurements that provide the most complete and accurate patient assessment.

Critical thinking evaluation in reflective writing: Development and testing of Carter Assessment of Critical Thinking in Midwifery (Reflection).

PubMed

Carter, Amanda G; Creedy, Debra K; Sidebotham, Mary

2017-11-01

develop and test a tool designed for use by academics to evaluate pre-registration midwifery students' critical thinking skills in reflective writing. a descriptive cohort design was used. a random sample (n = 100) of archived student reflective writings based on a clinical event or experience during 2014 and 2015. a staged model for tool development was used to develop a fifteen item scale involving item generation; mapping of draft items to critical thinking concepts and expert review to test content validity; inter-rater reliability testing; pilot testing of the tool on 100 reflective writings; and psychometric testing. Item scores were analysed for mean, range and standard deviation. Internal reliability, content and construct validity were assessed. expert review of the tool revealed a high content validity index score of 0.98. Using two independent raters to establish inter-rater reliability, good absolute agreement of 72% was achieved with a Kappa coefficient K = 0.43 (p<0.0001). Construct validity via exploratory factor analysis revealed three factors: analyses context, reasoned inquiry, and self-evaluation. The mean total score for the tool was 50.48 (SD = 12.86). Total and subscale scores correlated significantly. The scale achieved good internal reliability with a Cronbach's alpha coefficient of .93. this study establishedthe reliability and validity of the CACTiM (reflection) for use by academics to evaluate midwifery students' critical thinking in reflective writing. Validation with large diverse samples is warranted. reflective practice is a key learning and teaching strategy in undergraduate Bachelor of Midwifery programmes and essential for safe, competent practice. There is the potential to enhance critical thinking development by assessingreflective writing with the CACTiM (reflection) tool to provide formative and summative feedback to students and inform teaching strategies. Crown Copyright © 2017. Published by Elsevier Ltd. All rights reserved.

Spinal Cord Injury Pain Instrument and painDETECT questionnaire: Convergent construct validity in individuals with Spinal Cord Injury.

PubMed

Franz, S; Schuld, C; Wilder-Smith, E P; Heutehaus, L; Lang, S; Gantz, S; Schuh-Hofer, S; Treede, R-D; Bryce, T N; Wang, H; Weidner, N

2017-11-01

Neuropathic pain (NeuP) is a frequent sequel of spinal cord injury (SCI). The SCI Pain Instrument (SCIPI) was developed as a SCI-specific NeuP screening tool. A preliminary validation reported encouraging results requiring further evaluation in terms of psychometric properties. The painDETECT questionnaire (PDQ), a commonly applied NeuP assessment tool, was primarily validated in German, but not specifically developed for SCI and not yet validated according to current diagnostic guidelines. We aimed to provide convergent construct validity and to identify the optimal item combination for the SCIPI. The PDQ was re-evaluated according to current guidelines with respect to SCI-related NeuP. Prospective monocentric study. Subjects received a neurological examination according to the International Standards for Neurological Classification of SCI. After linguistic validation of the SCIPI, the IASP-grading system served as reference to diagnose NeuP, accompanied by the PDQ after its re-evaluation as binary classifier. Statistics were evaluated through ROC-analysis, with the area under the ROC curve (AUROC) as optimality criterion. The SCIPI was refined by systematic item permutation. Eighty-eight individuals were assessed with the German SCIPI. Of 127 possible combinations, a 4-item-SCIPI (cut-off-score = 1.5/sensitivity = 0.864/specificity = 0.839) was identified as most reasonable. The SCIPI showed a strong correlation (r sp  = 0.76) with PDQ. ROC-analysis of SCIPI/PDQ (AUROC = 0.877) revealed comparable results to SCIPI/IASP (AUROC = 0.916). ROC-analysis of PDQ/IASP delivered a score threshold of 10.5 (sensitivity = 0.727/specificity = 0.903). The SCIPI is a valid easy-to-apply NeuP screening tool in SCI. The PDQ is recommended as complementary NeuP assessment tool in SCI, e.g. to monitor pain severity and/or its time-dependent course. In SCI-related pain, both SCIPI and PainDETECT show strong convergent construct validity versus the current IASP-grading system. SCIPI is now optimized from a 7-item to an easy-to-apply 4-item screening tool in German and English. We provided evidence that the scope for PainDETECT can be expanded to individuals with SCI. © 2017 European Pain Federation - EFIC®.

Transcultural adaptation into Spanish of the Induction Compliance Checklist for assessing children's behaviour during induction of anaesthesia.

PubMed

Jerez-Molina, Carmen; Lázaro-Alcay, Juan J; Ullán-de la Fuente, Ana M

2017-10-17

Cross-cultural adaptation into Spanish of the Induction Compliance Checklist (ICC) for assessing children's behaviour during induction of anaesthesia. A descriptive cross-sectional observational study was conducted on a sample of 81 children aged 2 to 12 years operated in an ambulatory surgery unit of a paediatric hospital in Barcelona. Adaptation by translation-back translation of the tool and analysis of the scale's validity and reliability. Face validity of the tool was guaranteed through a discussion group and inter-observer reliability was evaluated, obtaining an intraclass correlation index of r = 0.956. The ICC scale validated for the Spanish population can be an effective tool for the presurgical evaluation of activities carried out to minimise children's anxiety. The ICC is an easy-to-use scale completed by operating room staff in one minute and would provide important information about children's behaviour, specifically during induction. Copyright © 2017 Elsevier España, S.L.U. All rights reserved.

Validation of the Mobile Information Software Evaluation Tool (MISET) With Nursing Students.

PubMed

Secco, M Loretta; Furlong, Karen E; Doyle, Glynda; Bailey, Judy

2016-07-01

This study evaluated the Mobile Information Software Evaluation Tool (MISET) with a sample of Canadian undergraduate nursing students (N = 240). Psychometric analyses determined how well the MISET assessed the extent that nursing students find mobile device-based information resources useful and supportive of learning in the clinical and classroom settings. The MISET has a valid three-factor structure with high explained variance (74.7%). Internal consistency reliabilities were high for the MISET total (.90) and three subscales: Usefulness/Helpfulness, Information Literacy Support, and Use of Evidence-Based Sources (.87 to .94). Construct validity evidence included significantly higher mean total MISET, Helpfulness/Usefulness, and Information Literacy Support scores for senior students and those with higher computer competence. The MISET is a promising tool to evaluate mobile information technologies and information literacy support; however, longitudinal assessment of changes in scores over time would determine scale sensitivity and responsiveness. [J Nurs Educ. 2016;55(7):385-390.]. Copyright 2016, SLACK Incorporated.

Development of a quality assessment tool for systematic reviews of observational studies (QATSO) of HIV prevalence in men having sex with men and associated risk behaviours

PubMed Central

Wong, William CW; Cheung, Catherine SK; Hart, Graham J

2008-01-01

Background Systematic reviews based on the critical appraisal of observational and analytic studies on HIV prevalence and risk factors for HIV transmission among men having sex with men are very useful for health care decisions and planning. Such appraisal is particularly difficult, however, as the quality assessment tools available for use with observational and analytic studies are poorly established. Methods We reviewed the existing quality assessment tools for systematic reviews of observational studies and developed a concise quality assessment checklist to help standardise decisions regarding the quality of studies, with careful consideration of issues such as external and internal validity. Results A pilot version of the checklist was developed based on epidemiological principles, reviews of study designs, and existing checklists for the assessment of observational studies. The Quality Assessment Tool for Systematic Reviews of Observational Studies (QATSO) Score consists of five items: External validity (1 item), reporting (2 items), bias (1 item) and confounding factors (1 item). Expert opinions were sought and it was tested on manuscripts that fulfil the inclusion criteria of a systematic review. Like all assessment scales, QATSO may oversimplify and generalise information yet it is inclusive, simple and practical to use, and allows comparability between papers. Conclusion A specific tool that allows researchers to appraise and guide study quality of observational studies is developed and can be modified for similar studies in the future. PMID:19014686

Assessing children’s competence to consent in research by a standardized tool: a validity study

PubMed Central

2012-01-01

Background Currently over 50% of drugs prescribed to children have not been evaluated properly for use in their age group. One key reason why children have been excluded from clinical trials is that they are not considered able to exercise meaningful autonomy over the decision to participate. Dutch law states that competence to consent can be presumed present at the age of 12 and above; however, in pediatric practice children’s competence is not that clearly presented and the transition from assent to active consent is gradual. A gold standard for competence assessment in children does not exist. In this article we describe a study protocol on the development of a standardized tool for assessing competence to consent in research in children and adolescents. Methods/design In this study we modified the MacCAT-CR, the best evaluated competence assessment tool for adults, for use in children and adolescents. We will administer the tool prospectively to a cohort of pediatric patients from 6 to18 years during the selection stages of ongoing clinical trials. The outcomes of the MacCAT-CR interviews will be compared to a reference standard, established by the judgments of clinical investigators, and an expert panel consisting of child psychiatrists, child psychologists and medical ethicists. The reliability, criterion-related validity and reproducibility of the tool will be determined. As MacCAT-CR is a multi-item scale consisting of 13 items, power was justified at 130–190 subjects, providing a minimum of 10–15 observations per item. MacCAT-CR outcomes will be correlated with age, life experience, IQ, ethnicity, socio-economic status and competence judgment of the parent(s). It is anticipated that 160 participants will be recruited over 2 years to complete enrollment. Discussion A validity study on an assessment tool of competence to consent is strongly needed in research practice, particularly in the child and adolescent population. In this study we will establish a reference standard of children’s competence to consent, combined with validation of an assessment instrument. Results can facilitate responsible involvement of children in clinical trials by further development of guidelines, health-care policies and legal policies. PMID:23009102

Reliability and concurrent validity of a Smartphone, bubble inclinometer and motion analysis system for measurement of hip joint range of motion.

PubMed

Charlton, Paula C; Mentiplay, Benjamin F; Pua, Yong-Hao; Clark, Ross A

2015-05-01

Traditional methods of assessing joint range of motion (ROM) involve specialized tools that may not be widely available to clinicians. This study assesses the reliability and validity of a custom Smartphone application for assessing hip joint range of motion. Intra-tester reliability with concurrent validity. Passive hip joint range of motion was recorded for seven different movements in 20 males on two separate occasions. Data from a Smartphone, bubble inclinometer and a three dimensional motion analysis (3DMA) system were collected simultaneously. Intraclass correlation coefficients (ICCs), coefficients of variation (CV) and standard error of measurement (SEM) were used to assess reliability. To assess validity of the Smartphone application and the bubble inclinometer against the three dimensional motion analysis system, intraclass correlation coefficients and fixed and proportional biases were used. The Smartphone demonstrated good to excellent reliability (ICCs>0.75) for four out of the seven movements, and moderate to good reliability for the remaining three movements (ICC=0.63-0.68). Additionally, the Smartphone application displayed comparable reliability to the bubble inclinometer. The Smartphone application displayed excellent validity when compared to the three dimensional motion analysis system for all movements (ICCs>0.88) except one, which displayed moderate to good validity (ICC=0.71). Smartphones are portable and widely available tools that are mostly reliable and valid for assessing passive hip range of motion, with potential for large-scale use when a bubble inclinometer is not available. However, caution must be taken in its implementation as some movement axes demonstrated only moderate reliability. Copyright © 2014 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

[Contributions of child psychology to the assessment of pain].

PubMed

Zabalia, M

2009-12-01

The efficacy of pain management is subordinated to the efficacy of pain assessment. Children are particularly difficult-to-assess patients. In this area, child psychology can contribute to the development of tools and to improving pain assessment. This paper highlights the relevance of studies on dialog in pain assessment situations and the importance of the specificity of both the patient and his pain. Assessing pain and listening to the patient's complaint means meeting the child in his own world. Nonetheless, it is necessary to use and develop assessment tools that are scientifically validated.

Assessing face validity of a physical activity questionnaire for Spanish-speaking women in California

USDA-ARS?s Scientific Manuscript database

Background: A review of the literature produced no rigorously tested and validated Spanish-language physical activity survey or evaluation tools for use by USDA’s food assistance and education programs. The purpose of the current study was to develop and evaluate the face validity of a visually enha...

Assessing the Validity of an Annual Survey for Measuring Principal Leadership Practice

ERIC Educational Resources Information Center

Camburn, Eric M.; Huff, Jason T.; Goldring, Ellen B.; May, Henry

2010-01-01

Because research has shown that principals' practices can significantly impact teaching and learning in their schools through multiple avenues, it is critical to understand the validity of tools that measure principal practice. While the field has relied heavily on self-report surveys, little is known about their validity. This study compares…

Validation of a Tool to Assess and Track Undergraduate Attitudes toward Those Living in Poverty

ERIC Educational Resources Information Center

Blair, Kevin D.; Brown, Marlo; Schoepflin, Todd; Taylor, David B.

2014-01-01

Purpose: This article describes the development and validation of the Undergraduate Perceptions of Poverty Tracking Survey (UPPTS). Method: Data were collected from 301 undergraduates at a small university in the Northeast and analyzed using exploratory factor analysis augmented by random qualitative validation. Results: The resulting survey…

Development, validation and psychometric properties of a diagnostic/prognostic tool for breakthrough pain in mixed chronic-pain patients.

PubMed

Samolsky Dekel, Boaz Gedaliahu; Remondini, Francesca; Gori, Alberto; Vasarri, Alessio; Di Nino, GianFranco; Melotti, Rita Maria

2016-02-01

Breakthrough pain (BTP) shows variable prevalence in different clinical contexts of cancer and non-cancer patients. BTP diagnostic tools with demonstrated reliability, validation and prognostic capability are lacking. We report the development, psychometric and validation properties of a diagnostic/prognostic tool, the IQ-BTP, for BTP recognition, its likelihood and clinical features among chronic-pain (CP) patients. n=120 consecutive mixed cancer/non-cancer CP in/outpatients. Development, psychometric analyses and formal validation included: Face/Content validity (by 'experts' opinion and assessing the relationship between the IQ-BTP classes and criteria derived from BTP operational-case-definition); Construct validity, by Principle Component Analysis (PCA); and the strength of Spearman correlation between IQ-BTP classes and the Brief Pain Inventory (BPI) items; Reliability, by Cronbach's alpha statistics. Associations with clinical/demographic moderators were assessed applying χ(2) analysis. Potential-BTP was found in 36.7% of patients (38.4% of non-cancer and 32.4% of cancer patients). Among these the likelihood for BTP diagnosis was 'high' in 25%, 'intermediate' in 41% and, 'low' 34% of patients. Analyses showed significant differences between IQ-BTP classes and between the latter BPI pain-item scores. Correlation between IQ-BTP classes and BPI items was moderate. PCA and scree test identified 3 components accounting for 62.3% of the variance. Cronbach's alpha was 0.71. The IQ-BTP showed satisfactory psychometric and validation properties. With adequate feasibility it enabled the allocating of cancer/non-cancer CP patients in three prognostic classes. Results are sufficient to warrant a subsequent impact study of the IQ-BTP as prognostic model and screening tool for BTP in both CP populations. Copyright © 2016 Elsevier B.V. All rights reserved.

Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly.

PubMed

Brouillette, Robert M; Foil, Heather; Fontenot, Stephanie; Correro, Anthony; Allen, Ray; Martin, Corby K; Bruce-Keller, Annadora J; Keller, Jeffrey N

2013-01-01

While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these "next-generation" assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST) in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, p<0.0001) and multiple measures of processing speed and attention (Digit Span: r = 0.427, p<0.0001; Trail Making Test: r = -0.651, p<0.00001; Digit Symbol Test: r = 0.508, p<0.0001). The CST was not correlated with naming and verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming) or memory tasks (Logical Memory Test). Test re-test reliability was observed to be significant (r = 0.726; p = 0.02). Together, these data are the first to demonstrate the feasibility, reliability, and validity of using a smartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries of cognitive and motor function in the elderly is discussed.

The East London glaucoma prediction score: web-based validation of glaucoma risk screening tool

PubMed Central

Stephen, Cook; Benjamin, Longo-Mbenza

2013-01-01

AIM It is difficult for Optometrists and General Practitioners to know which patients are at risk. The East London glaucoma prediction score (ELGPS) is a web based risk calculator that has been developed to determine Glaucoma risk at the time of screening. Multiple risk factors that are available in a low tech environment are assessed to provide a risk assessment. This is extremely useful in settings where access to specialist care is difficult. Use of the calculator is educational. It is a free web based service. Data capture is user specific. METHOD The scoring system is a web based questionnaire that captures and subsequently calculates the relative risk for the presence of Glaucoma at the time of screening. Three categories of patient are described: Unlikely to have Glaucoma; Glaucoma Suspect and Glaucoma. A case review methodology of patients with known diagnosis is employed to validate the calculator risk assessment. RESULTS Data from the patient records of 400 patients with an established diagnosis has been captured and used to validate the screening tool. The website reports that the calculated diagnosis correlates with the actual diagnosis 82% of the time. Biostatistics analysis showed: Sensitivity = 88%; Positive predictive value = 97%; Specificity = 75%. CONCLUSION Analysis of the first 400 patients validates the web based screening tool as being a good method of screening for the at risk population. The validation is ongoing. The web based format will allow a more widespread recruitment for different geographic, population and personnel variables. PMID:23550097

Psychometric Properties of Korean Version of the Second Victim Experience and Support Tool (K-SVEST).

PubMed

Kim, Eun-Mi; Kim, Sun-Aee; Lee, Ju-Ry; Burlison, Jonathan D; Oh, Eui Geum

2018-02-13

"Second victims" are defined as healthcare professionals whose wellness is influenced by adverse clinical events. The Second Victim Experience and Support Tool (SVEST) was used to measure the second-victim experience and quality of support resources. Although the reliability and validity of the original SVEST have been validated, those for the Korean tool have not been validated. The aim of the study was to evaluate the psychometric properties of the Korean version of the SVEST. The study included 305 clinical nurses as participants. The SVEST was translated into Korean via back translation. Content validity was assessed by seven experts, and test-retest reliability was evaluated by 30 clinicians. Internal consistency and construct validity were assessed via confirmatory factor analysis. The analyses were performed using SPSS 23.0 and STATA 13.0 software. The content validity index value demonstrated validity; item- and scale-level content validity index values were both 0.95. Test-retest reliability and internal consistency reliability were satisfactory: the intraclass consistent coefficient was 0.71, and Cronbach α values ranged from 0.59 to 0.87. The CFA showed a significantly good fit for an eight-factor structure (χ = 578.21, df = 303, comparative fit index = 0.92, Tucker-Lewis index = 0.90, root mean square error of approximation = 0.05). The K-SVEST demonstrated good psychometric properties and adequate validity and reliability. The results showed that the Korean version of SVEST demonstrated the extent of second victimhood and support resources in Korean healthcare workers and could aid in the development of support programs and evaluation of their effectiveness.

OA Go Away: Development and Preliminary Validation of a Self-Management Tool to Promote Adherence to Exercise and Physical Activity for People with Osteoarthritis of the Hip or Knee

PubMed Central

Toupin April, Karine; Backman, Catherine; Tugwell, Peter

2016-01-01

Purpose: To determine the face and content validity, construct validity, and test–retest reliability of the OA Go Away (OGA), a personalized self-management tool to promote adherence to exercise and physical activity for people with osteoarthritis (OA) of the hip or knee. Methods: The face and content validity of OGA version 1.0 were determined via interviews with 10 people with OA of the hip or knee and 10 clinicians. A revised OGA version 2.0 was then tested for construct validity and test–retest reliability with a new sample of 50 people with OA of the hip or knee by comparing key items in the OGA journal with validated outcome measures assessing similar health outcomes and comparing scores on key items of the journal 4–7 days apart. Face and content validity were then confirmed with a new sample of 5 people with OA of the hip or knee and 5 clinicians. Results: Eighteen of 30 items from the OGA version 1.0 and 41 of 43 items from the OGA version 2.0 journal, goals and action plan, and exercise log had adequate content validity. Construct validity and test–retest reliability were acceptable for the main items of the OGA version 2.0 journal. The OGA underwent modifications based on results and participant feedback. Conclusion: The OGA is a novel self-management intervention and assessment tool for people with OA of the hip or knee that shows adequate preliminary measurement properties. PMID:27909359

Validation of the second version of the LittlEARS® Early Speech Production Questionnaire (LEESPQ) in German-speaking children with normal hearing.

PubMed

Keilmann, Annerose; Friese, Barbara; Lässig, Anne; Hoffmann, Vanessa

2018-04-01

The introduction of neonatal hearing screening and the increasingly early age at which children can receive a cochlear implant has intensified the need for a validated questionnaire to assess the speech production of children aged 0‒18. Such a questionnaire has been created, the LittlEARS ® Early Speech Production Questionnaire (LEESPQ). This study aimed to validate a second, revised edition of the LEESPQ. Questionnaires were returned for 362 children with normal hearing. Completed questionnaires were analysed to determine if the LEESPQ is reliable, prognostically accurate, internally consistent, and if gender or multilingualism affects total scores. Total scores correlated positively with age. The LEESPQ is reliable, accurate, and consistent, and independent of gender or lingual status. A norm curve was created. This second version of the LEESPQ is a valid tool to assess the speech production development of children with normal hearing, aged 0‒18, regardless of their gender. As such, the LEESPQ may be a useful tool to monitor the development of paediatric hearing device users. The second version of the LEESPQ is a valid instrument for assessing early speech production of children aged 0‒18 months.

Development of a tool to support holistic generic assessment of clinical procedure skills.

PubMed

McKinley, Robert K; Strand, Janice; Gray, Tracey; Schuwirth, Lambert; Alun-Jones, Tom; Miller, Helen

2008-06-01

The challenges of maintaining comprehensive banks of valid checklists make context-specific checklists for assessment of clinical procedural skills problematic. This paper reports the development of a tool which supports generic holistic assessment of clinical procedural skills. We carried out a literature review, focus groups and non-participant observation of assessments with interview of participants, participant evaluation of a pilot objective structured clinical examination (OSCE), a national modified Delphi study with prior definitions of consensus and an OSCE. Participants were volunteers from a large acute teaching trust, a teaching primary care trust and a national sample of National Health Service staff. Results In total, 86 students, trainees and staff took part in the focus groups, observation of assessments and pilot OSCE, 252 in the Delphi study and 46 candidates and 50 assessors in the final OSCE. We developed a prototype tool with 5 broad categories amongst which were distributed 38 component competencies. There was > 70% agreement (our prior definition of consensus) at the first round of the Delphi study for inclusion of all categories and themes and no consensus for inclusion of additional categories or themes. Generalisability was 0.76. An OSCE based on the instrument has a predicted reliability of 0.79 with 12 stations and 1 assessor per station or 10 stations and 2 assessors per station. This clinical procedural skills assessment tool enables reliable assessment and has content and face validity for the assessment of clinical procedural skills. We have designated it the Leicester Clinical Procedure Assessment Tool (LCAT).

Social Validation of the New England Center for Children-Core Skills Assessment

ERIC Educational Resources Information Center

Dickson, Chata A.; MacDonald, Rebecca P. F.; Mansfield, Renee; Guilhardi, Paulo; Johnson, Cammarie; Ahearn, William H.

2014-01-01

We investigated the social validity of the NECC Core Skills Assessment (NECC-CSA) with parents and professionals as participants. The NECC-CSA is a measurement tool consisting of direct and indirect measures of skills important to all individuals with autism, across the lifespan. Participants (N = 245) were provided with a list of 66 skills, 47 of…

Second-Order Factor Analysis as a Validity Assessment Tool: A Case Study Example Involving Perceptions of Stereotypic Love.

ERIC Educational Resources Information Center

Borrello, Gloria M.; Thompson, Bruce

The calculation of second-order results in the validity assessment of measures and some useful interpretation aids are presented. First-order and second-order results give different and informative pictures of data dynamics. Several aspects of good practice in interpretation of second-order results are presented using data from 487 subjects…

Assessing Autistic Traits in a Taiwan Preschool Population: Cross-Cultural Validation of the Social Responsiveness Scale (SRS)

ERIC Educational Resources Information Center

Wang, Jessica; Lee, Li-Ching; Chen, Ying-Sheue; Hsu, Ju-Wei

2012-01-01

The cross-cultural validity of the Mandarin-adaptation of the social responsiveness scale (SRS) was examined in a sample of N = 307 participants in Taiwan, 140 typically developing and 167 with clinically-diagnosed developmental disorders. This scale is an autism assessment tool that provides a quantitative rather than categorical measure of…

Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

PubMed

Hobbelen, Johannes S M; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A

2008-08-01

Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability and validity of the PAI. A three-phase cross-sectional survey was conducted. In the first two phases, the PAI was developed and validated. In the third phase, the inter-observer reliability and feasibility of the instrument was tested. The original PAI consisted of five criteria that all needed to be met in order to make the diagnosis. On the basis of a qualitative analysis, one criterion was reformulated and another was removed. Following this, inter-observer reliability between the two assessors resulted in an improvement of Cohen's kappa from 0.532 in the initial phase to 0.677 in the second phase. This improvement was substantiated in the third phase by two independent assessors with Cohen's kappa ranging from 0.625 to 1. The PAI is a reliable and valid assessment tool for diagnosing paratonia in elderly people with dementia that can be applied easily in daily practice.

Measuring competence in endoscopic sinus surgery.

PubMed

Syme-Grant, J; White, P S; McAleer, J P G

2008-02-01

Competence based education is currently being introduced into higher surgical training in the UK. Valid and reliable performance assessment tools are essential to ensure competencies are achieved. No such tools have yet been reported in the UK literature. We sought to develop and pilot test an Endoscopic Sinus Surgery Competence Assessment Tool (ESSCAT). The ESSCAT was designed for in-theatre assessment of higher surgical trainees in the UK. The ESSCAT rating matrix was developed through task analysis of ESS procedures. All otolaryngology consultants and specialist registrars in Scotland were given the opportunity to contribute to its refinement. Two cycles of in-theatre testing were used to ensure utility and gather quantitative data on validity and reliability. Videos of trainees performing surgery were used in establishing inter-rater reliability. National consultation, the consensus derived minimum standard of performance, Cronbach's alpha = 0.89 and demonstration of trainee learning (p = 0.027) during the in vivo application of the ESSCAT suggest a high level of validity. Inter-rater reliability was moderate for competence decisions (Cohen's Kappa = 0.5) and good for total scores (Intra-Class Correlation Co-efficient = 0.63). Intra-rater reliability was good for both competence decisions (Kappa = 0.67) and total scores (Kendall's Tau-b = 0.73). The ESSCAT generates a valid and reliable assessment of trainees' in-theatre performance of endoscopic sinus surgery. In conjunction with ongoing evaluation of the instrument we recommend the use of the ESSCAT in higher specialist training in otolaryngology in the UK.

The letter knowledge assessment tool.

PubMed

Pedro, Cassandra; Lousada, Marisa; Pereira, Rita; Hall, Andreia; Jesus, Luis M T

2017-10-10

There is a need to develop letter knowledge assessment tools to characterise the letter knowledge in Portuguese pre-schoolers and to compare it with pre-schoolers from other countries, but there are no tools for this purpose in Portugal. The aim of this paper is to describe the development and validation procedures of the Prova de Avaliação de Competências de Pré-Literacia (PACPL), which assesses letter knowledge. This study includes data that has been gathered in two phases: pilot and main study. In the pilot study, an expert panel of six speech and language pathologists analysed the instrument. Children (n = 216) aged 5;0-7;11 participated in the main study that reports data related to the psychometric characteristics of the PACPL. Content validity, internal consistency, reliability and contributing factors to performance were examined statistically. A modified Bland-Altman method revealed good agreement amongst evaluators. The main study showed that the PACPL has a very good internal consistency and high inter-rater (96.2% of agreement and a Cohen's k value of 0.92) and intra-rater (95.6% of agreement and a Cohen's k value of 0.91) agreement. Construct validity of the PCAPL was also assured (Cronbach's α of 0.982). Significant differences were found between age groups with children increasing their letter knowledge with age. In addition, they were better at identifying than at producing both letter names and letter sounds. The PACPL is a valid and reliable instrument to assess letter knowledge in Portuguese children.

Falls risk assessment begins with hello: lessons learned from the use of one home health agency's fall risk tool.

PubMed

Flemming, Patricia J; Ramsay, Katherine

2012-10-01

Identifying older adults at risk for falls is a challenge all home healthcare agencies (HHAs) face. The process of assessing for falls risk begins with the initial home visit. One HHA affiliated with an academic medical center describes its experience in development and use of a Falls Risk Assessment (FRA) tool over a 10-year period. The FRA tool has been modified since initial development to clarify elements of the tool based on research and to reflect changes in the Outcome and Assessment Information Set (OASIS) document. The primary purpose of this article is to share a validated falls risk assessment tool to facilitate identification of fall-related risk factors in the homebound population. A secondary purpose is to share lessons learned by the HHA during the 10 years using the FRA.

Reliability and criterion-related validity testing (construct) of the Endotracheal Suction Assessment Tool (ESAT©).

PubMed

Davies, Kylie; Bulsara, Max K; Ramelet, Anne-Sylvie; Monterosso, Leanne

2018-05-01

To establish criterion-related construct validity and test-retest reliability for the Endotracheal Suction Assessment Tool© (ESAT©). Endotracheal tube suction performed in children can significantly affect clinical stability. Previously identified clinical indicators for endotracheal tube suction were used as criteria when designing the ESAT©. Content validity was reported previously. The final stages of psychometric testing are presented. Observational testing was used to measure construct validity and determine whether the ESAT© could guide "inexperienced" paediatric intensive care nurses' decision-making regarding endotracheal tube suction. Test-retest reliability of the ESAT© was performed at two time points. The researchers and paediatric intensive care nurse "experts" developed 10 hypothetical clinical scenarios with predetermined endotracheal tube suction outcomes. "Experienced" (n = 12) and "inexperienced" (n = 14) paediatric intensive care nurses were presented with the scenarios and the ESAT© guiding decision-making about whether to perform endotracheal tube suction for each scenario. Outcomes were compared with those predetermined by the "experts" (n = 9). Test-retest reliability of the ESAT© was measured at two consecutive time points (4 weeks apart) with "experienced" and "inexperienced" paediatric intensive care nurses using the same scenarios and tool to guide decision-making. No differences were observed between endotracheal tube suction decisions made by "experts" (n = 9), "inexperienced" (n = 14) and "experienced" (n = 12) nurses confirming the tool's construct validity. No differences were observed between groups for endotracheal tube suction decisions at T1 and T2. Criterion-related construct validity and test-retest reliability of the ESAT© were demonstrated. Further testing is recommended to confirm reliability in the clinical setting with the "inexperienced" nurse to guide decision-making related to endotracheal tube suction. The ESAT© is the first validated tool to systematically guide endotracheal nursing practice for the "inexperienced" nurse. © 2018 John Wiley & Sons Ltd.

Assessment Tools for Evaluation of Oral Feeding in Infants Younger Than 6 Months.

PubMed

Pados, Britt F; Park, Jinhee; Estrem, Hayley; Awotwi, Araba

2016-04-01

Feeding difficulty is common in infants younger than 6 months. Identification of infants in need of specialized treatment is critical to ensure appropriate nutrition and feeding skill development. Valid and reliable assessment tools help clinicians objectively evaluate feeding. To identify and evaluate assessment tools available for clinical assessment of bottle- and breastfeeding in infants younger than 6 months. CINAHL, HaPI, PubMed, and Web of Science were searched for "infant feeding" and "assessment tool." The literature (n = 237) was reviewed for relevant assessment tools. A secondary search was conducted in CINAHL and PubMed for additional literature on identified tools. Eighteen assessment tools met inclusion criteria. Of these, 7 were excluded because of limited available literature or because they were intended for use with a specific diagnosis or in research only. There are 11 assessment tools available for clinical practice. Only 2 of these were intended for bottle-feeding. All 11 indicated that they were appropriate for use with breastfeeding. None of the available tools have adequate psychometric development and testing. All of the tools should be used with caution. The Early Feeding Skills Assessment and Bristol Breastfeeding Assessment Tool had the most supportive psychometric development and testing. Feeding assessment tools need to be developed and tested to guide optimal clinical care of infants from birth through 6 months. A tool that assesses both bottle- and breastfeeding would allow for consistent assessment across feeding methods.

The significance of motivation in periodontal treatment: validity and reliability of the motivation assessment scale among patients undergoing periodontal treatment.

PubMed

Pac, A; Oruba, Z; Olszewska-Czyż, I; Chomyszyn-Gajewska, M

2014-03-01

The individual evaluation of patients' motivation should be introduced to the protocol of periodontal treatment, as it could impact positively on effective treatment planning and treatment outcomes. However, a standardised tool measuring the extent of periodontal patients' motivation has not yet been proposed in the literature. Thus, the objective of the present study was to determine the validity and reliability of the Zychlińscy motivation scale adjusted to the needs of periodontology. Cross sectional study. Department of Periodontology and Oral Medicine, Dental University Clinic, Jagiellonian University, Krakow, Poland. 199 adult periodontal patients, aged 20-78. 14-item questionnaire. The items were adopted from the original Zychlińscy motivation assessment scale. Validity and reliability of the proposed motivation assessment instrument. The assessed Cronbach's alpha of 0.79 indicates the scale is a reliable tool. Principal component analysis revealed a model with three factors, which explained half of the total variance. Those factors represented: the patient's attitude towards treatment and oral hygiene practice; previous experiences during treatment; and the influence of external conditions on the patient's attitude towards treatment. The proposed scale proved to be a reliable and accurate tool for the evaluation of periodontal patients' motivation.

Validation of Medical Tourism Service Quality Questionnaire (MTSQQ) for Iranian Hospitals.

PubMed

Qolipour, Mohammad; Torabipour, Amin; Khiavi, Farzad Faraji; Malehi, Amal Saki

2017-03-01

Assessing service quality is one of the basic requirements to develop the medical tourism industry. There is no valid and reliable tool to measure service quality of medical tourism. This study aimed to determine the reliability and validity of a Persian version of medical tourism service quality questionnaire for Iranian hospitals. To validate the medical tourism service quality questionnaire (MTSQQ), a cross-sectional study was conducted on 250 Iraqi patients referred to hospitals in Ahvaz (Iran) from 2015. To design a questionnaire and determine its content validity, the Delphi Technique (3 rounds) with the participation of 20 medical tourism experts was used. Construct validity of the questionnaire was assessed through exploratory and confirmatory factor analysis. Reliability was assessed using Cronbach's alpha coefficient. Data were analyzed by Excel 2007, SPSS version18, and Lisrel l8.0 software. The content validity of the questionnaire with CVI=0.775 was confirmed. According to exploratory factor analysis, the MTSQQ included 31 items and 8 dimensions (tangibility, reliability, responsiveness, assurance, empathy, exchange and travel facilities, technical and infrastructure facilities and safety and security). Construct validity of the questionnaire was confirmed, based on the goodness of fit quantities of model (RMSEA=0.032, CFI= 0.98, GFI=0.88). Cronbach's alpha coefficient was 0.837 and 0.919 for expectation and perception questionnaire. The results of the study showed that the medical tourism SERVQUAL questionnaire with 31 items and 8 dimensions was a valid and reliable tool to measure service quality of medical tourism in Iranian hospitals.

The development of a supportive care needs assessment tool for Indigenous people with cancer

PubMed Central

2012-01-01

Background Little is known about the supportive care needs of Indigenous people with cancer and to date, existing needs assessment tools have not considered cultural issues for this population. We aimed to adapt an existing supportive care needs assessment tool for use with Indigenous Australians with cancer. Methods Face-to-face interviews with Indigenous cancer patients (n = 29) and five focus groups with Indigenous key-informants (n = 23) were conducted to assess the face and content validity, cultural acceptability, utility and relevance of the Supportive Care Needs Survey - Short Form 34 (SCNS-SF34) for use with Indigenous patients with cancer. Results All items from the SCNS-SF34 were shortened and changed to use more appropriate language (e.g. the word 'anxiety' was substituted with 'worry'). Seven questions were omitted (e.g. items on death and future considerations) as they were deemed culturally inappropriate or irrelevant and 12 items were added (e.g. accessible transport). Optional instructions were added before the sexual items. The design and response format of the SCNS-SF34 was modified to make it easier to use for Indigenous cancer patients. Given the extensive modifications to the SCNS-SF34 and the liklihood of a different factor structure we consider this tool to be a new tool rather than a modification. The Supportive care needs assessment tool for Indigenous people (SCNAT-IP) shows promising face and content validity and will be useful in informing services where they need to direct their attention for these patients. Conclusions Indigenous people with cancer have language, customs and specific needs that are not accommodated within the standard SCNS-SF34. Our SCNAT-IP improves acceptability, relevance and face validity for Indigenous-specific concerns. Our SCNAT-IP will allow screening for supportive care needs that are specific to Indigenous cancer patients' and greatly inform targeted policy development and practice. PMID:22817614

Validation of the FFM PD count technique for screening personality pathology in later middle-aged and older adults.

PubMed

Van den Broeck, Joke; Rossi, Gina; De Clercq, Barbara; Dierckx, Eva; Bastiaansen, Leen

2013-01-01

Research on the applicability of the five factor model (FFM) to capture personality pathology coincided with the development of a FFM personality disorder (PD) count technique, which has been validated in adolescent, young, and middle-aged samples. This study extends the literature by validating this technique in an older sample. Five alternative FFM PD counts based upon the Revised NEO Personality Inventory (NEO PI-R) are computed and evaluated in terms of both convergent and divergent validity with the Assessment of DSM-IV Personality Disorders Questionnaire (shortly ADP-IV; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders - Fourth edition). For the best working count for each PD normative data are presented, from which cut-off scores are derived. The validity of these cut-offs and their usefulness as a screening tool is tested against both a categorical (i.e., the DSM-IV - Text Revision), and a dimensional (i.e., the Dimensional Assessment of Personality Pathology; DAPP) measure of personality pathology. All but the Antisocial and Obsessive-Compulsive counts exhibited adequate convergent and divergent validity, supporting the use of this method in older adults. Using the ADP-IV and the DAPP - Short Form as validation criteria, results corroborate the use of the FFM PD count technique to screen for PDs in older adults, in particular for the Paranoid, Borderline, Histrionic, Avoidant, and Dependent PDs. Given the age-neutrality of the NEO PI-R and the considerable lack of valid personality assessment tools, current findings appear to be promising for the assessment of pathology in older adults.

The Arthroscopic Surgical Skill Evaluation Tool (ASSET).

PubMed

Koehler, Ryan J; Amsdell, Simon; Arendt, Elizabeth A; Bisson, Leslie J; Braman, Jonathan P; Bramen, Jonathan P; Butler, Aaron; Cosgarea, Andrew J; Harner, Christopher D; Garrett, William E; Olson, Tyson; Warme, Winston J; Nicandri, Gregg T

2013-06-01

Surgeries employing arthroscopic techniques are among the most commonly performed in orthopaedic clinical practice; however, valid and reliable methods of assessing the arthroscopic skill of orthopaedic surgeons are lacking. The Arthroscopic Surgery Skill Evaluation Tool (ASSET) will demonstrate content validity, concurrent criterion-oriented validity, and reliability when used to assess the technical ability of surgeons performing diagnostic knee arthroscopic surgery on cadaveric specimens. Cross-sectional study; Level of evidence, 3. Content validity was determined by a group of 7 experts using the Delphi method. Intra-articular performance of a right and left diagnostic knee arthroscopic procedure was recorded for 28 residents and 2 sports medicine fellowship-trained attending surgeons. Surgeon performance was assessed by 2 blinded raters using the ASSET. Concurrent criterion-oriented validity, interrater reliability, and test-retest reliability were evaluated. Content validity: The content development group identified 8 arthroscopic skill domains to evaluate using the ASSET. Concurrent criterion-oriented validity: Significant differences in the total ASSET score (P < .05) between novice, intermediate, and advanced experience groups were identified. Interrater reliability: The ASSET scores assigned by each rater were strongly correlated (r = 0.91, P < .01), and the intraclass correlation coefficient between raters for the total ASSET score was 0.90. Test-retest reliability: There was a significant correlation between ASSET scores for both procedures attempted by each surgeon (r = 0.79, P < .01). The ASSET appears to be a useful, valid, and reliable method for assessing surgeon performance of diagnostic knee arthroscopic surgery in cadaveric specimens. Studies are ongoing to determine its generalizability to other procedures as well as to the live operating room and other simulated environments.

Assessing Student Understanding of the "New Biology": Development and Evaluation of a Criterion-Referenced Genomics and Bioinformatics Assessment

NASA Astrophysics Data System (ADS)

Campbell, Chad Edward

Over the past decade, hundreds of studies have introduced genomics and bioinformatics (GB) curricula and laboratory activities at the undergraduate level. While these publications have facilitated the teaching and learning of cutting-edge content, there has yet to be an evaluation of these assessment tools to determine if they are meeting the quality control benchmarks set forth by the educational research community. An analysis of these assessment tools indicated that <10% referenced any quality control criteria and that none of the assessments met more than one of the quality control benchmarks. In the absence of evidence that these benchmarks had been met, it is unclear whether these assessment tools are capable of generating valid and reliable inferences about student learning. To remedy this situation the development of a robust GB assessment aligned with the quality control benchmarks was undertaken in order to ensure evidence-based evaluation of student learning outcomes. Content validity is a central piece of construct validity, and it must be used to guide instrument and item development. This study reports on: (1) the correspondence of content validity evidence gathered from independent sources; (2) the process of item development using this evidence; (3) the results from a pilot administration of the assessment; (4) the subsequent modification of the assessment based on the pilot administration results and; (5) the results from the second administration of the assessment. Twenty-nine different subtopics within GB (Appendix B: Genomics and Bioinformatics Expert Survey) were developed based on preliminary GB textbook analyses. These subtopics were analyzed using two methods designed to gather content validity evidence: (1) a survey of GB experts (n=61) and (2) a detailed content analyses of GB textbooks (n=6). By including only the subtopics that were shown to have robust support across these sources, 22 GB subtopics were established for inclusion in the assessment. An expert panel subsequently developed, evaluated, and revised two multiple-choice items to align with each of the 22 subtopics, producing a final item pool of 44 items. These items were piloted with student samples of varying content exposure levels. Both Classical Test Theory (CTT) and Item Response Theory (IRT) methodologies were used to evaluate the assessment's validity, reliability and ability inferences, and its ability to differentiate students with different magnitudes of content exposure. A total of 18 items were subsequently modified and reevaluated by an expert panel. The 26 original and 18 modified items were once again piloted with student samples of varying content exposure levels. Both CTT and IRT methodologies were once again used to evaluate student responses in order to evaluate the assessment's validity and reliability inferences as well as its ability to differentiate students with different magnitudes of content exposure. Interviews with students from different content exposure levels were also performed in order to gather convergent validity evidence (external validity evidence) as well as substantive validity evidence. Also included are the limitations of the assessment and a set of guidelines on how the assessment can best be used.

Construct measurement quality improves predictive accuracy in violence risk assessment: an illustration using the personality assessment inventory.

PubMed

Hendry, Melissa C; Douglas, Kevin S; Winter, Elizabeth A; Edens, John F

2013-01-01

Much of the risk assessment literature has focused on the predictive validity of risk assessment tools. However, these tools often comprise a list of risk factors that are themselves complex constructs, and focusing on the quality of measurement of individual risk factors may improve the predictive validity of the tools. The present study illustrates this concern using the Antisocial Features and Aggression scales of the Personality Assessment Inventory (Morey, 1991). In a sample of 1,545 prison inmates and offenders undergoing treatment for substance abuse (85% male), we evaluated (a) the factorial validity of the ANT and AGG scales, (b) the utility of original ANT and AGG scales and newly derived ANT and AGG scales for predicting antisocial outcomes (recidivism and institutional infractions), and (c) whether items with a stronger relationship to the underlying constructs (higher factor loadings) were in turn more strongly related to antisocial outcomes. Confirmatory factor analyses (CFAs) indicated that ANT and AGG items were not structured optimally in these data in terms of correspondence to the subscale structure identified in the PAI manual. Exploratory factor analyses were conducted on a random split-half of the sample to derive optimized alternative factor structures, and cross-validated in the second split-half using CFA. Four-factor models emerged for both the ANT and AGG scales, and, as predicted, the size of item factor loadings was associated with the strength with which items were associated with institutional infractions and community recidivism. This suggests that the quality by which a construct is measured is associated with its predictive strength. Implications for risk assessment are discussed. Copyright © 2013 John Wiley & Sons, Ltd.

An international road map to improve pain assessment in people with impaired cognition: the development of the Pain Assessment in Impaired Cognition (PAIC) meta-tool.

PubMed

Corbett, Anne; Achterberg, Wilco; Husebo, Bettina; Lobbezoo, Frank; de Vet, Henrica; Kunz, Miriam; Strand, Liv; Constantinou, Marios; Tudose, Catalina; Kappesser, Judith; de Waal, Margot; Lautenbacher, Stefan

2014-12-10

Pain is common in people with dementia, yet identification is challenging. A number of pain assessment tools exist, utilizing observation of pain-related behaviours, vocalizations and facial expressions. Whilst they have been developed robustly, these often lack sufficient evidence of psychometric properties, like reliability, face and construct validity, responsiveness and usability, and are not internationally implemented. The EU-COST initiative "Pain in impaired cognition, especially dementia" aims to combine the expertise of clinicians and researchers to address this important issue by building on previous research in the area, identifying existing pain assessment tools for dementia, and developing consensus for items for a new universal meta-tool for use in research and clinical settings. This paper reports on the initial phase of this collaboration task. All existing observational pain behaviour tools were identified and elements categorised using a three-step reduction process. Selection and refinement of items for the draft Pain Assessment in Impaired Cognition (PAIC) meta-tool was achieved through scrutiny of the evidence, consensus of expert opinion, frequency of use and alignment with the American Geriatric Society guidelines. The main aim of this process was to identify key items with potential empirical, rather than theoretical value to take forward for testing. 12 eligible assessment tools were identified, and pain items categorised according to behaviour, facial expression and vocalisation according to the AGS guidelines (Domains 1 - 3). This has been refined to create the PAIC meta-tool for validation and further refinement. A decision was made to create a supporting comprehensive toolkit to support the core assessment tool to provide additional resources for the assessment of overlapping symptoms in dementia, including AGS domains four to six, identification of specific types of pain and assessment of duration and location of pain. This multidisciplinary, cross-cultural initiative has created a draft meta-tool for capturing pain behaviour to be used across languages and culture, based on the most promising items used in existing tools. The draft PAIC meta-tool will now be taken forward for evaluation according to COSMIN guidelines and the EU-COST protocol in order to exclude invalid items, refine included items and optimise the meta-tool.

Measuring Certified Registered Nurse Anesthetist Organizational Climate: Instrument Adaptation.

PubMed

Boyd, Donald; Poghosyan, Lusine

2017-08-01

No tool exists measuring certified registered nurse anesthetist (CRNA) organizational climate. The study's purpose is to adapt a validated tool to measure CRNA organizational climate. Content validity of the Certified Registered Nurse Anesthetist Organizational Climate Questionnaire (CRNA-OCQ) was established. Pilot testing was conducted to determine internal reliability consistency of the subscales. Experts rated the tool as content valid. The subscales had high internal consistency reliability (with respective Cronbach's alphas): CRNA-Anesthesiologist Relations (.753), CRNA-Physician Relations (.833), CRNA-Administration Relations (.895), Independent Practice (.830), Support for CRNA Practice (.683), and Professional Visibility (.772). Further refinement of the CRNA-OCQ is necessary. Measurement and assessment of CRNA organizational climate may produce evidence needed to improve provider and patient outcomes.

Is the Berg Balance Scale an effective tool for the measurement of early postural control impairments in patients with Parkinson's disease? Evidence from Rasch analysis.

PubMed

La Porta, F; Giordano, A; Caselli, S; Foti, C; Franchignoni, F

2015-12-01

It is unclear whether the BBS is an effective tool for the measurement of early postural control impairments in patients with Parkinson's disease (PD). The aim of this paper was to evaluate BBS' content validity, internal construct validity, reliability and targeting in patients with PD within the Rasch analysis framework. Observational, cross-sectional study. Outpatient Rehabilitation Unit. A sample of 285 outpatients with PD. The content validity of the BBS was assessed using standard linking techniques. The BBS was administered by trained physiotherapists. The data collected then underwent Rasch analysis. Content validity analysis showed a lack of items assessing postural responses to tripping and slips and stability during walking. On Rasch analysis, the BBS failed the requirements of monotonicity, local independence, unidimensionality and invariance. After rescoring 7 items, grouping of locally dependent items into testlets, and deletion of the static sitting balance item because mistargeted and underdiscriminating, the Rasch-modified BBS for PD (BBS-PD) showed adequate internal construct validity (χ(2)24=39.693; P=0.023), including absence of differential item functioning (DIF) across gender and age, and was, as a whole, sufficiently precise for individual person measurement (PSI=0.894). However, the scale was not well targeted to the sample in view of the prevalence of higher scores. This study demonstrated the internal construct validity and reliability of the BBS-PD as a measurement tool for patients with PD within the Rasch analysis framework. However, the lack of items critical to the assessment of postural control impairments typical of PD, affected negatively the targeting, so that a significant percentage of patients was located in the higher ability range of the measurement continuum, where precision of measurement is reduced. These findings suggest that the BBS, even if modified, may not be an effective tool for the measurement of early postural control in patients with PD.

The Development and Validation of the Transition Competence Battery for Adolescents and Young Adults with Deafness.

ERIC Educational Resources Information Center

Reiman, John; Bullis, Michael

1988-01-01

The lack of appropriate assessment tools designed for deaf adolescents and young adults making the transition from educational programs to adult life is cited as one of the most glaring deficits in the field of deafness. The Transition Competence Battery (TCB) is being developed as an assessment tool that will guide individual training decisions…

Additional Support for the Information Systems Analyst Exam as a Valid Program Assessment Tool

ERIC Educational Resources Information Center

Carpenter, Donald A.; Snyder, Johnny; Slauson, Gayla Jo; Bridge, Morgan K.

2011-01-01

This paper presents a statistical analysis to support the notion that the Information Systems Analyst (ISA) exam can be used as a program assessment tool in addition to measuring student performance. It compares ISA exam scores earned by students in one particular Computer Information Systems program with scores earned by the same students on the…

Development of a patient-administered self-assessment tool (SATp) for follow-up of colorectal cancer patients in general practice.

PubMed

Ngune, Irene; Jiwa, Moyez; McManus, Alexandra; Hughes, Jeff; Parsons, Richard; Hodder, Rupert; Entriken, Fiona

2014-01-01

Treatment for colorectal cancer (CRC) may result in physical, social, and psychological needs that affect patients' quality of life post-treatment. A comprehensive assessment should be conducted to identify these needs in CRC patients post treatment, however, there is a lack of tools and processes available in general practice. This study aimed to develop a patient-completed needs screening tool that identifies potentially unmet physical, psychological, and social needs in CRC and facilitates consultation with a general practitioner (GP) to address these needs. The development of the self-assessment tool for patients (SATp) included a review of the literature; face and content validity with reference to an expert panel; psychometric testing including readability, internal consistency, and test-retest reliability; and usability in clinical practice. The SATp contains 25 questions. The tool had internal consistency (Cronbach's alpha 0.70-0.97), readability (reading ease 82.5%), and test-retest reliability (kappa 0.689-1.000). A total of 66 patients piloted the SATp. Participants were on average 69.2 (SD 9.9) years old and had a median follow-up period of 26.7 months. The SATp identified a total of 547 needs (median 7 needs/per patient; IQR [3-12.25]). Needs were categorised into social (175[32%]), psychological (175[32%]), and physical (197[36%]) domains. SATp is a reliable self-assessment tool useful for identifying CRC patient needs. Further testing of this tool for validity and usability is underway.

Development and validation of a fine-motor assessment tool for use with young children in a Chinese population.

PubMed

Siu, Andrew M H; Lai, Cynthia Y Y; Chiu, Amy S M; Yip, Calvin C K

2011-01-01

Most of the fine-motor assessment tools used in Hong Kong have been designed in Western countries, so there is a need to develop a standardized assessment which is relevant to the culture and daily living tasks of the local (that is, Chinese) population. This study aimed to (1) develop a fine-motor assessment tool (the Hong Kong Preschool Fine-Motor Developmental Assessment [HK-PFMDA]) for use with young children in a Chinese population and (2) examine the HK-PFMDA's psychometric properties. The HK-PFMDA was developed by a group of occupational therapists specializing in the area of developmental disabilities in Hong Kong. A panel of 21 experts reviewed the content validity of the instrument. Rasch item analysis was used to examine the model fit of items against the rating scale model, and to explore the dimensionality of the test. Intra- and interrater reliability, convergent validity, and criterion-related validity were examined. The participants included 783 children without disabilities, 45 with autistic spectrum disorder, and 35 with developmental delay. The Rasch analysis suggested that the 87-item HK-PFMDA had a unidimensional structure, as the items explained most (91.6%) of the variance. The HK-PFMDA demonstrated excellent intra- (ICC = .99) and interrater reliability (ICC = .99), and internal consistency (α ranging from .83 to .92). In terms of validity, the HK-PFMDA had significant positive correlations with both age and the convergent measures of the Peabody Developmental Motor Scales (PDMS-2). A set of normative data for local children aged from birth to 6 years was established. The HK-PFMDA has shown excellent psychometric properties and is suitable for clinical application by occupational therapists in the assessment of fine-motor skills development of young children in Chinese populations. Copyright © 2010 Elsevier Ltd. All rights reserved.

The Mental Disability Military Assessment Tool: A Reliable Tool for Determining Disability in Veterans with Post-traumatic Stress Disorder.

PubMed

Fokkens, Andrea S; Groothoff, Johan W; van der Klink, Jac J L; Popping, Roel; Stewart, Roy E; van de Ven, Lex; Brouwer, Sandra; Tuinstra, Jolanda

2015-09-01

An assessment tool was developed to assess disability in veterans who suffer from post-traumatic stress disorder (PTSD) due to a military mission. The objective of this study was to determine the reliability, intra-rater and inter-rater variation of the Mental Disability Military (MDM) assessment tool. Twenty-four assessment interviews of veterans with an insurance physician were videotaped. Each videotaped interview was assessed by a group of five independent raters on limitations of the veterans using the MDM assessment tool. After 2 months the raters repeated this procedure. Next the intra-rater and inter-rater variation was assessed with an adjusted version of AG09 computing weighted percentage agreement. The results of this study showed that both the intra-rater variation and inter-rater variation on the ten subcategories of the MDM assessment tool were small, with an agreement of 84-100% within raters and 93-100% between raters. The MDM assessment tool proves to be a reliable instrument to measure PTSD limitations in functioning in Dutch military veterans who apply for disability compensation. Further research is needed to assess the validity of this instrument.

Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT) among professionals in Dutch disease-management partnerships.

PubMed

Cramm, Jane M; Strating, Mathilde Mh; Nieboer, Anna P

2011-06-30

The extent to which partnership synergy is created within quality improvement programmes in the Netherlands is unknown. In this article, we describe the psychometric testing of the Partnership Self-Assessment Tool (PSAT) among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability. The Dutch version of the PSAT was tested in twenty-two disease-management partnerships with 218 professionals. We tested the instrument by means of structural equation modelling, and examined its validity and reliability. After eliminating 14 items, the confirmatory factor analyses revealed good indices of fit with the resulting 15-item PSAT-Short version (PSAT-S). Internal consistency as represented by Cronbach's alpha ranged from acceptable (0.75) for the 'efficiency' subscale to excellent for the 'leadership' subscale (0.87). Convergent validity was provided with high correlations of the partnership dimensions and partnership synergy (ranged from 0.512 to 0.609) and high correlations with chronic illness care (ranged from 0.447 to 0.329). The psychometric properties and convergent validity of the PSAT-S were satisfactory rendering it a valid and reliable instrument for assessing partnership synergy and its dimensions of partnership functioning.

A tool to assess sex-gender when selecting health research projects.

PubMed

Tomás, Concepción; Yago, Teresa; Eguiluz, Mercedes; Samitier, M A Luisa; Oliveros, Teresa; Palacios, Gemma

2015-04-01

To validate the questionnaire "Gender Perspective in Health Research" (GPIHR) to assess the inclusion of gender perspective in research projects. Validation study in two stages. Feasibility was analysed in the first, and reliability, internal consistence and validity in the second. Aragón Institute of Health Science, Aragón, Spain. GPIHR was applied to 118 research projects funded in national and international competitive tenders from 2003 to 2012. Analysis of inter- and intra-observer reliability with Kappa index and internal consistency with Cronbach's alpha. Content validity analysed through literature review and construct validity with an exploratory factor analysis. Validated GPIHR has 10 questions: 3 in the introduction, 1 for objectives, 3 for methodology and 3 for research purpose. Average time of application was 13min Inter-observer reliability (Kappa) varied between 0.35 and 0.94 and intra-observer between 0.40 and 0.94. Theoretical construct is supported in the literature. Factor analysis identifies three levels of GP inclusion: "difference by sex", "gender sensitive" and "feminist research" with an internal consistency of 0.64, 0.87 and 0.81, respectively, which explain 74.78% of variance. GPIHR questionnaire is a valid tool to assess GP and useful for those researchers who would like to include GP in their projects. Copyright © 2014 Elsevier España, S.L.U. All rights reserved.

Quantitative analysis of the rubric as an assessment tool: an empirical study of student peer-group rating

NASA Astrophysics Data System (ADS)

Hafner, John C.; Hafner, Patti M.

2003-12-01

Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool in the hands of the students. This study focuses on the validity and reliability of the rubric as an assessment tool for student peer-group evaluation in an effort to further explore the use and effectiveness of the rubric. A total of 1577 peer-group ratings using a rubric for an oral presentation was used in this 3-year study involving 107 college biology students. A quantitative analysis of the rubric used in this study shows that it is used consistently by both students and the instructor across the study years. Moreover, the rubric appears to be 'gender neutral' and the students' academic strength has no significant bearing on the way that they employ the rubric. A significant, one-to-one relationship (slope = 1.0) between the instructor's assessment and the students' rating is seen across all years using the rubric. A generalizability study yields estimates of inter-rater reliability of moderate values across all years and allows for the estimation of variance components. Taken together, these data indicate that the general form and evaluative criteria of the rubric are clear and that the rubric is a useful assessment tool for peer-group (and self-) assessment by students. To our knowledge, these data provide the first statistical documentation of the validity and reliability of the rubric for student peer-group assessment.

Evaluation of the Validity and Response Burden of Patient Self-Report Measures of the Pain Assessment Screening Tool and Outcomes Registry (PASTOR).

PubMed

Cook, Karon F; Kallen, Michael A; Buckenmaier, Chester; Flynn, Diane M; Hanling, Steven R; Collins, Teresa S; Joltes, Kristin; Kwon, Kyung; Medina-Torne, Sheila; Nahavandi, Parisa; Suen, Joshua; Gershon, Richard

2017-07-01

In 2009, the Army Pain Management Task Force was chartered. On the basis of their findings, the Department of Defense recommended a comprehensive pain management strategy that included development of a standardized pain assessment system that would collect patient-reported outcomes data to inform the patient-provider clinical encounter. The result was the Pain Assessment Screening Tool and Outcomes Registry (PASTOR). The purpose of this study was to assess the validity and response burden of the patient-reported outcome measures in PASTOR. Data for analyses were collected from 681 individuals who completed PASTOR at baseline and follow-up as part of their routine clinical care. The survey tool included self-report measures of pain severity and pain interference (measured using the National Institutes of Health Patient-Reported Outcome Measurement Information System [PROMIS] and the Defense and Veterans Pain Rating scale). PROMIS measures of pain correlates also were administered. Validation analyses included estimation of score associations among measures, comparison of scores of known groups, responsiveness, ceiling and floor effects, and response burden. Results of psychometric testing provided substantial evidence for the validity of PASTOR self-report measures in this population. Expected associations among scores largely supported the concurrent validity of the measures. Scores effectively distinguished among respondents on the basis of their self-reported impressions of general health. PROMIS measures were administered using computer adaptive testing and each, on average, required less than 1 minute to administer. Statistical and graphical analyses demonstrated the responsiveness of PASTOR measures over time. Reprint & Copyright © 2017 Association of Military Surgeons of the U.S.

Comparative assessment of three standardized robotic surgery training methods.

PubMed

Hung, Andrew J; Jayaratna, Isuru S; Teruya, Kara; Desai, Mihir M; Gill, Inderbir S; Goh, Alvin C

2013-10-01

To evaluate three standardized robotic surgery training methods, inanimate, virtual reality and in vivo, for their construct validity. To explore the concept of cross-method validity, where the relative performance of each method is compared. Robotic surgical skills were prospectively assessed in 49 participating surgeons who were classified as follows: 'novice/trainee': urology residents, previous experience <30 cases (n = 38) and 'experts': faculty surgeons, previous experience ≥30 cases (n = 11). Three standardized, validated training methods were used: (i) structured inanimate tasks; (ii) virtual reality exercises on the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA); and (iii) a standardized robotic surgical task in a live porcine model with performance graded by the Global Evaluative Assessment of Robotic Skills (GEARS) tool. A Kruskal-Wallis test was used to evaluate performance differences between novices and experts (construct validity). Spearman's correlation coefficient (ρ) was used to measure the association of performance across inanimate, simulation and in vivo methods (cross-method validity). Novice and expert surgeons had previously performed a median (range) of 0 (0-20) and 300 (30-2000) robotic cases, respectively (P < 0.001). Construct validity: experts consistently outperformed residents with all three methods (P < 0.001). Cross-method validity: overall performance of inanimate tasks significantly correlated with virtual reality robotic performance (ρ = -0.7, P < 0.001) and in vivo robotic performance based on GEARS (ρ = -0.8, P < 0.0001). Virtual reality performance and in vivo tissue performance were also found to be strongly correlated (ρ = 0.6, P < 0.001). We propose the novel concept of cross-method validity, which may provide a method of evaluating the relative value of various forms of skills education and assessment. We externally confirmed the construct validity of each featured training tool. © 2013 BJU International.

Construction and Validation of a Questionnaire about Heart Failure Patients' Knowledge of Their Disease

PubMed Central

Bonin, Christiani Decker Batista; dos Santos, Rafaella Zulianello; Ghisi, Gabriela Lima de Melo; Vieira, Ariany Marques; Amboni, Ricardo; Benetti, Magnus

2014-01-01

Background The lack of tools to measure heart failure patients' knowledge about their syndrome when participating in rehabilitation programs demonstrates the need for specific recommendations regarding the amount or content of information required. Objectives To develop and validate a questionnaire to assess heart failure patients' knowledge about their syndrome when participating in cardiac rehabilitation programs. Methods The tool was developed based on the Coronary Artery Disease Education Questionnaire and applied to 96 patients with heart failure, with a mean age of 60.22 ± 11.6 years, 64% being men. Reproducibility was obtained via the intraclass correlation coefficient, using the test-retest method. Internal consistency was assessed by use of Cronbach's alpha, and construct validity, by use of exploratory factor analysis. Results The final version of the tool had 19 questions arranged in ten areas of importance for patient education. The proposed questionnaire had a clarity index of 8.94 ± 0.83. The intraclass correlation coefficient was 0.856, and Cronbach's alpha, 0.749. Factor analysis revealed five factors associated with the knowledge areas. Comparing the final scores with the characteristics of the population evidenced that low educational level and low income are significantly associated with low levels of knowledge. Conclusion The instrument has satisfactory clarity and validity indices, and can be used to assess the heart failure patients' knowledge about their syndrome when participating in cardiac rehabilitation programs. PMID:24652054

Developing and testing a measurement tool for assessing predictors of breakfast consumption based on a health promotion model.

PubMed

Dehdari, Tahereh; Rahimi, Tahereh; Aryaeian, Naheed; Gohari, Mahmood Reza; Esfeh, Jabiz Modaresi

2014-01-01

To develop an instrument for measuring Health Promotion Model constructs in terms of breakfast consumption, and to identify the constructs that were predictors of breakfast consumption among Iranian female students. A questionnaire on Health Promotion Model variables was developed and potential predictors of breakfast consumption were assessed using this tool. One hundred female students, mean age 13 years (SD ± 1.2 years). Two middle schools from moderate-income areas in Qom, Iran. Health Promotion Model variables were assessed using a 58-item questionnaire. Breakfast consumption was also measured. Internal consistency (Cronbach alpha), content validity index, content validity ratio, multiple linear regression using stepwise method, and Pearson correlation. Content validity index and content validity ratio scores of the developed scale items were 0.89 and 0.93, respectively. Internal consistencies (range, .74-.91) of subscales were acceptable. Prior related behaviors, perceived barriers, self-efficacy, and competing demand and preferences were 4 constructs that could predict 63% variance of breakfast frequency per week among subjects. The instrument developed in this study may be a useful tool for researchers to explore factors affecting breakfast consumption among students. Students with a high level of self-efficacy, more prior related behavior, fewer perceived barriers, and fewer competing demands were most likely to regularly consume breakfast. Copyright © 2014 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Undergraduate nursing students' perspectives on clinical assessment at transition to practice.

PubMed

Wu, Xi Vivien; Wang, Wenru; Pua, Lay Hoon; Heng, Doreen Gek Noi; Enskär, Karin

2015-01-01

Assessment of clinical competence requires explicitly defined standards meeting the national standards of the nursing profession. This is a complex process because of the diverse nature of nursing practice. To explore the perceptions of final-year undergraduate nursing students regarding clinical assessment at transition to practice. An exploratory qualitative approach was adopted. Twenty-four students participated in three focus group discussions. Thematic analysis was conducted. Five themes emerged: the need for a valid and reliable clinical assessment tool, the need for a flexible style of reflection and specific feedback, the dynamic clinical learning environment, students' efforts in learning and assessment, and the unclear support system for preceptors. Workload, time, resource availability, adequate preparation of preceptors, and the provision of valid and reliable clinical assessment tools were deemed to influence the quality of students' clinical learning and assessment. Nursing leadership in hospitals and educational institutions has a joint responsibility in shaping the clinical learning environment and providing clinical assessments for the students.

Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis.

PubMed

Brydges, Ryan; Hatala, Rose; Zendejas, Benjamin; Erwin, Patricia J; Cook, David A

2015-02-01

To examine the evidence supporting the use of simulation-based assessments as surrogates for patient-related outcomes assessed in the workplace. The authors systematically searched MEDLINE, EMBASE, Scopus, and key journals through February 26, 2013. They included original studies that assessed health professionals and trainees using simulation and then linked those scores with patient-related outcomes assessed in the workplace. Two reviewers independently extracted information on participants, tasks, validity evidence, study quality, patient-related and simulation-based outcomes, and magnitude of correlation. All correlations were pooled using random-effects meta-analysis. Of 11,628 potentially relevant articles, the 33 included studies enrolled 1,203 participants, including postgraduate physicians (n = 24 studies), practicing physicians (n = 8), medical students (n = 6), dentists (n = 2), and nurses (n = 1). The pooled correlation for provider behaviors was 0.51 (95% confidence interval [CI], 0.38 to 0.62; n = 27 studies); for time behaviors, 0.44 (95% CI, 0.15 to 0.66; n = 7); and for patient outcomes, 0.24 (95% CI, -0.02 to 0.47; n = 5). Most reported validity evidence was favorable, though studies often included only correlational evidence. Validity evidence of internal structure (n = 13 studies), content (n = 12), response process (n = 2), and consequences (n = 1) were reported less often. Three tools showed large pooled correlations and favorable (albeit incomplete) validity evidence. Simulation-based assessments often correlate positively with patient-related outcomes. Although these surrogates are imperfect, tools with established validity evidence may replace workplace-based assessments for evaluating select procedural skills.

Assessing peristomal skin changes in ostomy patients: validation of the Ostomy Skin Tool.

PubMed

Jemec, G B; Martins, L; Claessens, I; Ayello, E A; Hansen, A S; Poulsen, L H; Sibbald, R G

2011-02-01

Peristomal skin problems are common and are treated by a variety of health professionals. Clear and consistent communication among these professionals is therefore particularly important. The Ostomy Skin Tool (OST) is a new assessment instrument for the extent and severity of peristomal skin conditions. Formal tests of reliability and validity are necessary for its use in clinical practice, research, and education. To estimate inter- and intra nurse assessment variability of the OST and validity by comparison to a 'gold standard' (GS) defined by an expert panel. Thirty photographs of peristomal skin were presented twice to 20 ostomy care nurses--10 from Denmark (DK) and 10 from Spain (ES)--to determine intra- and inter nurse assessment variability. The same photographs were presented to an international group of experts (dermatologist and ostomy care nurses), to establish a GS for comparison and validation of the results. A high intra-nurse assessment agreement, κ=0·84, was found with no differences in the intra-nurse assessments from the two groups of nurses (DK and ES). The inter-nurse assessment agreement was 'moderate to good', κ=0·54, with the agreement between the experts higher, κ=0·70. A high correlation between the scores from the nurses and the GS were seen in the lower part of the two scales [Discoloration, Erosion, Tissue overgrowth (DET) score<7)]. The study supported the validity of the OST. It is suggested that a categorical scale can be used to illustrate the severity of the DET scores. © 2011 The Authors. BJD © 2011 British Association of Dermatologists.

Quantifying Risk for Anxiety Disorders in Preschool Children: A Machine Learning Approach.

PubMed

Carpenter, Kimberly L H; Sprechmann, Pablo; Calderbank, Robert; Sapiro, Guillermo; Egger, Helen L

2016-01-01

Early childhood anxiety disorders are common, impairing, and predictive of anxiety and mood disorders later in childhood. Epidemiological studies over the last decade find that the prevalence of impairing anxiety disorders in preschool children ranges from 0.3% to 6.5%. Yet, less than 15% of young children with an impairing anxiety disorder receive a mental health evaluation or treatment. One possible reason for the low rate of care for anxious preschoolers is the lack of affordable, timely, reliable and valid tools for identifying young children with clinically significant anxiety. Diagnostic interviews assessing psychopathology in young children require intensive training, take hours to administer and code, and are not available for use outside of research settings. The Preschool Age Psychiatric Assessment (PAPA) is a reliable and valid structured diagnostic parent-report interview for assessing psychopathology, including anxiety disorders, in 2 to 5 year old children. In this paper, we apply machine-learning tools to already collected PAPA data from two large community studies to identify sub-sets of PAPA items that could be developed into an efficient, reliable, and valid screening tool to assess a young child's risk for an anxiety disorder. Using machine learning, we were able to decrease by an order of magnitude the number of items needed to identify a child who is at risk for an anxiety disorder with an accuracy of over 96% for both generalized anxiety disorder (GAD) and separation anxiety disorder (SAD). Additionally, rather than considering GAD or SAD as discrete/binary entities, we present a continuous risk score representing the child's risk of meeting criteria for GAD or SAD. Identification of a short question-set that assesses risk for an anxiety disorder could be a first step toward development and validation of a relatively short screening tool feasible for use in pediatric clinics and daycare/preschool settings.

Quantifying Risk for Anxiety Disorders in Preschool Children: A Machine Learning Approach

PubMed Central

Calderbank, Robert; Sapiro, Guillermo; Egger, Helen L.

2016-01-01

Early childhood anxiety disorders are common, impairing, and predictive of anxiety and mood disorders later in childhood. Epidemiological studies over the last decade find that the prevalence of impairing anxiety disorders in preschool children ranges from 0.3% to 6.5%. Yet, less than 15% of young children with an impairing anxiety disorder receive a mental health evaluation or treatment. One possible reason for the low rate of care for anxious preschoolers is the lack of affordable, timely, reliable and valid tools for identifying young children with clinically significant anxiety. Diagnostic interviews assessing psychopathology in young children require intensive training, take hours to administer and code, and are not available for use outside of research settings. The Preschool Age Psychiatric Assessment (PAPA) is a reliable and valid structured diagnostic parent-report interview for assessing psychopathology, including anxiety disorders, in 2 to 5 year old children. In this paper, we apply machine-learning tools to already collected PAPA data from two large community studies to identify sub-sets of PAPA items that could be developed into an efficient, reliable, and valid screening tool to assess a young child’s risk for an anxiety disorder. Using machine learning, we were able to decrease by an order of magnitude the number of items needed to identify a child who is at risk for an anxiety disorder with an accuracy of over 96% for both generalized anxiety disorder (GAD) and separation anxiety disorder (SAD). Additionally, rather than considering GAD or SAD as discrete/binary entities, we present a continuous risk score representing the child’s risk of meeting criteria for GAD or SAD. Identification of a short question-set that assesses risk for an anxiety disorder could be a first step toward development and validation of a relatively short screening tool feasible for use in pediatric clinics and daycare/preschool settings. PMID:27880812

The THINC-Integrated Tool (THINC-it) Screening Assessment for Cognitive Dysfunction: Validation in Patients With Major Depressive Disorder.

PubMed

McIntyre, Roger S; Best, Michael W; Bowie, Christopher R; Carmona, Nicole E; Cha, Danielle S; Lee, Yena; Subramaniapillai, Mehala; Mansur, Rodrigo B; Barry, Harry; Baune, Bernhard T; Culpepper, Larry; Fossati, Philippe; Greer, Tracy L; Harmer, Catherine; Klag, Esther; Lam, Raymond W; Wittchen, Hans-Ulrich; Harrison, John

2017-07-01

To validate the THINC-integrated tool (THINC-it)-a freely available, patient-administered, computerized screening tool integrating subjective and objective measures of cognitive function in adults with major depressive disorder (MDD). Subjects aged 18 to 65 years (n = 100) with recurrent MDD experiencing a major depressive episode of at least moderate severity were evaluated and compared to age-, sex-, and education-matched healthy controls (n = 100). Between January and June 2016, subjects completed the THINC-it, which includes variants of the Choice Reaction Time Identification Task (IDN), One-Back Test, Digit Symbol Substitution Test, Trail Making Test-Part B, and the Perceived Deficits Questionnaire for Depression-5-item (PDQ-5-D). The THINC-it required approximately 10 to 15 minutes for administration and was capable of detecting cognitive deficits in adults with MDD. A total of 44.4% of adults with MDD exhibited cognitive performance at ≥ 1.0 SD below that of healthy controls on standardized mean scores of the THINC-it. Concurrent validity of the overall tool, based on a calculated composite score, was acceptable (r = 0.539, P < .001). Concurrent validity of the component tests ranged from -0.083 (IDN) to 0.929 (PDQ-5-D). Qualitative survey results indicated that there was a high level of satisfaction and perceived value in administering the THINC-it regarding its impact on the appropriateness and quality of care being received. The THINC-it is a valid and sensitive tool for detecting cognitive dysfunction in adults with MDD that is free, easy to use, and rapidly administered. The THINC-it should be incorporated into the assessment and measurement of all patients with MDD, particularly among those with enduring functional impairment. ClinicalTrials.gov identifier: NCT02508493. © Copyright 2017 Physicians Postgraduate Press, Inc.

Exploring the Reliability and Validity of the Social-Moral Awareness Test

ERIC Educational Resources Information Center

Livesey, Alexandra; Dodd, Karen; Pote, Helen; Marlow, Elizabeth

2012-01-01

Background: The aim of the study was to explore the validity of the social-moral awareness test (SMAT) a measure designed for assessing socio-moral rule knowledge and reasoning in people with learning disabilities. Comparisons between Theory of Mind and socio-moral reasoning allowed the exploration of construct validity of the tool. Factor…

cluML: A markup language for clustering and cluster validity assessment of microarray data.

PubMed

Bolshakova, Nadia; Cunningham, Pádraig

2005-01-01

cluML is a new markup language for microarray data clustering and cluster validity assessment. The XML-based format has been designed to address some of the limitations observed in traditional formats, such as inability to store multiple clustering (including biclustering) and validation results within a dataset. cluML is an effective tool to support biomedical knowledge representation in gene expression data analysis. Although cluML was developed for DNA microarray analysis applications, it can be effectively used for the representation of clustering and for the validation of other biomedical and physical data that has no limitations.

High-resolution computational algorithms for simulating offshore wind turbines and farms: Model development and validation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Calderer, Antoni; Yang, Xiaolei; Angelidis, Dionysios

2015-10-30

The present project involves the development of modeling and analysis design tools for assessing offshore wind turbine technologies. The computational tools developed herein are able to resolve the effects of the coupled interaction of atmospheric turbulence and ocean waves on aerodynamic performance and structural stability and reliability of offshore wind turbines and farms. Laboratory scale experiments have been carried out to derive data sets for validating the computational models.

Validation of an Instrument for Measuring Students' Understanding of Interdisciplinary Science in Grades 4-8 over Multiple Semesters: A Rasch Measurement Study

ERIC Educational Resources Information Center

Yang, Yang; He, Peng; Liu, Xiufeng

2018-01-01

So far, not enough effort has been invested in developing reliable, valid, and engaging assessments in school science, especially assessment of interdisciplinary science based on the new Next Generation Science Standards (NGSS). Furthermore, previous tools rely mostly on multiple-choice items and evaluation of student outcome is linked only to…

The Development and Validation of the "Academic Spoken English Strategies Survey (ASESS)" for Non-Native English Speaking Graduate Students

ERIC Educational Resources Information Center

Schroeder, Rui M.

2016-01-01

This study reports on the three-year development and validation of a new assessment tool--the Academic Spoken English Strategies Survey (ASESS). The questionnaire is the first of its kind to assess the listening and speaking strategy use of non-native English speaking (NNES) graduate students. A combination of sources was used to develop the…

Development and validation of the Italian version of the Mobile Application Rating Scale and its generalisability to apps targeting primary prevention.

PubMed

Domnich, Alexander; Arata, Lucia; Amicizia, Daniela; Signori, Alessio; Patrick, Bernard; Stoyanov, Stoyan; Hides, Leanne; Gasparini, Roberto; Panatto, Donatella

2016-07-07

A growing body of literature affirms the usefulness of mobile technologies, including mobile applications (apps), in the primary prevention field. The quality of health apps, which today number in the thousands, is a crucial parameter, as it may affect health-related decision-making and outcomes among app end-users. The mobile application rating scale (MARS) has recently been developed to evaluate the quality of such apps, and has shown good psychometric properties. Since there is no standardised tool for assessing the apps available in Italian app stores, the present study developed and validated an Italian version of MARS in apps targeting primary prevention. The original 23-item version of the MARS assesses mobile app quality in four objective quality dimensions (engagement, functionality, aesthetics, information) and one subjective dimension. Validation of this tool involved several steps; the universalist approach to achieving equivalence was adopted. Following two backward translations, a reconciled Italian version of MARS was produced and compared with the original scale. On the basis of sample size estimation, 48 apps from three major app stores were downloaded; the first 5 were used for piloting, while the remaining 43 were used in the main study in order to assess the psychometric properties of the scale. The apps were assessed by two raters, each working independently. The psychometric properties of the final version of the scale was assessed including the inter-rater reliability, internal consistency, convergent, divergent and concurrent validities. The intralingual equivalence of the Italian version of the MARS was confirmed by the authors of the original scale. A total of 43 apps targeting primary prevention were tested. The MARS displayed acceptable psychometric properties. The MARS total score showed an excellent level of both inter-rater agreement (intra-class correlation coefficient of .96) and internal consistency (Cronbach's α of .90 and .91 for the two raters, respectively). Other types of validity, including convergent, divergent, discriminative, known-groups and scalability, were also established. The Italian version of MARS is a valid and reliable tool for assessing the health-related primary prevention apps available in Italian app stores.

The predictive validity of common risk assessment tools in men with intellectual disabilities and problematic sexual behaviors.

PubMed

Fedoroff, J Paul; Richards, Deborah; Ranger, Rebekah; Curry, Susan

2016-10-01

This CIHR-funded study examined whether certain current risk assessment tools were effective in appraising risk of recidivism in a sample of sex offenders with intellectual disabilities (ID). Fifty men with ID who had engaged in problematic sexual behavior (PSB) were followed for an average of 2.5 years. Recidivism was defined and measured as any illegal or problematic behavior, as well as any problematic but not necessarily illegal behavior. At the beginning of the study, each participant was rated on two risk assessment tools: the Violence Risk Appraisal Guide (VRAG) and the Sex Offender Risk Appraisal Guide (SORAG). During each month of follow-up, participants were also rated on the Short-Dynamic Risk Scale (SDRS), an assessment tool intended to measure the risk of future problematic behaviors. Data was analyzed using t-tests, Cohen's d and area under the curve (AUC) to test predictive validity of the assessment tools. Using the AUC, results showed that the VRAG was predictive of sexual (AUC=0.74), sexual and/or violent (AUC=0.71) and of any criminally chargeable event (AUC=0.69). The SORAG was only significantly predictive of sexual events (AUC=0.70) and the SDRS was predictive of violent events (AUC=0.71). The t-test and Cohen's d analyses, which are less robust to deviations from the assumptions of normal and continuous distribution than AUC, did not yield significant results in each category, and therefore, while the results of this study suggest that the VRAG and the SORAG may be effective tools in measuring the short term risk of sexual recidivism; and the VRAG and SDRS may be effective tools in appraising long term risk of sexual and/or violent recidivism in this population, it should be used with caution. Regardless of the assessment tool used, risk assessments should take into account the differences between sex offenders with and without ID to ensure effective measurement. Copyright © 2016. Published by Elsevier Ltd.

The Shoulder Objective Practical Assessment Tool: Evaluation of a New Tool Assessing Residents Learning in Diagnostic Shoulder Arthroscopy.

PubMed

Talbot, Christopher L; Holt, Edward M; Gooding, Benjamin W T; Tennent, Thomas D; Foden, Philip

2015-08-01

To design and validate an objective practical assessment tool for diagnostic shoulder arthroscopy that would provide residents with a method to evaluate their progression in this field of surgery and to identify specific learning needs. We designed and evaluated the shoulder Objective Practical Assessment Tool (OPAT). The shoulder OPAT was designed by us, and scoring domains were created using a Delphi process. The shoulder OPAT was trialed by members of the British Elbow & Shoulder Society Education Committee for internal consistency and ease of use before being offered to other trainers and residents. Inter-rater reliability and intrarater reliability were calculated. One hundred forty orthopaedic residents, of varying seniority, within 5 training regions in the United Kingdom, were questioned regarding the tool. A pilot study of 6 residents was undertaken. Internal consistency was 0.77 (standardized Cronbach α). Inter-rater reliability was 0.60, and intrarater reliability was 0.82. The Spearman correlation coefficient (r) between the global summary score for the shoulder OPAT and the current assessment tool used in postgraduate training for orthopaedic residents undertaking diagnostic shoulder arthroscopy equaled 0.74. Of the residents, 82% agreed or strongly agreed when asked if the shoulder OPAT would be a useful tool in monitoring progression and 72% agreed or strongly agreed with the introduction of the shoulder OPAT within the orthopaedic domain. This study shows that the shoulder OPAT fulfills several aspects of reliability and validity when tested. Despite the inter-rater reliability being 0.60, we believe that the shoulder OPAT has the potential to play a role alongside the current assessment tool in the training of orthopaedic residents. The shoulder OPAT can be used to assess residents during shoulder arthroscopy and has the potential for use in medical education, as well as arthroscopic skills training in the operating theater. Copyright © 2015 Arthroscopy Association of North America. Published by Elsevier Inc. All rights reserved.

Validation of Nutritional Risk Screening-2002 in a Hospitalized Adult Population.

PubMed

Bolayir, Başak; Arik, Güneş; Yeşil, Yusuf; Kuyumcu, Mehmet Emin; Varan, Hacer Doğan; Kara, Özgür; Güngör, Anil Evrim; Yavuz, Burcu Balam; Cankurtaran, Mustafa; Halil, Meltem Gülhan

2018-03-30

Malnutrition in hospitalized patients is a serious problem and is associated with a number of adverse outcomes. The Nutritional Risk Screening-2002 (NRS-2002) tool was designed to identify patients at nutrition risk. The validation of NRS-2002 compared with detailed clinical assessment of nutrition status was not studied before in hospitalized Turkish adults. The aim of this study is to determine validity, sensitivity, and specificity of the Turkish version of NRS-2002 in a hospitalized adult population. A total of 271 consecutive hospitalized patients aged >18 years admitted to surgical and medical wards of a university hospital in Turkey were included in this single-center non interventional validity study. Assessment by geriatricians was used as the reference method. Two geriatricians experienced in the field of malnutrition interpreted the patients' nutrition status after the evaluation of several parameters. Patients were divided into "at nutrition risk" and "not at nutrition risk" groups by geriatricians. Concordance between the 2 geriatricians' clinical assessments was analyzed by κ statistics. Excellent concordance was found; therefore, the first geriatrician's decisions were accepted as the gold standard. The correlation of nutrition status of the patients, determined with NRS-2002 and experienced geriatrician's decisions, was evaluated for the validity. NRS-2002 has a sensitivity of 88% and specificity of 92% when compared with professional assessment. The positive and negative predictive values were 87% and 92%, respectively. Testretest agreement was excellent as represented by a κ coefficient of 0.956. NRS-2002 is a valid tool to assess malnutrition risk in Turkish hospitalized patients. © 2018 American Society for Parenteral and Enteral Nutrition.

Validity and Reliability of a General Nutrition Knowledge Questionnaire for Japanese Adults.

PubMed

Matsumoto, Mai; Tanaka, Rie; Ikemoto, Shinji

2017-01-01

Nutrition knowledge is necessary for individuals to adopt appropriate dietary habits, and needs to be evaluated before nutrition education is provided. However, there is no tool to assess general nutrition knowledge of adults in Japan. Our aims were to determine the validity and reliability of a general nutrition knowledge questionnaire for Japanese adults. We developed the pilot version of the Japanese general nutrition knowledge questionnaire (JGNKQ) and administered the pilot study to assess content validity and internal reliability to 1,182 Japanese adults aged 18-64 y. The JGNKQ was further modified based on the pilot study and the final version consisted of 5 sections and 147 items. The JGNKQ was administered to female undergraduate Japanese students in their senior year twice in 2015 to assess construct validity and test-retest reliability. Ninety-six students majoring in nutrition and 44 students in other majors who studied at the same university completed the first questionnaire. Seventy-five students completed the questionnaire twice. The responses from the first questionnaire and both questionnaires were used to assess construct validity and test-retest reliability, respectively. The students in nutrition major had significantly higher scores than the students in other majors on all sections of the questionnaire (p=0.000); therefore, the questionnaire had good construct validity. The test-retest reliability correlation coefficient value of overall and each section except "The use of dietary information to make dietary choices" were 0.75, 0.67, 0.67, 0.68 and 0.61, respectively. We suggest that the JGNKQ is an effective tool to assess the nutrition knowledge level of Japanese adults.

Development and Validation of the Children's Voice Handicap Index-10 for Parents.

PubMed

Ricci-Maccarini, Andrea; De Maio, Vincenzo; Murry, Thomas; Schindler, Antonio

2016-01-01

The Children's Voice Handicap Index-10 (CVHI-10) was introduced as a tool for self-assessment of children's dysphonia. However, in the management of children with voice disorders, both parents' and children's perspectives play an important role. Because a self-tool including both a children's and a parents' version does not exist yet, the aim of the study was to develop and validate an assessment tool which parallels the CVHI-10 for parents to assess the level of voice handicap in their child's voice. Observational, prospective, cross-sectional study. To develop a CVHI-10 for parents, called "CVHI-10-P", the CVHI-10 items were adapted to reflect parents' responses about their child. Fifty-five children aged 7-12 years completed the CVHI-10, whereas their parents completed the CVHI-10-P. Each child's voice was also perceptually assessed by an otolaryngologist using the Grade Breathness Roughness (GRB) scale. Fifty-one of the 55 children underwent voice therapy (VT) and were assessed afterward using the GRB, CVHI-10, and CVHI-10-P. CVHI-10-P internal consistency was satisfactory (Cronbach alpha = .78). Correlation between CVHI-10-P and CVHI-10 was moderate (r = 0.37). CVHI-10-P total scores were lower than CVHI-10 scores in most of the cases. Single-item mean scores were always lower in CVHI-10-P compared with CVHI-10, with the exception of the only one item of the CVHI-10-P that directly involves the parent's experience (item 10). Data gained from one tool are not directly related to the other, suggesting that these two tools appraise the child's voice handicap from different perspectives. The overall perceptual assessment scores of the 51 children after VT significantly improved. There was a statistically significant reduction of the total scores and for each item in CVHI-10 and CVHI-10-P after VT. These data support the adoption of the CVHI-10-P as an assessment tool and an outcome measure for management of children's voice disorders. CVHI-10-P is a valid tool to appraise parents' perspective of their child's voice disorder. The use of the CVHI-10 and the CVHI-10-P is recommended for objectively determining the level of voice handicap in children by parents and child. Copyright © 2016 The Voice Foundation. Published by Elsevier Inc. All rights reserved.

A systematic review of clinical assessment for undergraduate nursing students.

PubMed

Wu, Xi Vivien; Enskär, Karin; Lee, Cindy Ching Siang; Wang, Wenru

2015-02-01

Consolidated clinical practicum prepares pre-registration nursing students to function as beginning practitioners. The clinical competencies of final-year nursing students provide a key indication of professional standards of practice and patient safety. Thus, clinical assessment of nursing students is a crucial issue for educators and administrators. The aim of this systematic review was to explore the clinical competency assessment for undergraduate nursing students. PubMed, CINAHL, ScienceDirect, Web of Science, and EBSCO were systematically searched from January 2000 to December 2013. The systematic review was in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Published quantitative and qualitative studies that examined clinical assessment practices and tools used in clinical nursing education were retrieved. Quality assessment, data extraction, and analysis were completed on all included studies. This review screened 2073 titles, abstracts and full-text records, resulting in 33 included studies. Two reviewers assessed the quality of the included studies. Fourteen quantitative and qualitative studies were identified for this evaluation. The evidence was ordered into emergent themes; the overarching themes were current practices in clinical assessment, issues of learning and assessment, development of assessment tools, and reliability and validity of assessment tools. There is a need to develop a holistic clinical assessment tool with reasonable level of validity and reliability. Clinical assessment is a robust activity and requires collaboration between clinical partners and academia to enhance the clinical experiences of students, the professional development of preceptors, and the clinical credibility of academics. Copyright © 2014 Elsevier Ltd. All rights reserved.

Validation of the Greek translation of the Dundee Ready Education Environment Measure (DREEM).

PubMed

Dimoliatis, I D K; Vasilaki, E; Anastassopoulos, P; Ioannidis, J P A; Roff, S

2010-04-01

The educational environment makes an important contribution to student learning. The DREEM questionnaire is a validated tool assessing the environment. To translate and validate the DREEM into Greek. Forward translations from English were produced by three independent Greek translators and then back translations by five independent bilingual translators. The Greek DREEM.v0 that was produced was administered to 831 undergraduate students from six Greek medical schools. Cronbach's alpha and test-retest correlation were used to evaluate reliability and factor analysis was used to assess validity. Questions that increased alpha if deleted and/or sorted unexpectedly in factor analysis were further checked through two focus groups. Questionnaires were returned by 487 respondents (59%), who were representative of all surveyed students by gender but not by year of study or medical school. The instrument's overall alpha was 0.90, and for the learning, teachers, academic, atmosphere and social subscales the alphas were 0.79 (expected 0.69), 0.78 (0.67), 0.69 (0.60), 0.68 (0.69), 0.48 (0.57), respectively. In a subset of the whole sample, test and retest alphas were both 0.90, and mean item scores highly correlated (p<0.001). Factor analysis produced meaningful subscales but not always matching the original ones. Focus group evaluation revealed possible misunderstanding for questions 17, 25, 29 and 38, which were revised in the DREEM.Gr.v1. The group mean overall scale score was 107.7 (SD 20.2), with significant differences across medical schools (p<0.001). Alphas and test-retest correlation suggest the Greek translated and validated DREEM scale is a reliable tool for assessing the medical education environment and for informing policy. Factor analysis and focus group input suggest it is a valid tool. Reasonable school differences suggest the instrument's sensitivity.

Automated Pressure Injury Risk Assessment System Incorporated Into an Electronic Health Record System.

PubMed

Jin, Yinji; Jin, Taixian; Lee, Sun-Mi

Pressure injury risk assessment is the first step toward preventing pressure injuries, but traditional assessment tools are time-consuming, resulting in work overload and fatigue for nurses. The objectives of the study were to build an automated pressure injury risk assessment system (Auto-PIRAS) that can assess pressure injury risk using data, without requiring nurses to collect or input additional data, and to evaluate the validity of this assessment tool. A retrospective case-control study and a system development study were conducted in a 1,355-bed university hospital in Seoul, South Korea. A total of 1,305 pressure injury patients and 5,220 nonpressure injury patients participated for the development of a risk scoring algorithm: 687 and 2,748 for the validation of the algorithm and 237 and 994 for validation after clinical implementation, respectively. A total of 4,211 pressure injury-related clinical variables were extracted from the electronic health record (EHR) systems to develop a risk scoring algorithm, which was validated and incorporated into the EHR. That program was further evaluated for predictive and concurrent validity. Auto-PIRAS, incorporated into the EHR system, assigned a risk assessment score of high, moderate, or low and displayed this on the Kardex nursing record screen. Risk scores were updated nightly according to 10 predetermined risk factors. The predictive validity measures of the algorithm validation stage were as follows: sensitivity = .87, specificity = .90, positive predictive value = .68, negative predictive value = .97, Youden index = .77, and the area under the receiver operating characteristic curve = .95. The predictive validity measures of the Braden Scale were as follows: sensitivity = .77, specificity = .93, positive predictive value = .72, negative predictive value = .95, Youden index = .70, and the area under the receiver operating characteristic curve = .85. The kappa of the Auto-PIRAS and Braden Scale risk classification result was .73. The predictive performance of the Auto-PIRAS was similar to Braden Scale assessments conducted by nurses. Auto-PIRAS is expected to be used as a system that assesses pressure injury risk automatically without additional data collection by nurses.

Validation of a scenario-based assessment of critical thinking using an externally validated tool.

PubMed

Buur, Jennifer L; Schmidt, Peggy; Smylie, Dean; Irizarry, Kris; Crocker, Carlos; Tyler, John; Barr, Margaret

2012-01-01

With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.

Movement skill assessment of typically developing preschool children: a review of seven movement skill assessment tools.

PubMed

Cools, Wouter; Martelaer, Kristine De; Samaey, Christiane; Andries, Caroline

2009-06-01

The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child's physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools' usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children. Key pointsThis review discusses seven movement skill assessment tool's test content, reliability, validity and normative samples.The seven assessment tools all showed to be of great value. Strengths and weaknesses indicate that test choice will depend on specific purpose of test use.Further data collection should also include larger data samples of able bodied preschool children.Admitting PE specialists in assessment of fundamental movement skill performance among preschool children is recommended.The assessment tool's normative data samples would benefit from frequent movement skill performance follow-up of today's children. MOT 4-6: Motoriktest fur vier- bis sechsjährige Kinder, M-ABC: Movement Assessment Battery for Children, PDMS: Peabody Development Scales, KTK: Körper-Koordinationtest für Kinder, TGDM: Test of Gross Motor Development, MMT: Maastrichtse Motoriektest, BOTMP: Bruininks-Oseretsky Test of Motor Proficiency. ICC: intraclass correlation coefficient, NR: not reported, GM: gross motor, LV: long version, SV: short version, LF: long form, SF: short form, STV: subtest version, SEMs: standard errors of measurement, TMQ: Total Motor Quotient, TMC: Total Motor Composite, CSSA: Comprehensive Scales of Student Abilities MSEL: Mullen Scales of Early learning: AGS Edition AUC: Areas under curve BC: Battery composite ROC: Receiver operating characteristic.

Assessment Tools for Evaluation of Oral Feeding in Infants Less than Six Months Old

PubMed Central

Pados, Britt F.; Park, Jinhee; Estrem, Hayley; Awotwi, Araba

2015-01-01

Background Feeding difficulty is common in infants less than six months old. Identification of infants in need of specialized treatment is critical to ensure appropriate nutrition and feeding skill development. Valid and reliable assessment tools help clinicians objectively evaluate feeding. Purpose To identify and evaluate assessment tools available for clinical assessment of bottle- and breast-feeding in infants less than six months old. Methods/Search Strategy CINAHL, HaPI, PubMed, and Web of Science were searched for “infant feeding” and “assessment tool.” The literature (n=237) was reviewed for relevant assessment tools. A secondary search was conducted in CINAHL and PubMed for additional literature on identified tools. Findings/Results Eighteen assessment tools met inclusion criteria. Of these, seven were excluded because of limited available literature or because they were intended for use with a specific diagnosis or in research only. There are 11 assessment tools available for clinical practice. Only two of these were intended for bottle-feeding. All 11 indicated they were appropriate for use with breast-feeding. None of the available tools have adequate psychometric development and testing. Implications for Practice All of the tools should be used with caution. The Early Feeding Skills Assessment and Bristol Breastfeeding Assessment Tool had the most supportive psychometric development and testing. Implications for Research Feeding assessment tools need to be developed and tested to guide optimal clinical care of infants from birth through six months. A tool that assesses both bottle- and breast-feeding would allow for consistent assessment across feeding methods. PMID:26945280

Broadening Perspectives on Clinical Performance Assessment: Rethinking the Nature of In-Training Assessment

ERIC Educational Resources Information Center

Govaerts, Marjan J. B.; van der Vleuten, Cees P. M.; Schuwirth, Lambert W. T.; Muijtjens, Arno M. M.

2007-01-01

Context: In-training assessment (ITA), defined as multiple assessments of performance in the setting of day-to-day practice, is an invaluable tool in assessment programmes which aim to assess professional competence in a comprehensive and valid way. Research on clinical performance ratings, however, consistently shows weaknesses concerning…

Development, reliability, and validity of the My Child's Play (MCP) questionnaire.

PubMed

Schneider, Eleanor; Rosenblum, Sara

2014-01-01

This article describes the development, reliability, and validity of My Child's Play (MCP), a parent questionnaire designed to evaluate the play of children ages 3-9 yr. The first phase of the study determined the questionnaire's content and face validity. Subsequently, the internal reliability consistency and construct and concurrent validity were demonstrated using 334 completed questionnaires. The MCP showed good internal consistency (α = .86). The factor analysis revealed four distinct factors with acceptable levels of internal reliability (Cronbach's αs = .63-.81) and gender- and age-related differences in play characteristics; both findings attest to the tool's construct validity. Significant correlations (r = .33, p < .0001) with the Parent as a Teacher Inventory demonstrate the MCP's concurrent validity. The MCP demonstrated acceptable reliability and validity. It appears to be a promising standardized assessment tool for use in research and practice to promote understanding of a child's play. Copyright © 2014 by the American Occupational Therapy Association, Inc.

Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose.

PubMed

Loudon, Kirsty; Zwarenstein, Merrick; Sullivan, Frank; Donnan, Peter; Treweek, Shaun

2013-04-27

If you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention. The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters' subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of pragmatism. We have concrete suggestions for improving PRECIS and a growing list of enthusiastic individuals interested in contributing to this work. By early 2014 we expect to have a validated PRECIS-2.

[Validity of the suicidality assessment instrument PRISM-S (Pictoral Representation of Illness Self Measure - Suicidality)].

PubMed

Ring, Mariann; Harbauer, Gregor; Haas, Sebastian; Schuetz, Christopher; Andreae, Andreas; Maercker, Andreas; Ajdacic-Gross, Vladeta

2014-01-01

In routine clinical practice the assessment of suicidality proves to be difficult and complex. The aim of the present study was to examine if PRISM can be used to measure validly the person's subjectively perceived suicidality. The nonverbal visualization technique PRISM (Pictoral Representation of Illness and Self Measure) has been developed by Büchi et al. (2002) to evaluate the perceived burden of suffering due to physical illness. The adapted version of PRISM used in our study is called PRISM-S (Pictoral Representation of Illness and Self Measure - Suicidality). 156 eligible inpatients, admitted voluntarily to the crisis intervention centre Winterthur, participated in the study. We used as gold standards the well established assessment tools the Beck Scale of Suicide Ideation (BSS) and the Depressive Symptome Inventory - Subscale (DSI-SS). The results showed high correlations between PRISM-S and the BSS (r = - 0,73) and the DSI-SS scores (r = - 0,76). Clinicians, general practitioners, psychiatrists and psychologists receive with PRISM-S a valid suicidality assessment tool that is very brief and easy to administer in clinical settings.

Validation of a Brief Questionnaire Against Direct Observation to Assess Adolescents' School Lunchtime Beverage Consumption.

PubMed

Grummon, Anna H; Hampton, Karla E; Hecht, Amelie; Oliva, Ariana; McCulloch, Charles E; Brindis, Claire D; Patel, Anisha I

Beverage consumption is an important determinant of youth health outcomes. Beverage interventions often occur in schools, yet no brief validated questionnaires exist to assess whether these efforts improve in-school beverage consumption. This study validated a brief questionnaire to assess beverage consumption during school lunch. Researchers observed middle school students' (n = 25) beverage consumption during school lunchtime using a standardized tool. After lunch, students completed questionnaires regarding their lunchtime beverage consumption. Kappa statistics compared self-reported with observed beverage consumption across 15 beverage categories. Eight beverages showed at least fair agreement (kappa [κ] > 0.20) for both type and amount consumed, with most showing substantial agreement (κ > 0.60). One beverage had high raw agreement but κ < 0.20. Six beverages had too few ratings to compute κ's. This brief questionnaire was useful for assessing school lunchtime consumption of many beverages and provides a low-cost tool for evaluating school-based beverage interventions. Copyright © 2017 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Assessment of competence in video-assisted thoracoscopic surgery lobectomy: A Danish nationwide study.

PubMed

Petersen, René Horsleben; Gjeraa, Kirsten; Jensen, Katrine; Møller, Lars Borgbjerg; Hansen, Henrik Jessen; Konge, Lars

2018-04-18

Competence in video-assisted thoracoscopic surgery lobectomy has previously been established on the basis of numbers of procedures performed, but this approach does not ensure competence. Specific assessment tools, such as the newly developed video-assisted thoracoscopic surgery lobectomy assessment tool, allow for structured and objective assessment of competence. Our aim was to provide validity evidence for the video-assisted thoracoscopic surgery lobectomy assessment tool. Video recordings of 60 video-assisted thoracoscopic surgery lobectomies performed by 18 thoracic surgeons were rated using the video-assisted thoracoscopic surgery lobectomy assessment tool. All 4 centers of thoracic surgery in Denmark participated in the study. Two video-assisted thoracoscopic surgery experts rated the videos. They were blinded to surgeon and center. The total internal consistency reliability Cronbach's alpha was 0.93. Inter-rater reliability between the 2 raters was Pearson's r = 0.71 (P < .001). The mean video-assisted thoracoscopic surgery lobectomy assessment tool scores for the 10 procedures performed by beginners were 22.1 (standard deviation [SD], 8.6) for the 28 procedures performed by the intermediate surgeons, 31.2 (SD, 4.4), and for the 20 procedures performed by experts 35.9 (SD, 2.9) (P < .001). Bonferroni post hoc tests showed that experts were significantly better than intermediates (P < .008) and beginners (P < .001). Intermediates' mean scores were significantly better than beginners (P < .001). The pass/fail standard calculated using the contrasting group's method was 31 points. One of the beginners passed, and 2 experts failed the test. Validity evidence was provided for a newly developed assessment tool for video-assisted thoracoscopic surgery lobectomy (video-assisted thoracoscopic surgery lobectomy assessment tool) in a clinical setting. The discriminatory ability among expert surgeons, intermediate surgeons, and beginners proved highly significant. The video-assisted thoracoscopic surgery lobectomy assessment tool could be an important aid in the future training and certification of thoracic surgeons. Copyright © 2018 The American Association for Thoracic Surgery. Published by Elsevier Inc. All rights reserved.

Measuring Nutrition Literacy in Spanish-Speaking Latinos: An Exploratory Validation Study.

PubMed

Gibbs, Heather D; Camargo, Juliana M T B; Owens, Sarah; Gajewski, Byron; Cupertino, Ana Paula

2017-11-21

Nutrition is important for preventing and treating chronic diseases highly prevalent among Latinos, yet no tool exists for measuring nutrition literacy among Spanish speakers. This study aimed to adapt the validated Nutrition Literacy Assessment Instrument for Spanish-speaking Latinos. This study was developed in two phases: adaptation and validity testing. Adaptation included translation, expert item content review, and interviews with Spanish speakers. For validity testing, 51 participants completed the Short Assessment of Health Literacy-Spanish (SAHL-S), the Nutrition Literacy Assessment Instrument in Spanish (NLit-S), and socio-demographic questionnaire. Validity and reliability statistics were analyzed. Content validity was confirmed with a Scale Content Validity Index of 0.96. Validity testing demonstrated NLit-S scores were strongly correlated with SAHL-S scores (r = 0.52, p < 0.001). Entire reliability was substantial at 0.994 (CI 0.992-0.996) and internal consistency was excellent (Cronbach's α = 0.92). The NLit-S demonstrates validity and reliability for measuring nutrition literacy among Spanish-speakers.

Treatment of Neurogenic Cough with Tramadol: A Pilot Study.

PubMed

Dion, Gregory R; Teng, Stephanie E; Achlatis, Efstratios; Fang, Yixin; Amin, Milan R

2017-07-01

This study employs validated cough assessment tools to prospectively determine the impact of tramadol on cough severity and quality of life in subjects with neurogenic cough. The study was a prospective case series with planned data collection at a tertiary care academic medical center laryngology practice. Sixteen consecutive collected subjects with neurogenic cough prospectively completed pre- and posttreatment validated cough assessment tools, the cough severity index (CSI) and Leicester Cough Questionnaire (LCQ). All subjects in the study reported at least some improvement in their cough symptoms. In a Wilcoxon signed rank test that compared paired results, CSI scores improved from 23 to 14 and LCQ scores improved from 74 to 103 ( P = .003 and P = .005, respectively). This small preliminary assessment suggests that tramadol warrants additional evaluation as a treatment for neurogenic cough.

Implementing Lumberjacks and Black Swans Into Model-Based Tools to Support Human-Automation Interaction.

PubMed

Sebok, Angelia; Wickens, Christopher D

2017-03-01

The objectives were to (a) implement theoretical perspectives regarding human-automation interaction (HAI) into model-based tools to assist designers in developing systems that support effective performance and (b) conduct validations to assess the ability of the models to predict operator performance. Two key concepts in HAI, the lumberjack analogy and black swan events, have been studied extensively. The lumberjack analogy describes the effects of imperfect automation on operator performance. In routine operations, an increased degree of automation supports performance, but in failure conditions, increased automation results in more significantly impaired performance. Black swans are the rare and unexpected failures of imperfect automation. The lumberjack analogy and black swan concepts have been implemented into three model-based tools that predict operator performance in different systems. These tools include a flight management system, a remotely controlled robotic arm, and an environmental process control system. Each modeling effort included a corresponding validation. In one validation, the software tool was used to compare three flight management system designs, which were ranked in the same order as predicted by subject matter experts. The second validation compared model-predicted operator complacency with empirical performance in the same conditions. The third validation compared model-predicted and empirically determined time to detect and repair faults in four automation conditions. The three model-based tools offer useful ways to predict operator performance in complex systems. The three tools offer ways to predict the effects of different automation designs on operator performance.

The adaptation of the Sheffield Profile for Assessment and Referral for Care (SPARC) to the Polish clinical setting for needs assessment of advanced cancer patients.

PubMed

Leppert, Wojciech; Majkowicz, Mikolaj; Ahmedzai, Sam H

2012-12-01

Assessment of the needs of advanced cancer patients is a very important issue in palliative care. The aim of the study was to adapt the Sheffield Profile for Assessment and Referral for Care (SPARC) to the Polish environment and evaluate its usefulness in needs assessment of patients with advanced cancer. A forward-back translation of the SPARC to Polish was done. The SPARC was used once in 58 consecutive patients with advanced cancer during follow-up. The patients were enrolled from a palliative care unit (25 patients), home care (18 patients), and a day care center (15 patients). The reliability was evaluated by establishing the internal consistency using Cronbach's alpha coefficients. Content validity was analyzed in accordance with the theories of needs by Murray and Maslow as a nonstatistical method of validity assessment. Factor analysis with principal components extraction and varimax rotation of raw data was used to reduce the set of data and assess the construct validity. There were differences regarding religious and spiritual issues and independence and activity between patients in the palliative care unit (worse results) and those at the day care center (better scores). Communication and need for more information items were associated with psychological, social, spiritual, and treatment issues. Cronbach's alpha coefficients and factor analysis demonstrated, respectively, satisfactory reliability and construct validity of the tool. The study demonstrated that the Polish version of the SPARC is a valid and reliable tool recommended for the needs assessment and symptom evaluation of patients with advanced cancer. Copyright © 2012 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved.

Assessing the Validity and Reliability of the Peristomal Skin Lesion Assessment Instrument Adapted for Use in Turkey.

PubMed

Ay, Ali; Bulut, Hulya

2015-08-01

Many ostomy patients experience peristomal skin lesions. A descriptive study was conducted to assess the validity, usability, and reliability of the Peristomal Skin Lesions Assessment instrument (SACS instrument) adapted to Turkish from English. The SACS Instrument consists of 2 main assessments: lesion type (utilizing definitions and photographs) and lesion area by location around the ostomy. The study was performed in 2 stages: 1) the SACS language was changed and its content validity established; and 2) the instrument\\'92s content validity and inter-observer agreement (consistency) were determined among pairs of nurses who used the tool to assess peristomal skin lesions. Patients (included if they were >18 years old and receiving treatment/observation at 1 of the 4 participating stomatherapy units) and 8 stomatherapy nurses also completed appropriate sociodemographic questionnaires. Of the 393 patients screened during the 7-month study, 100 (average age 56.74 \\'b1 14.03 years, 55 men) participated; most (79) had a planned operation. A little more than half (59) of the patients had colorectal cancer and 28 had their stoma site marked preoperatively by a stomatherapy nurse. The most common peristomal skin lesion risk factors were having an ileostomy and unplanned surgery. The content validity index of the entire Turkish SACS instrument was 1, and the inter-observer agreement Kappa statistic was very good (K = 0.90, 95% CI 0.80- 0.99). Individual SACS item K values ranged from K = 0.84 (95% CI 0.63\\'961) to K = 1 (95% CI 1). Most (62.5%) nurses found the terms and pictures used in the SACS classification adequate and suitable, and 50% believed the Turkish version of the SACS instrument was a valid and suitable assessment tool for use by Turkish stomatherapy nurses. Validity and reliability studies involving larger and more diverse patient and nurse samples are warranted.

Computer-Aided Nodule Assessment and Risk Yield Risk Management of Adenocarcinoma: The Future of Imaging?

PubMed

Foley, Finbar; Rajagopalan, Srinivasan; Raghunath, Sushravya M; Boland, Jennifer M; Karwoski, Ronald A; Maldonado, Fabien; Bartholmai, Brian J; Peikert, Tobias

2016-01-01

Increased clinical use of chest high-resolution computed tomography results in increased identification of lung adenocarcinomas and persistent subsolid opacities. However, these lesions range from very indolent to extremely aggressive tumors. Clinically relevant diagnostic tools to noninvasively risk stratify and guide individualized management of these lesions are lacking. Research efforts investigating semiquantitative measures to decrease interrater and intrarater variability are emerging, and in some cases steps have been taken to automate this process. However, many such methods currently are still suboptimal, require validation and are not yet clinically applicable. The computer-aided nodule assessment and risk yield software application represents a validated tool for the automated, quantitative, and noninvasive tool for risk stratification of adenocarcinoma lung nodules. Computer-aided nodule assessment and risk yield correlates well with consensus histology and postsurgical patient outcomes, and therefore may help to guide individualized patient management, for example, in identification of nodules amenable to radiological surveillance, or in need of adjunctive therapy. Copyright © 2016 Elsevier Inc. All rights reserved.

Ensuring relational competency in critical care: Importance of nursing students' communication skills.

PubMed

Sánchez Expósito, Judit; Leal Costa, César; Díaz Agea, José Luis; Carrillo Izquierdo, María Dolores; Jiménez Rodríguez, Diana

2018-02-01

The aim of this study was to analyse the communication skills of students in interactions with simulated critically-ill patients using a new assessment tool to study the relationships between communication skills, teamwork and clinical skills and to analyse the psychometric properties of the tool. A cross-sectional study was conducted to assess the communications skills of 52 students with critically-ill patients through the use of a new measurement tool to score video recordings of simulated clinical scenarios. The 52 students obtained low scores on their skills in communicating with patients. The reliability of the measuring instrument showed good inter-observer agreement (ICC between 0.71 and 0.90) and the validity yielded a positive correlation (p<0.01). The results provide evidence that nursing students lack skills when communicating with critically ill patients in simulated scenarios. The measuring instrument used is therefore deemed valid and reliable for assessing nursing students through a clinical simulation. Copyright © 2017 Elsevier Ltd. All rights reserved.

The Chelsea critical care physical assessment tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study.

PubMed

Corner, E J; Wood, H; Englebretsen, C; Thomas, A; Grant, R L; Nikoletou, D; Soni, N

2013-03-01

To develop a scoring system to measure physical morbidity in critical care - the Chelsea Critical Care Physical Assessment Tool (CPAx). The development process was iterative involving content validity indices (CVI), a focus group and an observational study of 33 patients to test construct validity against the Medical Research Council score for muscle strength, peak cough flow, Australian Therapy Outcome Measures score, Glasgow Coma Scale score, Bloomsbury sedation score, Sequential Organ Failure Assessment score, Short Form 36 (SF-36) score, days of mechanical ventilation and inter-rater reliability. Trauma and general critical care patients from two London teaching hospitals. Users of the CPAx felt that it possessed content validity, giving a final CVI of 1.00 (P<0.05). Construct validation data showed moderate to strong significant correlations between the CPAx score and all secondary measures, apart from the mental component of the SF-36 which demonstrated weak correlation with the CPAx score (r=0.024, P=0.720). Reliability testing showed internal consistency of α=0.798 and inter-rater reliability of κ=0.988 (95% confidence interval 0.791 to 1.000) between five raters. This pilot work supports proof of concept of the CPAx as a measure of physical morbidity in the critical care population, and is a cogent argument for further investigation of the scoring system. Copyright © 2012 Chartered Society of Physiotherapy. Published by Elsevier Ltd. All rights reserved.

The water balance questionnaire: design, reliability and validity of a questionnaire to evaluate water balance in the general population.

PubMed

Malisova, Olga; Bountziouka, Vassiliki; Panagiotakos, Demosthenes B; Zampelas, Antonis; Kapsokefalou, Maria

2012-03-01

There is a need to develop a questionnaire as a research tool for the evaluation of water balance in the general population. The water balance questionnaire (WBQ) was designed to evaluate water intake from fluid and solid foods and drinking water, and water loss from urine, faeces and sweat at sedentary conditions and physical activity. For validation purposes, the WBQ was administrated in 40 apparently healthy participants aged 22-57 years (37.5% males). Hydration indices in urine (24 h volume, osmolality, specific gravity, pH, colour) were measured through established procedures. Furthermore, the questionnaire was administered twice to 175 subjects to evaluate its reliability. Kendall's τ-b and the Bland and Altman method were used to assess the questionnaire's validity and reliability. The proposed WBQ to assess water balance in healthy individuals was found to be valid and reliable, and it could thus be a useful tool in future projects that aim to evaluate water balance.

Neighborhood Quality and Attachment: Validation of the Revised Residential Environment Assessment Tool.

PubMed

Poortinga, Wouter; Calve, Tatiana; Jones, Nikki; Lannon, Simon; Rees, Tabitha; Rodgers, Sarah E; Lyons, Ronan A; Johnson, Rhodri

2017-04-01

Various studies have shown that neighborhood quality is linked to neighborhood attachment and satisfaction. However, most have relied upon residents' own perceptions rather than independent observations of the neighborhood environment. This study examines the reliability and validity of the revised Residential Environment Assessment Tool (REAT 2.0), an audit instrument covering both public and private spaces of the neighborhood environment. The research shows that REAT 2.0 is a reliable, easy-to-use instrument and that most underlying constructs can be validated against residents' own neighborhood perceptions. The convergent validity of the instrument, which was tested against digital map data, can be improved for a number of miscellaneous urban form items. The research further found that neighborhood attachment was significantly associated with the overall REAT 2.0 score. This association can mainly be attributed to the property-level neighborhood quality and natural elements components. The research demonstrates the importance of private spaces in the outlook of the neighborhood environment.

Health-related quality of life in cancer patients at the end of life, translation, validation, and longitudinal analysis of specific tools: study protocol for a randomized controlled trial.

PubMed

Poirier, Anne-Lise; Kwiatkowski, Fabrice; Commer, Jean-Marie; D'Aillières, Bénédicte; Berger, Virginie; Mercier, Mariette; Bonnetain, Franck

2012-04-20

The end of life for cancer patients is the ultimate stage of the disease, and care in this setting is important as it can improve the wellbeing not only of patients, but also the patients' family and close friends. As it is a matter of profoundly personal concerns, patients' perception of this phase of the disease is difficult to assess and has thus been insufficiently studied. Nonetheless, caregivers are required to provide specific care to help patients and to treat them in order to improve their wellbeing during this period.While tools to assess health-related quality of life (QoL) in cancer patients at the end of life exist in English, to our knowledge, no validated tools are available in French. This randomized multicenter cohort study will be carried out to cross-culturally adapt and validate a French version of the English QUAL-E and the Missoula Vitas Quality Of Life Index (MVQOLI) questionnaires for advanced cancer patients in a palliative setting. A randomized clinical trial component in addition to a cohort study is implemented in order to test psychometric hypotheses: order effect and improvement of sensibility to change.The validation procedure will ensure that the psychometric properties are maintained.The main criterion to assess the reliability of the questionnaires will be reproducibility (test-retest method) using intraclass correlation coefficients. It will be necessary to include 372 patients. The sensitivity to change, discriminant capability as well as convergent validity will be also investigated. If the cross-cultural validation of the MVQOLI and QUAL-E questionnaires for advanced cancer patients in a palliative setting have satisfactory psychometric properties, it will allow us to assess the specific dimensions of QoL at the end of life. Current Controlled Trials NCT01545921.

Development and validation of a tool to assess knowledge and attitudes towards generic medicines among students in Greece: The ATtitude TOwards GENerics (ATTOGEN) questionnaire.

PubMed

Domeyer, Philip J; Aletras, Vassilis; Anagnostopoulos, Fotios; Katsari, Vasiliki; Niakas, Dimitris

2017-01-01

The use of generic medicines is a cost-effective policy, often dictated by fiscal restraints. To our knowledge, no fully validated tool exploring the students' knowledge and attitudes towards generic medicines exists. The aim of our study was to develop and validate a questionnaire exploring the knowledge and attitudes of M.Sc. in Health Care Management students and recent alumni's towards generic drugs in Greece. The development of the questionnaire was a result of literature review and pilot-testing of its preliminary versions to researchers and students. The final version of the questionnaire contains 18 items measuring the respondents' knowledge and attitude towards generic medicines on a 5-point Likert scale. Given the ordinal nature of the data, ordinal alpha and polychoric correlations were computed. The sample was randomly split into two halves. Exploratory factor analysis, performed in the first sample, was used for the creation of multi-item scales. Confirmatory factor analysis and Generalized Linear Latent and Mixed Model analysis (GLLAMM) with the use of the rating scale model were used in the second sample to assess goodness of fit. An assessment of internal consistency reliability, test-retest reliability, and construct validity was also performed. Among 1402 persons contacted, 986 persons completed our questionnaire (response rate = 70.3%). Overall Cronbach's alpha was 0.871. The conjoint use of exploratory and confirmatory factor analysis resulted in a six-scale model, which seemed to fit the data well. Five of the six scales, namely trust, drug quality, state audit, fiscal impact and drug substitution were found to be valid and reliable, while the knowledge scale suffered only from low inter-scale correlations and a ceiling effect. However, the subsequent confirmatory factor and GLLAMM analyses indicated a good fit of the model to the data. The ATTOGEN instrument proved to be a reliable and valid tool, suitable for assessing students' knowledge and attitudes towards generic medicines.

Development of a Hypertension Health Literacy Assessment Tool for use in primary healthcare clinics in South Africa, Gauteng

PubMed Central

Mogotlane, Sophie

2017-01-01

Background Hypertension is a universal risk factor for cardiovascular morbidity and mortality in both the ageing and obese populations and patients must be literate in hypertension health issues to participate actively in the management of their disease. Little research has been done to investigate hypertension health literacy levels among South Africans. Aim To develop a Hypertension Heath Literacy Assessment Tool to establish patients’ comprehension of the health education they receive in primary healthcare (PHC) clinics in Tshwane, Gauteng, South Africa. Setting PHC clinics in Tshwane, Gauteng, South Africa. Methods The design was quantitative, descriptive and contextual in nature. The study population comprised health promoters who were experts in the field of health, documents containing hypertension health education content and individuals with hypertension. Participants were conveniently and purposefully selected. A modified Delphi technique was used to develop and validate the Hypertension Health Literacy Assessment Tool (HHLAT). To ensure validity and reliability of the HHLAT, the tool was administered to 195 participants concurrently with the Learning Ability Battery (LAB). Results There was a strong positive (F = 76.0, p < 0.0001, R2 = 28.25%) correlation between the LAB and the HHLAT. The HHLAT indicated that only 37 (19%) of the patients with hypertension had poor hypertension health literacy levels. Conclusion The HHLAT is a valid tool that can be used in busy PHC clinics as it takes less than two minutes to administer. This tool can inform the healthcare worker on the depth of hypertension health education to be given to the patient, empowering the patient and saving time in PHC facilities. PMID:28828872

[Assessment of IDC-Pal as a Diagnostic Tool for Family Physicians to Identify Patients with Complex Palliative Care Needs in Germany: a Pilot Study].

PubMed

Comino, María Rosa Salvador; Garcia, Victor Regife; López, Maria Auxiliadora Fernández; Feddersen, Berend; Roselló, María Luisa Martin; Sanftenberg, Linda; Schelling, Jörg

2017-07-11

Background Palliative medicine is an essential component of the health care system. Basic palliative care should be provided by primary care services (family physician and home nursing) with palliative-medical basic qualification. Often it is very difficult to identify patients that would profit from a specialized palliative care team. For the evaluation of the case complexity of a palliative patient, we present a Spanish diagnostic tool IDC-Pal, which tries to specify when, why and where a palliative patient should be referred. The aims of this study were the translation and cultural adaptation of the diagnostic tool for complexity in palliative care IDC-Pal to the German language, and the measurement of its feasibility and face validity. Methods During the first phase, a forward-backward translation with linguistic and cultural adaptation of the tool IDC-Pal as well as the validation of its content by a review committee was performed. During the second phase, the preliminary version of the tool was tested by 38 family physicians that were asked for a qualitative assessment using a 10-points Likert scale (1 = "strongly disagree" and 10 = "totally agree"). Finally, a definitive version was developed. Results The translation and adaptation were achieved without major problems. Both feasibility and apparent validity of the tool IDC-Pal were rated as high. The mean response in the Likert scale was 7.79, with a SD of 0.36. Participants strongly agreed on the apparent validity of the tool with a mean of 7.82 and a SD of 0.26 and on its feasibility with a mean of 7.79, and a SD of 0.39. Conclusions A conceptually, culturally and linguistically equivalent version of the original instrument IDC-Pal was obtained. German family physicians agreed on the usability of IDC-Pal as a tool for rating the case complexity of palliative patients. These results indicate that physicians in Bavaria and eventually in Germany could benefit of the full validation of IDC-Pal. © Georg Thieme Verlag KG Stuttgart · New York.

Validation of a New Metric for Assessing the Integration of Health Protection and Health Promotion in a Sample of Small- and Medium-Sized Employer Groups.

PubMed

Williams, Jessica A R; Nelson, Candace C; Cabán-Martinez, Alberto J; Katz, Jeffrey N; Wagner, Gregory R; Pronk, Nicolaas P; Sorensen, Glorian; McLellan, Deborah L

2015-09-01

To conduct validation analyses for a new measure of the integration of worksite health protection and health promotion approaches developed in earlier research. A survey of small- to medium-sized employers located in the United States was conducted between October 2013 and March 2014 (n = 111). Cronbach α coefficient was used to assess reliability, and Pearson correlation coefficients were used to assess convergent validity. The integration score was positively associated with the measures of occupational safety and health and health promotion activities/policies-supporting its convergent validity (Pearson correlation coefficients of 0.32 to 0.47). Cronbach α coefficient was 0.94, indicating excellent reliability. The integration score seems to be a promising tool for assessing integration of health promotion and health protection. Further work is needed to test its dimensionality and validate its use in other samples.

The conservation physiology toolbox: status and opportunities

PubMed Central

Love, Oliver P; Hultine, Kevin R

2018-01-01

Abstract For over a century, physiological tools and techniques have been allowing researchers to characterize how organisms respond to changes in their natural environment and how they interact with human activities or infrastructure. Over time, many of these techniques have become part of the conservation physiology toolbox, which is used to monitor, predict, conserve, and restore plant and animal populations under threat. Here, we provide a summary of the tools that currently comprise the conservation physiology toolbox. By assessing patterns in articles that have been published in ‘Conservation Physiology’ over the past 5 years that focus on introducing, refining and validating tools, we provide an overview of where researchers are placing emphasis in terms of taxa and physiological sub-disciplines. Although there is certainly diversity across the toolbox, metrics of stress physiology (particularly glucocorticoids) and studies focusing on mammals have garnered the greatest attention, with both comprising the majority of publications (>45%). We also summarize the types of validations that are actively being completed, including those related to logistics (sample collection, storage and processing), interpretation of variation in physiological traits and relevance for conservation science. Finally, we provide recommendations for future tool refinement, with suggestions for: (i) improving our understanding of the applicability of glucocorticoid physiology; (ii) linking multiple physiological and non-physiological tools; (iii) establishing a framework for plant conservation physiology; (iv) assessing links between environmental disturbance, physiology and fitness; (v) appreciating opportunities for validations in under-represented taxa; and (vi) emphasizing tool validation as a core component of research programmes. Overall, we are confident that conservation physiology will continue to increase its applicability to more taxa, develop more non-invasive techniques, delineate where limitations exist, and identify the contexts necessary for interpretation in captivity and the wild. PMID:29942517

Developing a tool for the preparation of GMP audit of pharmaceutical contract manufacturer.

PubMed

Linna, Anu; Korhonen, Mirka; Mannermaa, Jukka-Pekka; Airaksinen, Marja; Juppo, Anne Mari

2008-06-01

Outsourcing is rapidly growing in the pharmaceutical industry. When the manufacturing activities are outsourced, control of the product's quality has to be maintained. One way to confirm contract manufacturer's GMP (Good Manufacturing Practice) compliance is auditing. Audits can be supported for instance by using GMP questionnaires. The objective of this study was to develop a tool for the audit preparation of pharmaceutical contract manufacturers and to validate its contents by using Delphi method. At this phase of the study the tool was developed for non-sterile finished product contract manufacturers. A modified Delphi method was used with expert panel consisting of 14 experts from pharmaceutical industry, authorities and university. The content validity of the developed tool was assessed by a Delphi questionnaire round. The response rate in Delphi questionnaire round was 86%. The tool consisted of 103 quality items, from which 90 (87%) achieved the pre-defined agreement rate level (75%). Thirteen quality items which did not achieve the pre-defined agreement rate were excluded from the tool. The expert panel suggested only minor changes to the tool. The results show that the content validity of the developed audit preparation tool was good.

A comprehensive clinical assessment tool to inform policy and practice: applications of the minimum data set.

PubMed

Mor, Vincent

2004-04-01

The Minimum Data Set (MDS) for nursing home (NH) resident assessment, designed to assess elders functional status and care needs, exemplifies how the information needs of clinical practice are congruent with those of research. Building on a review of the published literature, this article describes the development of the MDS, its reliability and validity testing, as well as the variety of different policy and research uses to which it has been applied. Interrater reliability of items and internal consistency of MDS summary scales is generally good to excellent. Validation studies reveal good correspondence to research quality instruments for cognition, activities of daily living, and diagnoses with more variable results for vision, pain, mood, and behavior scales. To date, no consistent evidence suggests that applications of MDS data for case-mix reimbursement and quality indicator monitoring systematically bias the data. Although facility variation in data quality could compromise some applications, creation of the MDS as a clinical tool for care planning provides an example of how assessment tools with clinical use can be used in administrative databases for research and policy applications.

The Arthroscopic Surgical Skill Evaluation Tool (ASSET)

PubMed Central

Koehler, Ryan J.; Amsdell, Simon; Arendt, Elizabeth A; Bisson, Leslie J; Braman, Jonathan P; Butler, Aaron; Cosgarea, Andrew J; Harner, Christopher D; Garrett, William E; Olson, Tyson; Warme, Winston J.; Nicandri, Gregg T.

2014-01-01

Background Surgeries employing arthroscopic techniques are among the most commonly performed in orthopaedic clinical practice however, valid and reliable methods of assessing the arthroscopic skill of orthopaedic surgeons are lacking. Hypothesis The Arthroscopic Surgery Skill Evaluation Tool (ASSET) will demonstrate content validity, concurrent criterion-oriented validity, and reliability, when used to assess the technical ability of surgeons performing diagnostic knee arthroscopy on cadaveric specimens. Study Design Cross-sectional study; Level of evidence, 3 Methods Content validity was determined by a group of seven experts using a Delphi process. Intra-articular performance of a right and left diagnostic knee arthroscopy was recorded for twenty-eight residents and two sports medicine fellowship trained attending surgeons. Subject performance was assessed by two blinded raters using the ASSET. Concurrent criterion-oriented validity, inter-rater reliability, and test-retest reliability were evaluated. Results Content validity: The content development group identified 8 arthroscopic skill domains to evaluate using the ASSET. Concurrent criterion-oriented validity: Significant differences in total ASSET score (p<0.05) between novice, intermediate, and advanced experience groups were identified. Inter-rater reliability: The ASSET scores assigned by each rater were strongly correlated (r=0.91, p <0.01) and the intra-class correlation coefficient between raters for the total ASSET score was 0.90. Test-retest reliability: there was a significant correlation between ASSET scores for both procedures attempted by each individual (r = 0.79, p<0.01). Conclusion The ASSET appears to be a useful, valid, and reliable method for assessing surgeon performance of diagnostic knee arthroscopy in cadaveric specimens. Studies are ongoing to determine its generalizability to other procedures as well as to the live OR and other simulated environments. PMID:23548808

A reliability and validity study of the Palliative Performance Scale

PubMed Central

Ho, Francis; Lau, Francis; Downing, Michael G; Lesperance, Mary

2008-01-01

Background The Palliative Performance Scale (PPS) was first introduced in1996 as a new tool for measurement of performance status in palliative care. PPS has been used in many countries and has been translated into other languages. Methods This study evaluated the reliability and validity of PPS. A web-based, case scenarios study with a test-retest format was used to determine reliability. Fifty-three participants were recruited and randomly divided into two groups, each evaluating 11 cases at two time points. The validity study was based on the content validation of 15 palliative care experts conducted over telephone interviews, with discussion on five themes: PPS as clinical assessment tool, the usefulness of PPS, PPS scores affecting decision making, the problems in using PPS, and the adequacy of PPS instruction. Results The intraclass correlation coefficients for absolute agreement were 0.959 and 0.964 for Group 1, at Time-1 and Time-2; 0.951 and 0.931 for Group 2, at Time-1 and Time-2 respectively. Results showed that the participants were consistent in their scoring over the two times, with a mean Cohen's kappa of 0.67 for Group 1 and 0.71 for Group 2. In the validity study, all experts agreed that PPS is a valuable clinical assessment tool in palliative care. Many of them have already incorporated PPS as part of their practice standard. Conclusion The results of the reliability study demonstrated that PPS is a reliable tool. The validity study found that most experts did not feel a need to further modify PPS and, only two experts requested that some performance status measures be defined more clearly. Areas of PPS use include prognostication, disease monitoring, care planning, hospital resource allocation, clinical teaching and research. PPS is also a good communication tool between palliative care workers. PMID:18680590

Validation of Medical Tourism Service Quality Questionnaire (MTSQQ) for Iranian Hospitals

PubMed Central

Qolipour, Mohammad; Torabipour, Amin; Khiavi, Farzad Faraji; Malehi, Amal Saki

2017-01-01

Introduction Assessing service quality is one of the basic requirements to develop the medical tourism industry. There is no valid and reliable tool to measure service quality of medical tourism. This study aimed to determine the reliability and validity of a Persian version of medical tourism service quality questionnaire for Iranian hospitals. Methods To validate the medical tourism service quality questionnaire (MTSQQ), a cross-sectional study was conducted on 250 Iraqi patients referred to hospitals in Ahvaz (Iran) from 2015. To design a questionnaire and determine its content validity, the Delphi Technique (3 rounds) with the participation of 20 medical tourism experts was used. Construct validity of the questionnaire was assessed through exploratory and confirmatory factor analysis. Reliability was assessed using Cronbach’s alpha coefficient. Data were analyzed by Excel 2007, SPSS version18, and Lisrel l8.0 software. Results The content validity of the questionnaire with CVI=0.775 was confirmed. According to exploratory factor analysis, the MTSQQ included 31 items and 8 dimensions (tangibility, reliability, responsiveness, assurance, empathy, exchange and travel facilities, technical and infrastructure facilities and safety and security). Construct validity of the questionnaire was confirmed, based on the goodness of fit quantities of model (RMSEA=0.032, CFI= 0.98, GFI=0.88). Cronbach’s alpha coefficient was 0.837 and 0.919 for expectation and perception questionnaire. Conclusion The results of the study showed that the medical tourism SERVQUAL questionnaire with 31 items and 8 dimensions was a valid and reliable tool to measure service quality of medical tourism in Iranian hospitals. PMID:28461863

Validation of a virtual reality balance module for use in clinical concussion assessment and management.

PubMed

Teel, Elizabeth F; Slobounov, Semyon M

2015-03-01

To determine the criterion and content validity of a virtual reality (VR) balance module for use in clinical practice. Retrospective, VR balance module completed by participants during concussion baseline or assessment testing session. A Pennsylvania State University research laboratory. A total of 60 control and 28 concussed students and athletes from the Pennsylvania State University. None. This study examined: (1) the relationship between VR composite balance scores (final, stationary, yaw, pitch, and roll) and area of the center-of-pressure (eyes open and closed) scores and (2) group differences (normal volunteers and concussed student-athletes) on VR composite balance scores. With the exception of the stationary composite score, all other VR balance composite scores were significantly correlated with the center of pressure data obtained from a force platform. Significant correlations ranged from r = -0.273 to -0.704 for the eyes open conditions and from r = -0.353 to -0.876 for the eyes closed condition. When examining group differences on the VR balance composite modules, the concussed group did significantly (P < 0.01) worse on all measures compared with the control group. The VR balance module met or exceeded the criterion and content validity standard set by the current balance tools and may be appropriate for use in a clinical concussion setting. Virtual reality balance module is a valid tool for concussion assessment in clinical settings. This novel type of balance assessment may be more sensitive to concussion diagnoses, especially later (7-10 days) in the recovery phase than current clinical balance tools.

Linguistic validation of the Alberta Context Tool and two measures of research use, for German residential long term care

PubMed Central

2014-01-01

Background To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks’ Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity–an initial stage of validity, necessary before more advanced validity assessment. Methods We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants’ answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants. Results Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups. Conclusion Cognitive debriefing is essential to detect and respond to problematic instrument items, particularly when translating instruments for heterogeneous, less well educated provider groups such as HCAs. Cognitive debriefing is an important step in research tool development and a vital component of establishing response process validity evidence. Publishing cognitive debriefing results helps researchers to determine potentially critical elements of the translated tools and assists with interpreting scores. PMID:24479645

Pictorial representation of illness and self measure (PRISM): an effective tool to assess the burden of psoriasis.

PubMed

Fotiou, K; Hofmann, M; Kaufmann, R; Thaci, D

2015-12-01

Psoriasis has a negative impact on patient's' quality of life, which may reflect the physical and psychosocial impact of the disease. However, little is known about the perceived burden of disease. Pictorial Representation of Illness and Self Measure (PRISM), a pictorial tool, was developed to measure this global suffering. Capturing quality of life impairments in psoriasis is an integral part of assessing the overall disease severity and planning effective treatment strategies. To validate Pictorial Representation of Illness and Self Measure (PRISM) in psoriasis patients. An investigator initiated, prospective cohort study was performed in 108 outpatients with psoriasis to validate PRISM as an appropriate tool; Reliability was assessed in a cohort with controlled stable disease, repeated after 1 hour and after a 3-month follow-up phase; Convergent validity determined by correlation against measures of life quality (DLQI) and reported severity of psoriasis (PASI, PGA and BSA) was performed. For PRISM, we found a high reliability for short-term measurements (ICC = 0.98, P < 0.001) and for follow-up assessments at 3 months (ICC = 0.76, P < 0.001). Our investigation showed significant differences between patients with clear to minimal (PGA 0, 1), mild (PGA 2) and moderate-to-severe disease (PGA 3, 4) and significant (P < 0.001) correlation with DLQI, PGA, PASI and BSA (ρ = -0.62, -0.54, -0.44 and -0.39 respectively). PRISM is a highly reliable tool and new approach to assess perceived burden of patients with psoriasis. It is easy to use and we recommend the application as an additional outcome measure in clinical trial settings. © 2015 European Academy of Dermatology and Venereology.

Judging the Neonatal Abstinence Syndrome Assessment Tools to Guide Future Tool Development: The use of Clinimetrics as Opposed to Psychometrics.

PubMed

Westgate, Philip M; Gomez-Pomar, Enrique

2017-01-01

In the face of the current Neonatal Abstinence Syndrome (NAS) epidemic, there is considerable variability in the assessment and management of infants with NAS. In this manuscript, we particularly focus on NAS assessment, with special attention given to the popular Finnegan Neonatal Abstinence Score (FNAS). A major instigator of the problem of variable practices is that multiple modified versions of the FNAS exist and continue to be proposed, including shortened versions. Furthermore, the validity of such assessment tools has been questioned, and as a result, the need for better tools has been suggested. The ultimate purpose of this manuscript, therefore, is to increase researchers' and clinicians' understanding on how to judge the usefulness of NAS assessment tools in order to guide future tool development and to reduce variable practices. In short, we suggest that judgment of NAS assessment tools should be made on a clinimetrics viewpoint as opposed to psychometrically. We provide examples, address multiple issues that must be considered, and discuss future tool development. Furthermore, we urge researchers and clinicians to come together, utilizing their knowledge and experience, to assess the utility and practicality of existing assessment tools and to determine if one or more new or modified tools are needed with the goal of increased agreement on the assessment of NAS in practice.

Design and validation of a questionnaire to assess organizational culture in French hospital wards.

PubMed

Saillour-Glénisson, F; Domecq, S; Kret, M; Sibe, M; Dumond, J P; Michel, P

2016-09-17

Although many organizational culture questionnaires have been developed, there is a lack of any validated multidimensional questionnaire assessing organizational culture at hospital ward level and adapted to health care context. Facing the lack of an appropriate tool, a multidisciplinary team designed and validated a dimensional organizational culture questionnaire for healthcare settings to be administered at ward level. A database of organizational culture items and themes was created after extensive literature review. Items were regrouped into dimensions and subdimensions (classification validated by experts). Pre-test and face validation was conducted with 15 health care professionals. In a stratified cluster random sample of hospitals, the psychometric validation was conducted in three phases on a sample of 859 healthcare professionals from 36 multidisciplinary medicine services: 1) the exploratory phase included a description of responses' saturation levels, factor and correlations analyses and an internal consistency analysis (Cronbach's alpha coefficient); 2) confirmatory phase used the Structural Equation Modeling (SEM); 3) reproducibility was studied by a test-retest. The overall response rate was 80 %; the completion average was 97 %. The metrological results were: a global Cronbach's alpha coefficient of 0.93, higher than 0.70 for 12 sub-dimensions; all Dillon-Goldstein's rho coefficients higher than 0.70; an excellent quality of external model with a Goodness of Fitness (GoF) criterion of 0.99. Seventy percent of the items had a reproducibility ranging from moderate (Intra-Class Coefficient between 50 and 70 % for 25 items) to good (ICC higher than 70 % for 33 items). COMEt (Contexte Organisationnel et Managérial en Etablissement de Santé) questionnaire is a validated multidimensional organizational culture questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first dimensional organizational culture questionnaire, specific to healthcare context, for a unit level assessment showing robust psychometric properties (validity and reliability). This tool is suited for research purposes, especially for assessing organizational context in research analysing the effectiveness of hospital quality improvement strategies. Our tool is also suited for an overall assessment of ward culture and could be a powerful trigger to improve management and clinical performance. Its psychometric properties in other health systems need to be tested.

Quadruplex digital flight control system assessment

NASA Technical Reports Server (NTRS)

Mulcare, D. B.; Downing, L. E.; Smith, M. K.

1988-01-01

Described are the development and validation of a double fail-operational digital flight control system architecture for critical pitch axis functions. Architectural tradeoffs are assessed, system simulator modifications are described, and demonstration testing results are critiqued. Assessment tools and their application are also illustrated. Ultimately, the vital role of system simulation, tailored to digital mechanization attributes, is shown to be essential to validating the airworthiness of full-time critical functions such as augmented fly-by-wire systems for relaxed static stability airplanes.

Development of a Brief Questionnaire to Assess Contraceptive Intent

PubMed Central

Raine-Bennett, Tina R; Rocca, Corinne H

2015-01-01

Objective We sought to develop and validate an instrument that can enable providers to identify young women who may be at risk of contraceptive non-adherence. Methods Item response theory based methods were used to evaluate the psychometric properties of the Contraceptive Intent Questionnaire, a 15-item self-administered questionnaire, based on theory and prior qualitative and quantitative research. The questionnaire was administered to 200 women aged 15–24 years who were initiating contraceptives. We assessed item fit to the item response model, internal consistency, internal structure validity, and differential item functioning. Results All items fit a one-dimensional model. The separation reliability coefficient was 0.73. Participants’ overall scores covered the full range of the scale (0–15), and items appropriately matched the range of participants’ contraceptive intent. Items met the criteria for internal structure validity and most items functioned similarly between groups of women. Conclusion The Contraceptive Intent Questionnaire appears to be a reliable and valid tool. Future testing is needed to assess predictive ability and clinical utility. Practice Implications The Contraceptive Intent Questionnaire may serve as a valid tool to help providers identify women who may have problems with contraceptive adherence, as well as to pinpoint areas in which counseling may be directed. PMID:26104994

Development of a brief questionnaire to assess contraceptive intent.

PubMed

Raine-Bennett, Tina R; Rocca, Corinne H

2015-11-01

We sought to develop and validate an instrument that can enable providers to identify young women who may be at risk of contraceptive non-adherence. Item response theory based methods were used to evaluate the psychometric properties of the Contraceptive Intent Questionnaire, a 15-item self-administered questionnaire, based on theory and prior qualitative and quantitative research. The questionnaire was administered to 200 women aged 15-24 years who were initiating contraceptives. We assessed item fit to the item response model, internal consistency, internal structure validity, and differential item functioning. All items fit a one-dimensional model. The separation reliability coefficient was 0.73. Participants' overall scores covered the full range of the scale (0-15), and items appropriately matched the range of participants' contraceptive intent. Items met the criteria for internal structure validity and most items functioned similarly between groups of women. The Contraceptive Intent Questionnaire appears to be a reliable and valid tool. Future testing is needed to assess predictive ability and clinical utility. The Contraceptive Intent Questionnaire may serve as a valid tool to help providers identify women who may have problems with contraceptive adherence, as well as to pinpoint areas in which counseling may be directed. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

[Validation, adaptation and translation of the MacCAT-T into Spanish: a tool to assess the ability to make health decisions].

PubMed

Hernando Robles, P; Lechuga Pérez, X; Solé Llop, P; Diestre, G; Mariné Torrent, A; Rodríguez Jornet, A; Marquina Parra, D; Colomer Mirabell, O

2012-01-01

Capacity assessment is an essential element of the informed consent process and is the duty of the physician. The MacCAT-T instrument explores four skills needed to consent a treatment. There is no Spanish version, and the main objective of this work is to validate, adapt and translate the MacCAT-T into Spanish. The MacCAT-T was translated into Spanish and then back-translated into English. It was validated as regards its appearance and content (by 15 experts), construct (inter-rater reliability and internal consistency) and criteria (the validity of an instrument by comparing it to some external criterion, in this case the Mini Examen Cognoscitivo de Lobo). Ninety medical and surgical outpatients over 18 years were included with no deficits of expression and/or severe disorders of consciousness that did not allow them to be interviewed. They have been optimal considering different types of validity. The average application time was between 9 and 13minutes. Data are consistent with those obtained in other applications of MacCAT-T in the English language and facilitate the provision of a Spanish tool for assessing capacity. Copyright © 2011 SECA. Published by Elsevier Espana. All rights reserved.

A Confirmatory Factor Analysis of the Student Evidence-Based Practice Questionnaire (S-EBPQ) in an Australian sample.

PubMed

Beccaria, Lisa; Beccaria, Gavin; McCosker, Catherine

2018-03-01

It is crucial that nursing students develop skills and confidence in using Evidence-Based Practice principles early in their education. This should be assessed with valid tools however, to date, few measures have been developed and applied to the student population. To examine the structural validity of the Student Evidence-Based Practice Questionnaire (S-EBPQ), with an Australian online nursing student cohort. A cross-sectional study for constructing validity. Three hundred and forty-five undergraduate nursing students from an Australian regional university were recruited across two semesters. Confirmatory Factor Analysis was used to examine the structural validity. Confirmatory Factor Analysis was applied which resulted in a good fitting model, based on a revised 20-item tool. The S-EBPQ tool remains a psychometrically robust measure of evidence-based practice use, attitudes, and knowledge and skills and can be applied in an online Australian student context. The findings of this study provided further evidence of the reliability and four factor structure of the S-EBPQ. Opportunities for further refinement of the tool may result in improvements in structural validity. Copyright © 2018 Elsevier Ltd. All rights reserved.

Life Satisfaction in Persons with Schizophrenia Living in the Community: Validation of the Satisfaction with Life Scale

ERIC Educational Resources Information Center

Wu, Chia-Huei; Wu, Chin-Yu

2008-01-01

Subjective well-being is an increasingly common indicator of adequacy of psychiatric services. An easy-to-administer assessment tool of subjective well-being that is conceptually sound, valid, and reliable is needed for use in persons with schizophrenia. The purpose of this paper was to validate the 5-item Satisfaction with Life Scale…

Design, development, testing and validation of a Photonics Virtual Laboratory for the study of LEDs

NASA Astrophysics Data System (ADS)

Naranjo, Francisco L.; Martínez, Guadalupe; Pérez, Ángel L.; Pardo, Pedro J.

2014-07-01

This work presents the design, development, testing and validation of a Photonic Virtual Laboratory, highlighting the study of LEDs. The study was conducted from a conceptual, experimental and didactic standpoint, using e-learning and m-learning platforms. Specifically, teaching tools that help ensure that our students perform significant learning have been developed. It has been brought together the scientific aspect, such as the study of LEDs, with techniques of generation and transfer of knowledge through the selection, hierarchization and structuring of information using concept maps. For the validation of the didactic materials developed, it has been used procedures with various assessment tools for the collection and processing of data, applied in the context of an experimental design. Additionally, it was performed a statistical analysis to determine the validity of the materials developed. The assessment has been designed to validate the contributions of the new materials developed over the traditional method of teaching, and to quantify the learning achieved by students, in order to draw conclusions that serve as a reference for its application in the teaching and learning processes, and comprehensively validate the work carried out.

Validation of the one pass measure for motivational interviewing competence.

PubMed

McMaster, Fiona; Resnicow, Ken

2015-04-01

This paper examines the psychometric properties of the OnePass coding system: a new, user-friendly tool for evaluating practitioner competence in motivational interviewing (MI). We provide data on reliability and validity with the current gold-standard: Motivational Interviewing Treatment Integrity tool (MITI). We compared scores from 27 videotaped MI sessions performed by student counselors trained in MI and simulated patients using both OnePass and MITI, with three different raters for each tool. Reliability was estimated using intra-class coefficients (ICCs), and validity was assessed using Pearson's r. OnePass had high levels of inter-rater reliability with 19/23 items found from substantial to almost perfect agreement. Taking the pair of scores with the highest inter-rater reliability on the MITI, the concurrent validity between the two measures ranged from moderate to high. Validity was highest for evocation, autonomy, direction and empathy. OnePass appears to have good inter-rater reliability while capturing similar dimensions of MI as the MITI. Despite the moderate concurrent validity with the MITI, the OnePass shows promise in evaluating both traditional and novel interpretations of MI. OnePass may be a useful tool for developing and improving practitioner competence in MI where access to MITI coders is limited. Copyright © 2015. Published by Elsevier Ireland Ltd.

The military social health index: a partial multicultural validation.

PubMed

Van Breda, Adrian D

2008-05-01

Routine military deployments place great stress on military families. Before South African soldiers can be deployed, they undergo a comprehensive health assessment, which includes a social work assessment. The assessment focuses on the resilience of the family system to estimate how well the family will cope when exposed to the stress of deployments. This article reports on the development and validation of a new measuring tool, the Military Social Health Index, or MSHI. The MSHI is made up of four scales, each comprising 14 items, viz social support, problem solving, stressor appraisal, and generalized resistance resources. An initial, large-scale, multicultural validation of the MSHI revealed strong levels of reliability (Cronbach a and standard error of measurement) and validity (factorial, construct, convergent, and discriminant).

A Known Group Analysis Validity Study of the Vanderbilt Assessment of Leadership in Education in US Elementary and Secondary Schools

ERIC Educational Resources Information Center

Covay Minor, Elizabeth; Porter, Andrew C.; Murphy, Joseph; Goldring, Ellen B.; Cravens, Xiu; Elloitt, Stephen N.

2014-01-01

The Vanderbilt Assessment of Leadership in Education (VAL-ED) provides educators with a tool for principal evaluation based on principal, teacher, and supervisor reports of principals' learning-centered leadership. In this study, we conduct a known group analysis as part of a larger argument for the validity of the VAL-ED in US elementary and…

The Development and Validation of the Indian Family Violence and Control Scale

PubMed Central

Kalokhe, Ameeta S.; Stephenson, Rob; Kelley, Mary E.; Dunkle, Kristin L.; Paranjape, Anuradha; Solas, Vikram; Karve, Latika; del Rio, Carlos; Sahay, Seema

2016-01-01

The high prevalence of domestic violence (DV) among married women in India and associated negative health repercussions highlight the need for effective prevention strategies and tools to measure the efficacy of such interventions. Literature supporting differing manifestations of DV by culture underscores the need for a culturally-tailored scale to more effectively measure DV in the Indian context. We therefore aimed to develop and validate such a tool, the Indian Family Violence and Control Scale (IFVCS), through a mixed-methods study. The psychometric development of IFVCS is herein discussed. After field pre-testing and expert review, a 63-item questionnaire was administered to a random sample of 630 married women from May-July 2013 in Pune, India. The item response theory approach for binary data to explore the IFVCS structure suggested that IFVCS is reliable, with the majority of items having high (>0.5) and significant factor loadings. Concurrent validity, assessed by comparing responses to IFVCS with the validated, abridged Conflict Tactics Scale-2, was high (r = 0.899, p<0.001) as was the construct validity, demonstrated by its significant association with several established DV correlates. Therefore, initial assessment of the IFVCS psychometric properties suggests that it is an effective tool for measuring DV among married women in India and speaks to its capacity for enhancing understanding of DV epidemiology and for evaluating the effectiveness of future DV interventions. PMID:26824611

The Development and Validation of the Indian Family Violence and Control Scale.

PubMed

Kalokhe, Ameeta S; Stephenson, Rob; Kelley, Mary E; Dunkle, Kristin L; Paranjape, Anuradha; Solas, Vikram; Karve, Latika; del Rio, Carlos; Sahay, Seema

2016-01-01

The high prevalence of domestic violence (DV) among married women in India and associated negative health repercussions highlight the need for effective prevention strategies and tools to measure the efficacy of such interventions. Literature supporting differing manifestations of DV by culture underscores the need for a culturally-tailored scale to more effectively measure DV in the Indian context. We therefore aimed to develop and validate such a tool, the Indian Family Violence and Control Scale (IFVCS), through a mixed-methods study. The psychometric development of IFVCS is herein discussed. After field pre-testing and expert review, a 63-item questionnaire was administered to a random sample of 630 married women from May-July 2013 in Pune, India. The item response theory approach for binary data to explore the IFVCS structure suggested that IFVCS is reliable, with the majority of items having high (>0.5) and significant factor loadings. Concurrent validity, assessed by comparing responses to IFVCS with the validated, abridged Conflict Tactics Scale-2, was high (r = 0.899, p<0.001) as was the construct validity, demonstrated by its significant association with several established DV correlates. Therefore, initial assessment of the IFVCS psychometric properties suggests that it is an effective tool for measuring DV among married women in India and speaks to its capacity for enhancing understanding of DV epidemiology and for evaluating the effectiveness of future DV interventions.

Developing a Questionnaire for Iranian Women's Attitude on Medical Ethics in Vaginal Childbirth.

PubMed

Mirzaee Rabor, Firoozeh; Taghipour, Ali; Mirzaee, Moghaddameh; Mirzaii Najmabadi, Khadigeh; Fazilat Pour, Masoud; Fattahi Masoum, Seyed Hosein

2015-12-01

Vaginal delivery is one of the challenging issues in medical ethics. It is important to use an appropriate instrument to assess medical ethics attitudes in normal delivery, but the lack of tool for this purpose is clear. The aim of this study was to develop and validate a questionnaire for the assessment of women's attitude on medical ethics application in normal vaginal delivery. This methodological study was carried out in Iran in 2013 - 2014. Medical ethics attitude in vaginal delivery questionnaire (MEAVDQ) was developed using the findings of a qualitative data obtained from a grounded theory research conducted on 20 women who had vaginal childbirth, in the first phase. Then, the validation criteria of this tool were tested by content and face validity in the second phase. Exploratory factor analysis was used for construct validity and reliability was also tested by Cronbach's alpha coefficient in the third phase of this study. SPSS version 13 was used in this study. The sample size for construct validity was 250 females who had normal vaginal childbirth. In the first phase of this study (tool development), by the use of four obtained categories and nine subcategories from grounded theory and literature review, three parts (98-items) of this tool were obtained (A, B and J). Part A explained the first principle of medical ethics, part B pointed to the second and third principles of medical ethics, and part J explained the fourth principle of medical ethics. After evaluating and confirming its face and content validity, 75 items remained in the questionnaire. In construct validity, by the employment of exploratory factor analysis, in parts A, B and J, 3, 7 and 3 factors were formed, respectively; and 62.8%, 64% and 51% of the total variances were explained by the obtained factors in parts A, B and J, respectively. The names of these factors in the three parts were achieved by consideration of the loading factor and medical ethics principles. The subscales of MEAVDQ showed significant reliability. In parts A, B and J, Cronbach's alpha coefficients were 0.76, 0.72 and 0.68, respectively and for the total questionnaire, it was 0.72. The results of the test-retest were satisfactory for all the items (ICC = 0.60 - 0.95). The present study showed that the 59-item MEAVDQ was a valid and reliable questionnaire for the assessment of women's attitudes toward medical ethics application in vaginal childbirth. This tool might assist specialists in making a judgment and plan appropriate for women in vaginal delivery management.

Validation of the French version of the yale food addiction scale: an examination of its factor structure, reliability, and construct validity in a nonclinical sample.

PubMed

Brunault, Paul; Ballon, Nicolas; Gaillard, Philippe; Réveillère, Christian; Courtois, Robert

2014-05-01

The concept of food addiction has recently been proposed by applying the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision, criteria for substance dependence to eating behaviour. Food addiction has received increased attention given that it may play a role in binge eating, eating disorders, and the recent increase in obesity prevalence. Currently, there is no psychometrically sound tool for assessing food addiction in French. Our study aimed to test the psychometric properties of a French version of the Yale Food Addiction Scale (YFAS) by establishing its factor structure and construct validity in a nonclinical population. A total of 553 participants were assessed for food addiction (French version of the YFAS) and binge eating behaviour (Bulimic Investigatory Test Edinburgh and Binge Eating Scale). We tested the scale's factor structure (factor analysis for dichotomous data based on tetrachoric correlation coefficients), internal consistency, and construct validity with measures of binge eating. Our results supported a 1-factor structure, which accounted for 54.1% of the variance. This tool had adequate reliability and high construct validity with measures of binge eating in this population, both in its diagnosis and symptom count version. A 2-factor structure explained an additional 9.1% of the variance, and could differentiate between patients with high, compared with low, levels of insight regarding addiction symptoms. In our study, we validated a psychometrically sound French version of the YFAS, both in its symptom count and diagnostic version. Future studies should validate this tool in clinical samples.

The Optimal Screening for Prediction of Referral and Outcome (OSPRO) in patients with musculoskeletal pain conditions: a longitudinal validation cohort from the USA

PubMed Central

George, Steven Z; Beneciuk, Jason M; Lentz, Trevor A; Wu, Samuel S

2017-01-01

Purpose There is an increased need for determining which patients with musculoskeletal pain benefit from additional diagnostic testing or psychologically informed intervention. The Optimal Screening for Prediction of Referral and Outcome (OSPRO) cohort studies were designed to develop and validate standard assessment tools for review of systems and yellow flags. This cohort profile paper provides a description of and future plans for the validation cohort. Participants Patients (n=440) with primary complaint of spine, shoulder or knee pain were recruited into the OSPRO validation cohort via a national Orthopaedic Physical Therapy-Investigative Network. Patients were followed up at 4 weeks, 6 months and 12 months for pain, functional status and quality of life outcomes. Healthcare utilisation outcomes were also collected at 6 and 12 months. Findings to date There are no longitudinal findings reported to date from the ongoing OSPRO validation cohort. The previously completed cross-sectional OSPRO development cohort yielded two assessment tools that were investigated in the validation cohort. Future plans Follow-up data collection was completed in January 2017. Primary analyses will investigate how accurately the OSPRO review of systems and yellow flag tools predict 12-month pain, functional status, quality of life and healthcare utilisation outcomes. Planned secondary analyses include prediction of pain interference and/or development of chronic pain, investigation of treatment expectation on patient outcomes and analysis of patient satisfaction following an episode of physical therapy. Trial registration number The OSPRO validation cohort was not registered. PMID:28600371

The Individualized Classroom Assessment Scoring System (inCLASS): Preliminary Reliability and Validity of a System for Observing Preschoolers’ Competence in Classroom Interactions

PubMed Central

Downer, Jason T.; Booren, Leslie M.; Lima, Olivia K.; Luckner, Amy E.; Pianta, Robert C.

2012-01-01

This paper introduces the Individualized Classroom Assessment Scoring System (inCLASS), an observation tool that targets children’s interactions in preschool classrooms with teachers, peers, and tasks. In particular, initial evidence is reported of the extent to which the inCLASS meets the following psychometric criteria: inter-rater reliability, normal distributions and adequate range, construct validity, and criterion-related validity. These initial findings suggest that the inCLASS has the potential to provide an authentic, contextualized assessment of young children’s classroom behaviors. Future directions for research with the inCLASS are discussed. PMID:23175598

Data-driven traffic impact assessment tool for work zones.

DOT National Transportation Integrated Search

2017-03-01

Traditionally, traffic impacts of work zones have been assessed using planning software such as Quick Zone, custom spreadsheets, and others. These software programs generate delay, queuing, and other mobility measures but are difficult to validate du...

Exploring the reliability and validity of the social-moral awareness test.

PubMed

Livesey, Alexandra; Dodd, Karen; Pote, Helen; Marlow, Elizabeth

2012-11-01

The aim of the study was to explore the validity of the social-moral awareness test (SMAT) a measure designed for assessing socio-moral rule knowledge and reasoning in people with learning disabilities. Comparisons between Theory of Mind and socio-moral reasoning allowed the exploration of construct validity of the tool. Factor structure, reliability and discriminant validity were also assessed. Seventy-one participants with mild-moderate learning disabilities completed the two scales of the SMAT and two False Belief Tasks for Theory of Mind. Reliability of the SMAT was very good, and the scales were shown to be uni-dimensional in factor structure. There was a significant positive relationship between Theory of Mind and both SMAT scales. There is early evidence of the construct validity and reliability of the SMAT. Further assessment of the validity of the SMAT will be required. © 2012 Blackwell Publishing Ltd.

Inter-agency communication and operations capabilities during a hospital functional exercise: reliability and validity of a measurement tool.

PubMed

Savoia, Elena; Biddinger, Paul D; Burstein, Jon; Stoto, Michael A

2010-01-01

As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise. The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement. Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self reported scores and external evaluators' scores ranged from moderate to good. The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.

Validation of an iPad activity to measure preschool children's food and physical activity knowledge and preferences.

PubMed

Wiseman, Nicola; Harris, Neil; Downes, Martin

2017-02-01

Preschool children's knowledge of, and preference for food and physical activity play an important role in the development of lifestyle behaviors throughout childhood. Valid and reliable instruments that are interactive and appealing to preschool children are needed, to obtain quality information in a way that actively engages children and encourages willing participation. The purpose of the current research is to assess the reliability and validity of an adapted computerized (iPad) version of the photo-pair food and exercise questionnaire (PPFEQ). The adaptation of the PPFEQ involved generating the questionnaire as an iPad-based tool, updating the photo-pairs within the questionnaire and testing for validity and reliability. This involved four phases of investigation to assess test-retest reliability, internal consistency, sensitivity to change and percent agreement of the questionnaire. The adaption of the PPFEQ resulted in an 18-item questionnaire, titled the preschool food and play questionnaire (Pre-FPQ). The Pre-FPQ demonstrated acceptable reliability and sensitivity to change. Test-retest reliability and internal consistency improved with age, however, it was evident that the tool was not suitable for children younger than 4 years of age. Children encounter a dynamic world that shapes their knowledge, preferences, choices and behaviors. The Pre-FPQ is an innovative tool to measure preschool children's knowledge of and preference for food and physical activity. The questionnaire offers the advantage of being presented in a well-received modality for preschool children as well as being easy and inexpensive to administer. This new tool is likely to be useful for the assessment of the effectiveness of healthy lifestyle programs implemented in the childcare setting. Future work is needed to refine and improve measures of physical activity preference in preschool children.

Validity and reliability of Chinese version of Adult Carer Quality of Life questionnaire (AC-QoL) in family caregivers of stroke survivors

PubMed Central

Li, Yingshuang; Ding, Chunge

2017-01-01

The Adult Carer Quality of Life questionnaire (AC-QoL) is a reliable and valid instrument used to assess the quality of life (QoL) of adult family caregivers. We explored the psychometric properties and tested the reliability and validity of a Chinese version of the AC-QoL with reliability and validity testing in 409 Chinese stroke caregivers. We used item-total correlation and extreme group comparison to do item analysis. To evaluate its reliability, we used a test-retest reliability approach, intraclass correlation coefficient (ICC), together with Cronbach’s alpha and model-based internal consistency index; to evaluate its validity, we used scale content validity, confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) via principal component analysis with varimax rotation. We found that the CFA did not in fact confirm the original factor model and our EFA yielded a 31-item measure with a five-factor model. In conclusions, although some items performed differently in our analysis of the original English language version and our Chinese language version, our translated AC-QoL is a reliable and valid tool which can be used to assess the quality of life of stroke caregivers in mainland China. Chinese version AC-QoL is a comprehensive and good measurement to understand caregivers and has the potential to be a screening tool to assess QoL of caregiver. PMID:29131845

A Psychometric Study of the Bayley Scales of Infant and Toddler Development in Persian Language Children.

PubMed

Azari, Nadia; Soleimani, Farin; Vameghi, Roshanak; Sajedi, Firoozeh; Shahshahani, Soheila; Karimi, Hossein; Kraskian, Adis; Shahrokhi, Amin; Teymouri, Robab; Gharib, Masoud

2017-01-01

Bayley Scales of infant & toddler development is a well-known diagnostic developmental assessment tool for children aged 1-42 months. Our aim was investigating the validity & reliability of this scale in Persian speaking children. The method was descriptive-analytic. Translation- back translation and cultural adaptation was done. Content & face validity of translated scale was determined by experts' opinions. Overall, 403 children aged 1 to 42 months were recruited from health centers of Tehran, during years of 2013-2014 for developmental assessment in cognitive, communicative (receptive & expressive) and motor (fine & gross) domains. Reliability of scale was calculated through three methods; internal consistency using Cronbach's alpha coefficient, test-retest and interrater methods. Construct validity was calculated using factor analysis and comparison of the mean scores methods. Cultural and linguistic changes were made in items of all domains especially on communication subscale. Content and face validity of the test were approved by experts' opinions. Cronbach's alpha coefficient was above 0.74 in all domains. Pearson correlation coefficient in various domains, were ≥ 0.982 in test retest method, and ≥0.993 in inter-rater method. Construct validity of the test was approved by factor analysis. Moreover, the mean scores for the different age groups were compared and statistically significant differences were observed between mean scores of different age groups, that confirms validity of the test. The Bayley Scales of Infant and Toddler Development is a valid and reliable tool for child developmental assessment in Persian language children.

Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT) among professionals in Dutch disease-management partnerships

PubMed Central

2011-01-01

Background The extent to which partnership synergy is created within quality improvement programmes in the Netherlands is unknown. In this article, we describe the psychometric testing of the Partnership Self-Assessment Tool (PSAT) among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability. Methods The Dutch version of the PSAT was tested in twenty-two disease-management partnerships with 218 professionals. We tested the instrument by means of structural equation modelling, and examined its validity and reliability. Results After eliminating 14 items, the confirmatory factor analyses revealed good indices of fit with the resulting 15-item PSAT-Short version (PSAT-S). Internal consistency as represented by Cronbach's alpha ranged from acceptable (0.75) for the 'efficiency' subscale to excellent for the 'leadership' subscale (0.87). Convergent validity was provided with high correlations of the partnership dimensions and partnership synergy (ranged from 0.512 to 0.609) and high correlations with chronic illness care (ranged from 0.447 to 0.329). Conclusion The psychometric properties and convergent validity of the PSAT-S were satisfactory rendering it a valid and reliable instrument for assessing partnership synergy and its dimensions of partnership functioning. PMID:21714931

Using patients' charts to assess medical trainees in the workplace: a systematic review.

PubMed

Al-Wassia, Heidi; Al-Wassia, Rolina; Shihata, Shadi; Park, Yoon Soo; Tekian, Ara

2015-04-01

The objective of this review is to summarize and critically appraise existing evidence on the use of chart stimulated recall (CSR) and case-based discussion (CBD) as an assessment tool for medical trainees. Medline, Embase, CINAHL, PsycINFO, Educational Resources Information Centre (ERIC), Web of Science, and the Cochrane Central Register of Controlled Trials were searched for original articles on the use of CSR or CBD as an assessment method for trainees in all medical specialties. Four qualitative and three observational non-comparative studies were eligible for this review. The number of patient-chart encounters needed to achieve sufficient reliability varied across studies. None of the included studies evaluated the content validity of the tool. Both trainees and assessors expressed high level of satisfaction with the tool; however, inadequate training, different interpretation of the scoring scales and skills needed to give feedback were addressed as limitations for conducting the assessment. There is still no compelling evidence for the use of patient's chart to evaluate medical trainees in the workplace. A body of evidence that is valid, reliable, and documents the educational effect in support of the use of patients' charts to assess medical trainees is needed.

Dietary assessment in minority ethnic groups: a systematic review of instruments for portion-size estimation in the United Kingdom

PubMed Central

Almiron-Roig, Eva; Aitken, Amanda; Galloway, Catherine

2017-01-01

Context: Dietary assessment in minority ethnic groups is critical for surveillance programs and for implementing effective interventions. A major challenge is the accurate estimation of portion sizes for traditional foods and dishes. Objective: The aim of this systematic review was to assess records published up to 2014 describing a portion-size estimation element (PSEE) applicable to the dietary assessment of UK-residing ethnic minorities. Data sources, selection, and extraction: Electronic databases, internet sites, and theses repositories were searched, generating 5683 titles, from which 57 eligible full-text records were reviewed. Data analysis: Forty-two publications about minority ethnic groups (n = 20) or autochthonous populations (n = 22) were included. The most common PSEEs (47%) were combination tools (eg, food models and portion-size lists), followed by portion-size lists in questionnaires/guides (19%) and image-based and volumetric tools (17% each). Only 17% of PSEEs had been validated against weighed data. Conclusions: When developing ethnic-specific dietary assessment tools, it is important to consider customary portion sizes by sex and age, traditional household utensil usage, and population literacy levels. Combining multiple PSEEs may increase accuracy, but such methods require validation. PMID:28340101

Evaluation of the efficacy of six nutritional screening tools to predict malnutrition in the elderly.

PubMed

Poulia, Kalliopi-Anna; Yannakoulia, Mary; Karageorgou, Dimitra; Gamaletsou, Maria; Panagiotakos, Demosthenes B; Sipsas, Nikolaos V; Zampelas, Antonis

2012-06-01

Malnutrition in the elderly is a multifactorial problem, more prevalent in hospitals and care homes. The absence of a gold standard in evaluating nutritional risk led us to evaluate the efficacy of six nutritional screening tools used in the elderly. Two hundred forty eight elderly patients (129 men, 119 female women, aged 75.2 ± 8.5 years) were examined. Nutritional screening was performed on admission using the following tools: Nutritional Risk Index (NRI), Geriatric Nutritional Risk Index (GNRI), Subjective Global Assessment (SGA), Mini Nutritional Assessment - Screening Form (MNA-SF), Malnutrition Universal Screening Tool (MUST) and Nutritional Risk Screening 2002 (NRS 2002). A combined index for malnutrition was also calculated. Nutritional risk and/or malnutrition varied greatly, ranging from 47.2 to 97.6%, depending on the nutritional screening tool used. MUST was the most valid screening tool (validity coefficient = 0.766, CI 95%: 0.690-0.841), while SGA was in better agreement with the combined index (κ = 0.707, p = 0.000). NRS 2002 although was the highest in sensitivity (99.4%), it was the lowest in specificity (6.1%) and positive predictive value (68.2%). MUST seem to be the most valid in the evaluation of the risk for malnutrition in the elderly upon admission to the hospital. NRS 2002 was found to overestimate nutritional risk in the elderly. Copyright © 2011 Elsevier Ltd and European Society for Clinical Nutrition and Metabolism. All rights reserved.

The PRECIS-2 tool has good interrater reliability and modest discriminant validity.

PubMed

Loudon, Kirsty; Zwarenstein, Merrick; Sullivan, Frank M; Donnan, Peter T; Gágyor, Ildikó; Hobbelen, Hans J S M; Althabe, Fernando; Krishnan, Jerry A; Treweek, Shaun

2017-08-01

PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders. Copyright © 2017 Elsevier Inc. All rights reserved.

How to Assess the External Validity and Model Validity of Therapeutic Trials: A Conceptual Approach to Systematic Review Methodology

PubMed Central

2014-01-01

Background. Evidence rankings do not consider equally internal (IV), external (EV), and model validity (MV) for clinical studies including complementary and alternative medicine/integrative medicine (CAM/IM) research. This paper describe this model and offers an EV assessment tool (EVAT©) for weighing studies according to EV and MV in addition to IV. Methods. An abbreviated systematic review methodology was employed to search, assemble, and evaluate the literature that has been published on EV/MV criteria. Standard databases were searched for keywords relating to EV, MV, and bias-scoring from inception to Jan 2013. Tools identified and concepts described were pooled to assemble a robust tool for evaluating these quality criteria. Results. This study assembled a streamlined, objective tool to incorporate for the evaluation of quality of EV/MV research that is more sensitive to CAM/IM research. Conclusion. Improved reporting on EV can help produce and provide information that will help guide policy makers, public health researchers, and other scientists in their selection, development, and improvement in their research-tested intervention. Overall, clinical studies with high EV have the potential to provide the most useful information about “real-world” consequences of health interventions. It is hoped that this novel tool which considers IV, EV, and MV on equal footing will better guide clinical decision making. PMID:24734111

Screening and assessment tools for pediatric malnutrition.

PubMed

Huysentruyt, Koen; Vandenplas, Yvan; De Schepper, Jean

2016-06-18

The ideal measures for screening and assessing undernutrition in children remain a point of discussion in literature. This review aims to provide an overview of recent advances in the nutritional screening and assessment methods in children. This review focuses on two major topics that emerged in literature since 2015: the practical endorsement of the new definition for pediatric undernutrition, with a focus on anthropometric measurements and the search for a consensus on pediatric nutritional screening tools in different settings. Few analytical tools exist for the assessment of the nutritional status in children. The subjective global nutritional assessment has been validated by anthropometric as well as clinical outcome parameters. Nutritional screening can help in selecting patients that benefit the most from a full nutritional assessment. Two new screening tools have been developed for use in a general (mixed) hospital population, and one for a population of children with cancer. The value of screening tools in different disease-specific and outpatient pediatric populations remains to be proven.