Sample records for validated assessment tools

  1. Visual Impairment Screening Assessment (VISA) tool: pilot validation.

    PubMed

    Rowe, Fiona J; Hepworth, Lauren R; Hanna, Kerry L; Howard, Claire

    2018-03-06

    To report and evaluate a new Vision Impairment Screening Assessment (VISA) tool intended for use by the stroke team to improve identification of visual impairment in stroke survivors. Prospective case cohort comparative study. Stroke units at two secondary care hospitals and one tertiary centre. 116 stroke survivors were screened, 62 by naïve and 54 by non-naïve screeners. Both the VISA screening tool and the comprehensive specialist vision assessment measured case history, visual acuity, eye alignment, eye movements, visual field and visual inattention. Full completion of VISA tool and specialist vision assessment was achieved for 89 stroke survivors. Missing data for one or more sections typically related to patient's inability to complete the assessment. Sensitivity and specificity of the VISA screening tool were 90.24% and 85.29%, respectively; the positive and negative predictive values were 93.67% and 78.36%, respectively. Overall agreement was significant; k=0.736. Lowest agreement was found for screening of eye movement and visual inattention deficits. This early validation of the VISA screening tool shows promise in improving detection accuracy for clinicians involved in stroke care who are not specialists in vision problems and lack formal eye training, with potential to lead to more prompt referral with fewer false positives and negatives. Pilot validation indicates acceptability of the VISA tool for screening of visual impairment in stroke survivors. Sensitivity and specificity were high indicating the potential accuracy of the VISA tool for screening purposes. Results of this study have guided the revision of the VISA screening tool ahead of full clinical validation. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

  2. Content Validation and Evaluation of an Endovascular Teamwork Assessment Tool.

    PubMed

    Hull, L; Bicknell, C; Patel, K; Vyas, R; Van Herzeele, I; Sevdalis, N; Rudarakanchana, N

    2016-07-01

    To modify, content validate, and evaluate a teamwork assessment tool for use in endovascular surgery. A multistage, multimethod study was conducted. Stage 1 included expert review and modification of the existing Observational Teamwork Assessment for Surgery (OTAS) tool. Stage 2 included identification of additional exemplar behaviours contributing to effective teamwork and enhanced patient safety in endovascular surgery (using real-time observation, focus groups, and semistructured interviews of multidisciplinary teams). Stage 3 included content validation of exemplar behaviours using expert consensus according to established psychometric recommendations and evaluation of structure, content, feasibility, and usability of the Endovascular Observational Teamwork Assessment Tool (Endo-OTAS) by an expert multidisciplinary panel. Stage 4 included final team expert review of exemplars. OTAS core team behaviours were maintained (communication, coordination, cooperation, leadership team monitoring). Of the 114 OTAS behavioural exemplars, 19 were modified, four removed, and 39 additional endovascular-specific behaviours identified. Content validation of these 153 exemplar behaviours showed that 113/153 (73.9%) reached the predetermined Item-Content Validity Index rating for teamwork and/or patient safety. After expert team review, 140/153 (91.5%) exemplars were deemed to warrant inclusion in the tool. More than 90% of the expert panel agreed that Endo-OTAS is an appropriate teamwork assessment tool with observable behaviours. Some concerns were noted about the time required to conduct observations and provide performance feedback. Endo-OTAS is a novel teamwork assessment tool, with evidence for content validity and relevance to endovascular teams. Endo-OTAS enables systematic objective assessment of the quality of team performance during endovascular procedures. Copyright © 2016. Published by Elsevier Ltd.

  3. Development and validation of a Clinical Assessment Tool for Nursing Education (CAT-NE).

    PubMed

    Skúladóttir, Hafdís; Svavarsdóttir, Margrét Hrönn

    2016-09-01

    The aim of this study was to develop a valid assessment tool to guide clinical education and evaluate students' performance in clinical nursing education. The development of the Clinical Assessment Tool for Nursing Education (CAT-NE) was based on the theory of nursing as professional caring and the Bologna learning outcomes. Benson and Clark's four steps of instrument development and validation guided the development and assessment of the tool. A mixed-methods approach with individual structured cognitive interviewing and quantitative assessments was used to validate the tool. Supervisory teachers, a pedagogical consultant, clinical expert teachers, clinical teachers, and nursing students at the University of Akureyri in Iceland participated in the process. This assessment tool is valid to assess the clinical performance of nursing students; it consists of rubrics that list the criteria for the students' expected performance. According to the students and their clinical teachers, the assessment tool clarified learning objectives, enhanced the focus of the assessment process, and made evaluation more objective. Training clinical teachers on how to assess students' performances in clinical studies and use the tool enhanced the quality of clinical assessment in nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

  4. Developing and Validating a New Classroom Climate Observation Assessment Tool

    PubMed Central

    Leff, Stephen S.; Thomas, Duane E.; Shapiro, Edward S.; Paskewich, Brooke; Wilson, Kim; Necowitz-Hoffman, Beth; Jawad, Abbas F.

    2011-01-01

    The climate of school classrooms, shaped by a combination of teacher practices and peer processes, is an important determinant for children’s psychosocial functioning and is a primary factor affecting bullying and victimization. Given that there are relatively few theoretically-grounded and validated assessment tools designed to measure the social climate of classrooms, our research team developed an observation tool through participatory action research (PAR). This article details how the assessment tool was designed and preliminarily validated in 18 third-, fourth-, and fifth-grade classrooms in a large urban public school district. The goals of this study are to illustrate the feasibility of a PAR paradigm in measurement development, ascertain the psychometric properties of the assessment tool, and determine associations with different indices of classroom levels of relational and physical aggression. PMID:21643447

  5. Developing and Validating a New Classroom Climate Observation Assessment Tool.

    PubMed

    Leff, Stephen S; Thomas, Duane E; Shapiro, Edward S; Paskewich, Brooke; Wilson, Kim; Necowitz-Hoffman, Beth; Jawad, Abbas F

    2011-01-01

    The climate of school classrooms, shaped by a combination of teacher practices and peer processes, is an important determinant for children's psychosocial functioning and is a primary factor affecting bullying and victimization. Given that there are relatively few theoretically-grounded and validated assessment tools designed to measure the social climate of classrooms, our research team developed an observation tool through participatory action research (PAR). This article details how the assessment tool was designed and preliminarily validated in 18 third-, fourth-, and fifth-grade classrooms in a large urban public school district. The goals of this study are to illustrate the feasibility of a PAR paradigm in measurement development, ascertain the psychometric properties of the assessment tool, and determine associations with different indices of classroom levels of relational and physical aggression.

  6. Development and Validity Testing of an Arthritis Self-Management Assessment Tool.

    PubMed

    Oh, HyunSoo; Han, SunYoung; Kim, SooHyun; Seo, WhaSook

    Because of the chronic, progressive nature of arthritis and the substantial effects it has on quality of life, patients may benefit from self-management. However, no valid, reliable self-management assessment tool has been devised for patients with arthritis. This study was conducted to develop a comprehensive self-management assessment tool for patients with arthritis, that is, the Arthritis Self-Management Assessment Tool (ASMAT). To develop a list of qualified items corresponding to the conceptual definitions and attributes of arthritis self-management, a measurement model was established on the basis of theoretical and empirical foundations. Content validity testing was conducted to evaluate whether listed items were suitable for assessing arthritis self-management. Construct validity and reliability of the ASMAT were tested. Construct validity was examined using confirmatory factor analysis and nomological validity. The 32-item ASMAT was developed with a sample composed of patients in a clinic in South Korea. Content validity testing validated the 32 items, which comprised medical (10 items), behavioral (13 items), and psychoemotional (9 items) management subscales. Construct validity testing of the ASMAT showed that the 32 items properly corresponded with conceptual constructs of arthritis self-management, and were suitable for assessing self-management ability in patients with arthritis. Reliability was also well supported. The ASMAT devised in the present study may aid the evaluation of patient self-management ability and the effectiveness of self-management interventions. The authors believe the developed tool may also aid the identification of problems associated with the adoption of self-management practice, and thus improve symptom management, independence, and quality of life of patients with arthritis.

  7. Understanding Interrater Reliability and Validity of Risk Assessment Tools Used to Predict Adverse Clinical Events.

    PubMed

    Siedlecki, Sandra L; Albert, Nancy M

    This article will describe how to assess interrater reliability and validity of risk assessment tools, using easy-to-follow formulas, and to provide calculations that demonstrate principles discussed. Clinical nurse specialists should be able to identify risk assessment tools that provide high-quality interrater reliability and the highest validity for predicting true events of importance to clinical settings. Making best practice recommendations for assessment tool use is critical to high-quality patient care and safe practices that impact patient outcomes and nursing resources. Optimal risk assessment tool selection requires knowledge about interrater reliability and tool validity. The clinical nurse specialist will understand the reliability and validity issues associated with risk assessment tools, and be able to evaluate tools using basic calculations. Risk assessment tools are developed to objectively predict quality and safety events and ultimately reduce the risk of event occurrence through preventive interventions. To ensure high-quality tool use, clinical nurse specialists must critically assess tool properties. The better the tool's ability to predict adverse events, the more likely that event risk is mediated. Interrater reliability and validity assessment is relatively an easy skill to master and will result in better decisions when selecting or making recommendations for risk assessment tool use.

  8. Development and validation of the TOCO-TURBT tool: a summative assessment tool that measures surgical competency in transurethral resection of bladder tumour.

    PubMed

    de Vries, Anna H; Muijtjens, Arno M M; van Genugten, Hilde G J; Hendrikx, Ad J M; Koldewijn, Evert L; Schout, Barbara M A; van der Vleuten, Cees P M; Wagner, Cordula; Tjiam, Irene M; van Merriënboer, Jeroen J G

    2018-06-05

    The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT). The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory. The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases. This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in

  9. Development and validation of a Malawian version of the primary care assessment tool.

    PubMed

    Dullie, Luckson; Meland, Eivind; Hetlevik, Øystein; Mildestvedt, Thomas; Gjesdal, Sturla

    2018-05-16

    Malawi does not have validated tools for assessing primary care performance from patients' experience. The aim of this study was to develop a Malawian version of Primary Care Assessment Tool (PCAT-Mw) and to evaluate its reliability and validity in the assessment of the core primary care dimensions from adult patients' perspective in Malawi. A team of experts assessed the South African version of the primary care assessment tool (ZA-PCAT) for face and content validity. The adapted questionnaire underwent forward and backward translation and a pilot study. The tool was then used in an interviewer administered cross-sectional survey in Neno district, Malawi, to test validity and reliability. Exploratory factor analysis was performed on a random half of the sample to evaluate internal consistency, reliability and construct validity of items and scales. The identified constructs were then tested with confirmatory factor analysis. Likert scale assumption testing and descriptive statistics were done on the final factor structure. The PCAT-Mw was further tested for intra-rater and inter-rater reliability. From the responses of 631 patients, a 29-item PCAT-Mw was constructed comprising seven multi-item scales, representing five primary care dimensions (first contact, continuity, comprehensiveness, coordination and community orientation). All the seven scales achieved good internal consistency, item-total correlations and construct validity. Cronbach's alpha coefficient ranged from 0.66 to 0.91. A satisfactory goodness of fit model was achieved (GFI = 0.90, CFI = 0.91, RMSEA = 0.05, PCLOSE = 0.65). The full range of possible scores was observed for all scales. Scaling assumptions tests were achieved for all except the two comprehensiveness scales. Intra-class correlation coefficient (ICC) was 0.90 (n = 44, 95% CI 0.81-0.94, p < 0.001) for intra-rater reliability and 0.84 (n = 42, 95% CI 0.71-0.96, p < 0.001) for inter-rater reliability

  10. Development and Validation of a Family Meeting Assessment Tool (FMAT).

    PubMed

    Hagiwara, Yuya; Healy, Jennifer; Lee, Shuko; Ross, Jeanette; Fischer, Dixie; Sanchez-Reilly, Sandra

    2018-01-01

    A cornerstone procedure in Palliative Medicine is to perform family meetings. Learning how to lead a family meeting is an important skill for physicians and others who care for patients with serious illnesses and their families. There is limited evidence on how to assess best practice behaviors during end-of-life family meetings. Our aim was to develop and validate an observational tool to assess trainees' ability to lead a simulated end-of-life family meeting. Building on evidence from published studies and accrediting agency guidelines, an expert panel at our institution developed the Family Meeting Assessment Tool. All fourth-year medical students (MS4) and eight geriatric and palliative medicine fellows (GPFs) were invited to participate in a Family Meeting Objective Structured Clinical Examination, where each trainee assumed the physician role leading a complex family meeting. Two evaluators observed and rated randomly chosen students' performances using the Family Meeting Assessment Tool during the examination. Inter-rater reliability was measured using percent agreement. Internal consistency was measured using Cronbach α. A total of 141 trainees (MS4 = 133 and GPF = 8) and 26 interdisciplinary evaluators participated in the study. Internal reliability (Cronbach α) of the tool was 0.85. Number of trainees rated by two evaluators was 210 (MS4 = 202 and GPF = 8). Rater agreement was 84%. Composite scores, on average, were significantly higher for fellows than for medical students (P < 0.001). Expert-based content, high inter-rater reliability, good internal consistency, and ability to predict educational level provided initial evidence for construct validity for this novel assessment tool. Copyright © 2017 American Academy of Hospice and Palliative Medicine. All rights reserved.

  11. Embedded performance validity testing in neuropsychological assessment: Potential clinical tools.

    PubMed

    Rickards, Tyler A; Cranston, Christopher C; Touradji, Pegah; Bechtold, Kathleen T

    2018-01-01

    The article aims to suggest clinically-useful tools in neuropsychological assessment for efficient use of embedded measures of performance validity. To accomplish this, we integrated available validity-related and statistical research from the literature, consensus statements, and survey-based data from practicing neuropsychologists. We provide recommendations for use of 1) Cutoffs for embedded performance validity tests including Reliable Digit Span, California Verbal Learning Test (Second Edition) Forced Choice Recognition, Rey-Osterrieth Complex Figure Test Combination Score, Wisconsin Card Sorting Test Failure to Maintain Set, and the Finger Tapping Test; 2) Selecting number of performance validity measures to administer in an assessment; and 3) Hypothetical clinical decision-making models for use of performance validity testing in a neuropsychological assessment collectively considering behavior, patient reporting, and data indicating invalid or noncredible performance. Performance validity testing helps inform the clinician about an individual's general approach to tasks: response to failure, task engagement and persistence, compliance with task demands. Data-driven clinical suggestions provide a resource to clinicians and to instigate conversation within the field to make more uniform, testable decisions to further the discussion, and guide future research in this area.

  12. Face and Content Validity of the MacArthur Competence Assessment Tool for the Treatment of Iranian Patients.

    PubMed

    Saber, Ali; Tabatabaei, Seyed Mahmoud; Akasheh, Godarz; Sehat, Mojtaba; Zanjani, Zahra; Larijani, Bagher

    2017-01-01

    There is not a valid Persian tool for measuring the decision-making competency of patients. The aim of this study is to evaluate the face and content validity of the MacArthur Competence Assessment Tool for the treatment of Iranian Persian-speaking patients. To assess the validity of the Persian version of the tool, a self-administrated questionnaire was designed. The Lawshe method was also used for assessing each item. Content validity ratio (CVR) and content validity index (CVI) were used to assess the content validity quantitatively. According to the experts' judgment, questions with a CVR ≥0.62 and CVR <0.62 were maintainable and unmaintainable, respectively. The questions were designed in a manner to achieve the desirable result (CVR ≥0.62). The CVI scale (S-CVI) and CVI (S-CVI/Ave) were 0.94 (higher than 0.79). Thus, the content validity was confirmed. Since capacity assessments are usually based on physician's subjective judgment, they are likely to bias and therefore, with this suitably validated tool, we can improve judgment of physicians and health-care providers in out- and in-patient cases.

  13. Translating and validating a Training Needs Assessment tool into Greek

    PubMed Central

    Markaki, Adelais; Antonakis, Nikos; Hicks, Carolyn M; Lionis, Christos

    2007-01-01

    Background The translation and cultural adaptation of widely accepted, psychometrically tested tools is regarded as an essential component of effective human resource management in the primary care arena. The Training Needs Assessment (TNA) is a widely used, valid instrument, designed to measure professional development needs of health care professionals, especially in primary health care. This study aims to describe the translation, adaptation and validation of the TNA questionnaire into Greek language and discuss possibilities of its use in primary care settings. Methods A modified version of the English self-administered questionnaire consisting of 30 items was used. Internationally recommended methodology, mandating forward translation, backward translation, reconciliation and pretesting steps, was followed. Tool validation included assessing item internal consistency, using the alpha coefficient of Cronbach. Reproducibility (test – retest reliability) was measured by the kappa correlation coefficient. Criterion validity was calculated for selected parts of the questionnaire by correlating respondents' research experience with relevant research item scores. An exploratory factor analysis highlighted how the items group together, using a Varimax (oblique) rotation and subsequent Cronbach's alpha assessment. Results The psychometric properties of the Greek version of the TNA questionnaire for nursing staff employed in primary care were good. Internal consistency of the instrument was very good, Cronbach's alpha was found to be 0.985 (p < 0.001) and Kappa coefficient for reproducibility was found to be 0.928 (p < 0.0001). Significant positive correlations were found between respondents' current performance levels on each of the research items and amount of research involvement, indicating good criterion validity in the areas tested. Factor analysis revealed seven factors with eigenvalues of > 1.0, KMO (Kaiser-Meyer-Olkin) measure of sampling adequacy = 0.680 and

  14. Development and validation of the Vietnamese primary care assessment tool

    PubMed Central

    2018-01-01

    Objective To adapt the consumer version of the Primary Care Assessment Tool (PCAT) for Vietnam and determine its internal consistency and validity. Design A quantitative cross sectional study. Setting 56 communes in 3 representative provinces of central Vietnam. Participants Total of 3289 people who used health care services at health facility at least once over the past two years. Results The Vietnamese adult expanded consumer version of the PCAT (VN PCAT-AE) is an instrument for evaluation of primary care in Vietnam with 70 items comprising six scales representing four core primary care domains, and three additional scales representing three derivative domains. Sixteen other items from the original tool were not included in the final instrument, due to problems with missing values, floor or ceiling effects, and item-total correlations. All the retained scales have a Cronbach’s alpha above 0.70 except for the subscale of Family Centeredness. Conclusions The VN PCAT-AE demonstrates adequate internal consistency and validity to be used as an effective tool for measuring the quality of primary care in Vietnam from the consumer perspective. Additional work in the future to optimize valid measurement in all domains consistent with the original version of the tool may be helpful as the primary care system in Vietnam further develops. PMID:29324851

  15. Development and validation of the Vietnamese primary care assessment tool.

    PubMed

    Hoa, Nguyen Thi; Tam, Nguyen Minh; Peersman, Wim; Derese, Anselme; Markuns, Jeffrey F

    2018-01-01

    To adapt the consumer version of the Primary Care Assessment Tool (PCAT) for Vietnam and determine its internal consistency and validity. A quantitative cross sectional study. 56 communes in 3 representative provinces of central Vietnam. Total of 3289 people who used health care services at health facility at least once over the past two years. The Vietnamese adult expanded consumer version of the PCAT (VN PCAT-AE) is an instrument for evaluation of primary care in Vietnam with 70 items comprising six scales representing four core primary care domains, and three additional scales representing three derivative domains. Sixteen other items from the original tool were not included in the final instrument, due to problems with missing values, floor or ceiling effects, and item-total correlations. All the retained scales have a Cronbach's alpha above 0.70 except for the subscale of Family Centeredness. The VN PCAT-AE demonstrates adequate internal consistency and validity to be used as an effective tool for measuring the quality of primary care in Vietnam from the consumer perspective. Additional work in the future to optimize valid measurement in all domains consistent with the original version of the tool may be helpful as the primary care system in Vietnam further develops.

  16. The development and validity of the Salford Gait Tool: an observation-based clinical gait assessment tool.

    PubMed

    Toro, Brigitte; Nester, Christopher J; Farren, Pauline C

    2007-03-01

    To develop the construct, content, and criterion validity of the Salford Gait Tool (SF-GT) and to evaluate agreement between gait observations using the SF-GT and kinematic gait data. Tool development and comparative evaluation. University in the United Kingdom. For designing construct and content validity, convenience samples of 10 children with hemiplegic, diplegic, and quadriplegic cerebral palsy (CP) and 152 physical therapy students and 4 physical therapists were recruited. For developing criterion validity, kinematic gait data of 13 gait clusters containing 56 children with hemiplegic, diplegic, and quadriplegic CP and 11 neurologically intact children was used. For clinical evaluation, a convenience sample of 23 pediatric physical therapists participated. We developed a sagittal plane observational gait assessment tool through a series of design, test, and redesign iterations. The tool's grading system was calibrated using kinematic gait data of 13 gait clusters and was evaluated by comparing the agreement of gait observations using the SF-GT with kinematic gait data. Criterion standard kinematic gait data. There was 58% mean agreement based on grading categories and 80% mean agreement based on degree estimations evaluated with the least significant difference method. The new SF-GT has good concurrent criterion validity.

  17. Validation to Spanish of the Caring Assessment Tool (CAT-V)

    PubMed Central

    Ayuso, Rosa María Fernández; Velázquez, Juan Manuel Morillo; Ayuso, David Fernández; de la Torre-Montero, Julio César

    2017-01-01

    Resume Objective: to translate and validate to Spanish the Caring Assessment Scale tool, CAT-V, by Joanne Duffy, within the framework of Jean Watson; as a secondary objective, it is proposed to evaluate its psychometric properties. There are tools designed to measure the patient’s perception of provided cares, including CAT-V, the subject of our interest, in a way that it can be used in Spanish-speaking patients. Methods: to meet the objectives, it was performed sequential translation and retro-translation of the scale to be validated, through a standardized procedure. The final version of that scale was validated in a sample of 349 patients from four public and two private hospitals in Madrid, Spain. Results: The instrument was translated and validated with high internal consistency (Cronbach’s alpha .953). The subsequent factor analysis revealed a three-factor structure, not coincident with the data from the US population. Conclusion: it is considered that the translation of CAT-V is a suitable instrument to be used in the evaluation of patient care in Ibero-american health centers whose language is Spanish. PMID:29069268

  18. Validity and reliability of the infant breastfeeding assessment tool, the mother baby assessment tool, and the LATCH scoring system.

    PubMed

    Altuntas, Nilgun; Turkyilmaz, Canan; Yildiz, Havva; Kulali, Ferit; Hirfanoglu, Ibrahim; Onal, Esra; Ergenekon, Ebru; Koç, Esin; Atalay, Yıldız

    2014-05-01

    We aimed to evaluate the validity and reliability of the Infant Breastfeeding Assessment Tool (IBFAT), the Mother Baby Assessment (MBA) Tool, and the LATCH scoring system. Mothers who delivered healthy, full-term infants in the Obstetrics & Gynecology Service of Gazi University, Ankara, Turkey, between December 2013 and January 2014 and their infants were included in the study. Forty-six randomly selected breastfeeding sessions were monitored and scored simultaneously by three researchers (Raters 1, 2, and 3) using LATCH, IBFAT, and the MBA Tool. Researchers put the score sheets in an envelope in order to hide them from each other. The compatibility of the scores given by three researchers was assessed by statistical methods. We found positive and significant correlation coefficients between 0.81 to 0.88 for the total MBA score, between 0.90 to 0.95 for the total IBFAT score, and between 0.85 to 0.91 for the total LATCH score. Correlation coefficients testing these three tools ranged from 0.71 to 0.88, with the minimum value being noted for the correlation between LATCH and IBFAT scores and the maximum value being noted for the correlation between LATCH and MBA scores. We found positive and significant correlations between researchers' scores for 46 observations using the three assessment tools. This study showed that these above-mentioned tools were compatible for the assessment of the efficiency of breastfeeding.

  19. Development and validation of a risk assessment tool for gastric cancer in a general Japanese population.

    PubMed

    Iida, Masahiro; Ikeda, Fumie; Hata, Jun; Hirakawa, Yoichiro; Ohara, Tomoyuki; Mukai, Naoko; Yoshida, Daigo; Yonemoto, Koji; Esaki, Motohiro; Kitazono, Takanari; Kiyohara, Yutaka; Ninomiya, Toshiharu

    2018-05-01

    There have been very few reports of risk score models for the development of gastric cancer. The aim of this study was to develop and validate a risk assessment tool for discerning future gastric cancer risk in Japanese. A total of 2444 subjects aged 40 years or over were followed up for 14 years from 1988 (derivation cohort), and 3204 subjects of the same age group were followed up for 5 years from 2002 (validation cohort). The weighting (risk score) of each risk factor for predicting future gastric cancer in the risk assessment tool was determined based on the coefficients of a Cox proportional hazards model in the derivation cohort. The goodness of fit of the established risk assessment tool was assessed using the c-statistic and the Hosmer-Lemeshow test in the validation cohort. During the follow-up, gastric cancer developed in 90 subjects in the derivation cohort and 35 subjects in the validation cohort. In the derivation cohort, the risk prediction model for gastric cancer was established using significant risk factors: age, sex, the combination of Helicobacter pylori antibody and pepsinogen status, hemoglobin A1c level, and smoking status. The incidence of gastric cancer increased significantly as the sum of risk scores increased (P trend < 0.001). The risk assessment tool was validated internally and showed good discrimination (c-statistic = 0.76) and calibration (Hosmer-Lemeshow test P = 0.43) in the validation cohort. We developed a risk assessment tool for gastric cancer that provides a useful guide for stratifying an individual's risk of future gastric cancer.

  20. Validation of the Virtual MET as an assessment tool for executive functions.

    PubMed

    Rand, Debbie; Basha-Abu Rukan, Soraya; Weiss, Patrice L Tamar; Katz, Noomi

    2009-08-01

    The purpose of this study was to establish ecological validity and initial construct validity of a Virtual Multiple Errands Test (VMET) as an assessment tool for executive functions. It was implemented within the Virtual Mall (VMall), a novel functional video-capture virtual shopping environment. The main objectives were (1) to examine the relationships between the performance of three groups of participants in the Multiple Errands Test (MET) carried out in a real shopping mall and their performance in the VMET, (2) to assess the relationships between the MET and VMET of the post-stroke participant's level of executive functioning and independence in instrumental activities of daily living, and (3) to compare the performance of post-stroke participants to those of healthy young and older controls in both the MET and VMET. The study population included three groups; post-stroke participants (n = 9), healthy young participants (n = 20), and healthy older participants (n = 20). The VMET was able to differentiate between two age groups of healthy participants and between healthy and post-stroke participants thus demonstrating that it is sensitive to brain injury and ageing and supports construct validity between known groups. In addition, significant correlations were found between the MET and the VMET for both the post-stroke participants and older healthy participants. This provides initial support for the ecological validity of the VMET as an assessment tool of executive functions. However, further psychometric data on temporal stability are needed, namely test-retest reliability and responsiveness, before it is ready for clinical application. Further research using the VMET as an assessment tool within the VMall with larger groups and in additional populations is also recommended.

  1. Measuring assessment standards in undergraduate medical programs: Development and validation of AIM tool.

    PubMed

    Sajjad, Madiha; Khan, Rehan Ahmed; Yasmeen, Rahila

    2018-01-01

    To develop a tool to evaluate faculty perceptions of assessment quality in an undergraduate medical program. The Assessment Implementation Measure (AIM) tool was developed by a mixed method approach. A preliminary questionnaire developed through literature review was submitted to a panel of 10 medical education experts for a three-round 'Modified Delphi technique'. Panel agreement of > 75% was considered the criterion for inclusion of items in the questionnaire. Cognitive pre-testing of five faculty members was conducted. Pilot study was done with 30 randomly selected faculty members. Content validity index (CVI) was calculated for individual items (I-CVI) and composite scale (S-CVI). Cronbach's alpha was calculated to determine the internal consistency reliability of the tool. The final AIM tool had 30 items after the Delphi process. S-CVI was 0.98 with the S-CVI/Avg method and 0.86 by S-CVI/UA method, suggesting good content validity. Cut-off value of < 0.9 I-CVI was taken as criterion for item deletion. Cognitive pre-testing revealed good item interpretation. Cronbach's alpha calculated for the AIM was 0.9, whereas Cronbach's alpha for the four domains ranged from 0.67 to 0.80. 'AIM' is a relevant and useful instrument with good content validity and reliability of results, and may be used to evaluate the teachers´ perceptions about assessment quality.

  2. Development and validation of a cancer-specific swallowing assessment tool: MASA-C.

    PubMed

    Carnaby, Giselle D; Crary, Michael A

    2014-03-01

    We present data from a sample of patients receiving radiotherapy for head/neck cancer to define and measure the validity of a new clinical assessment measure for swallowing. Fifty-eight patients undergoing radiotherapy (±chemotherapy) for head/neck cancer (HNC) supported the development of a physiology-based assessment tool of swallowing (Mann Assessment of Swallowing Ability--Cancer: MASA-C) administered at two time points (baseline and following radiotherapy treatment). The new exam was evaluated for internal consistency of items using Cronbach's alpha. Reliability of measurement was evaluated with intraclass correlation (ICC) and the Kappa statistic between two independent raters. Concurrent validity was established through comparison with the original MASA examination and against the referent standard videofluoroscopic swallowing examination (VFE). Sensitivity, specificity, and likelihood ratios along with 95 % confidence intervals (CIs) were derived for comparison of the two evaluation forms (MASA vs. MASA-C). Accuracy of diagnostic precision was displayed using receiver operator characteristic curves. The new MASA-C tool demonstrated superior validity to the original MASA examination applied to a HNC population. In comparison to the VFE referent exam, the MASA-C revealed strong sensitivity and specificity (Se 83, Sp 96), predictive values (positive predictive value (PPV) 0.95, negative predictive value (NPV) 0.86), and likelihood ratios (21.6). In addition, it demonstrated good reliability (ICC = 0.96) between speech-language pathology raters. The MASA-C is a reliable and valid scale that is sensitive to differences in swallowing performance in HNC patients with and without dysphagia. Future longitudinal evaluation of this tool in larger samples is suggested. The development and refinement of this swallowing assessment tool for use in multidisciplinary HNC teams will facilitate earlier identification of patients with swallowing difficulties and enable

  3. Psychometric validation of the Chinese version of the Johns Hopkins Fall Risk Assessment Tool for older Chinese inpatients.

    PubMed

    Zhang, Junhong; Wang, Min; Liu, Yu

    2016-10-01

    To culturally adapt and evaluate the reliability and validity of the Chinese version of the Johns Hopkins Fall Risk Assessment Tool among older inpatients in the mainland of China. Patient falls are an important safety consideration within hospitals among older inpatients. Nurses need specific risk assessment tools for older inpatients to reliably identify at-risk populations and guide interventions that highlight fixable risk factors for falls and consequent injuries. In China, a few tools have been developed to measure fall risk. However, they lack the solid psychometric development necessary to establish their validity and reliability, and they are not widely used for elderly inpatients. A cross-sectional study. A convenient sampling was used to recruit 201 older inpatients from two tertiary-level hospitals in Beijing and Xiamen, China. The Johns Hopkins Fall Risk Assessment Tool was translated using forward and backward translation procedures and was administered to these 201 older inpatients. Reliability of the tool was calculated by inter-rater reliability and Cronbach's alpha. Validity was analysed through content validity index and construct validity. The Inter-rater reliability of Chinese version of Johns Hopkins Fall Risk Assessment Tool was 97·14% agreement with Cohen's Kappa of 0·903. Cronbach's α was 0·703. Content of Validity Index was 0·833. Two factors represented intrinsic and extrinsic risk factors were explored that together explained 58·89% of the variance. This study provided evidence that Johns Hopkins Fall Risk Assessment Tool is an acceptable, valid and reliable tool to identify older inpatients at risk of falls and falls with injury. Further psychometric testing on criterion validity and evaluation of its advanced utility in geriatric clinical settings are warranted. The Chinese version of Johns Hopkins Fall Risk Assessment Tool may be useful for health care personnel to identify older Chinese inpatients at risk of falls and falls

  4. Development and testing of a DVT risk assessment tool: providing evidence of validity and reliability.

    PubMed

    McCaffrey, Ruth; Bishop, Mary; Adonis-Rizzo, Marie; Williamson, Ellen; McPherson, Melanie; Cruikshank, Alice; Carrier, Vicki Jo; Sands, Simone; Pigano, Diane; Girard, Patricia; Lauzon, Cathy

    2007-01-01

    Hospital-acquired deep vein thrombosis (DVT) and pulmonary embolisms (PE) are preventable problems that can increase mortality. Early assessment and recognition of risk as well as initiating appropriate prevention measures can prevent DVT or PE. The purpose of this research project was to develop a DVT risk assessment tool and test the tool for validity and reliability. Three phases were undertaken in developing and testing the JFK Medical Center DVT risk assessment tool. Investigation and clarification of risk and predisposing factors for DVT were identified from the literature, expert nursing knowledge, and medical staff input. Second, item development and weighting were undertaken. Third, parametric testing for content validity measured the differences in mean assessment tool scores between a group of patients who developed DVT in the hospital and a demographically similar group who did not develop DVT. Interrater reliability was measured by having three different nurses score each patient and compare the differences in scores among the three. The DVT group had significantly higher scores on the JFK DVT assessment scale than did those who did not experience DVT. Interrater reliability showed a strong correlation among the scores of the three nurses (.98). Providing a valid and reliable tool for measuring the risk for DVT or PE in hospitalized patients will enable nurses to intervene early in patients at risk. Basing DVT risk assessment on the evidence provided in this study will assist nurses in becoming more confident in recognizing the necessity for interventions in hospitalized patients and decreasing risk. Nurses can now evaluate patients at risk for DVT or PE using the JFK Medial Center's risk assessment tool.

  5. Furthering the Validity of a Tool to Assess Simulated Pregnancy Options Counseling Skills.

    PubMed

    Lupi, Carla; Ward-Peterson, Melissa; Coxe, Stefany; Minor, Suzanne; Eliacin, Irmanie; Obeso, Vivian

    2016-10-01

    To further the validity of a tool to assess nondirective pregnancy options counseling skills. Using a cross-sectional design, we explored four sources of construct validity evidence for an objective structured clinical examination for training and assessment of nondirective pregnancy options counseling: content, response process, internal structure, and relations to other variables. Content of the previously developed tool was enhanced through input from five family medicine educators. The objective structured clinical examination was implemented in a family medicine clerkship with third-year medical students from 2014 to 2015 using trained raters. Response process was addressed after a pilot round. Three new raters evaluated videotapes of 46 performances. Cronbach's alpha, intraclass correlation coefficients, and Spearman's rho were estimated with 95% confidence intervals. The content validity was affirmed. Cronbach's alpha was 0.71. According to Landis and Koch's criteria, all but two items unique to the clinical situation of pregnancy options counseling generated substantial to perfect agreement (0.62-1.00). Relations to other variables within the checklist were strong, ranging from 0.66 to 0.87. This tool for assessing pregnancy options counseling skills has excellent content and strong internal structure. Further work to improve the Global Rating Scale may be necessary for summative use.

  6. Development and Validation of the Narrative Quality Assessment Tool.

    PubMed

    Kim, Wonsun Sunny; Shin, Cha-Nam; Kathryn Larkey, Linda; Roe, Denise J

    2017-04-01

    The use of storytelling in health promotion has grown over the past 2 decades, showing promise for moving people to initiate healthy behavior change. Given the increasingly prevalent role of storytelling in health promotion research and the need to more clearly identify what storytelling elements and mediators may better predict behavior change, there is a need to develop measures to specifically assess these factors in a cultural community context. The purpose of this study is to develop and preliminarily validate a narrative quality assessment tool for measuring elements of storytelling that are predicted to affect attitude and behavior change (i.e., narrative characteristics, identification, and transportation) within a cultural community setting using a culture-centric model. Reliability and validity of these scales were assessed with repeated administrations among 74 Latino men and women with a mean age of 39.6 years (SD = 11.47 years). The confirmatory factor analysis in addition to internal consistency tests revealed preliminary evidence for reliability and validity of the narrative characteristics, identification, and transportation scales. Cronbach's alpha ranged from .92 to .94. Items revealed adequate factor loadings (.85-.98) and good model fit. The new scales provide the first step in moving the assessment of narrative quality into a culturally relevant context for evaluation of story use in health promotion. The results present valuable information for nurse researchers to guide the development and testing of culturally grounded storytelling interventions' potential to predict attitude and behavior change for patients.

  7. Validation of the self-assessment teamwork tool (SATT) in a cohort of nursing and medical students.

    PubMed

    Roper, Lucinda; Shulruf, Boaz; Jorm, Christine; Currie, Jane; Gordon, Christopher J

    2018-02-09

    Poor teamwork has been implicated in medical error and teamwork training has been shown to improve patient care. Simulation is an effective educational method for teamwork training. Post-simulation reflection aims to promote learning and we have previously developed a self-assessment teamwork tool (SATT) for health students to measure teamwork performance. This study aimed to evaluate the psychometric properties of a revised self-assessment teamwork tool. The tool was tested in 257 medical and nursing students after their participation in one of several mass casualty simulations. Using exploratory and confirmatory factor analysis, the revised self-assessment teamwork tool was shown to have strong construct validity, high reliability, and the construct demonstrated invariance across groups (Medicine & Nursing). The modified SATT was shown to be a reliable and valid student self-assessment tool. The SATT is a quick and practical method of guiding students' reflection on important teamwork skills.

  8. A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

    PubMed

    Rana, Gurpreet K; Bradley, Doreen R; Hamstra, Stanley J; Ross, Paula T; Schumacher, Robert E; Frohna, John G; Haftel, Hilary M; Lypson, Monica L

    2011-01-01

    The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center. Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies. Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts. The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

  9. An investigation of the validity of the Work Assessment Triage Tool clinical decision support tool for selecting optimal rehabilitation interventions for workers with musculoskeletal injuries.

    PubMed

    Qin, Ziling; Armijo-Olivo, Susan; Woodhouse, Linda J; Gross, Douglas P

    2016-03-01

    To evaluate the concurrent validity of a clinical decision support tool (Work Assessment Triage Tool (WATT)) developed to select rehabilitation treatments for injured workers with musculoskeletal conditions. Methodological study with cross-sectional and prospective components. Data were obtained from the Workers' Compensation Board of Alberta rehabilitation facility in Edmonton, Canada. A total of 432 workers' compensation claimants evaluated between November 2011 and June 2012. Percentage agreement between the Work Assessment Triage Tool and clinician recommendations was used to determine concurrent validity. In claimants returning to work, frequencies of matching were calculated and compared between clinician and Work Assessment Triage Tool recommendations and the actual programs undertaken by claimants. The frequency of each intervention recommended by clinicians, Work Assessment Triage Tool, and case managers were also calculated and compared. Percentage agreement between clinician and Work Assessment Triage Tool recommendations was poor (19%) to moderate (46%) and Kappa = 0.37 (95% CI -0.02, 0.76). The Work Assessment Triage Tool did not improve upon clinician recommendations as only 14 out of 31 claimants returning to work had programs that contradicted clinician recommendations, but were consistent with Work Assessment Triage Tool recommendations. Clinicians and case managers were inclined to recommend functional restoration, physical therapy, or no rehabilitation while the Work Assessment Triage Tool recommended additional evidence-based interventions, such as workplace-based interventions. Our findings do not provide evidence of concurrent validity for the Work Assessment Triage Tool compared with clinician recommendations. Based on these results, we cannot recommend further implementation of the Work Assessment Triage Tool. However, the Work Assessment Triage Tool appeared more likely than clinicians to recommend interventions supported by evidence

  10. Development, Validation, and Verification of a Self-Assessment Tool to Estimate Agnibala (Digestive Strength).

    PubMed

    Singh, Aparna; Singh, Girish; Patwardhan, Kishor; Gehlot, Sangeeta

    2017-01-01

    According to Ayurveda, the traditional system of healthcare of Indian origin, Agni is the factor responsible for digestion and metabolism. Four functional states (Agnibala) of Agni have been recognized: regular, irregular, intense, and weak. The objective of the present study was to develop and validate a self-assessment tool to estimate Agnibala The developed tool was evaluated for its reliability and validity by administering it to 300 healthy volunteers of either gender belonging to 18 to 40-year age group. Besides confirming the statistical validity and reliability, the practical utility of the newly developed tool was also evaluated by recording serum lipid parameters of all the volunteers. The results show that the lipid parameters vary significantly according to the status of Agni The tool, therefore, may be used to screen normal population to look for possible susceptibility to certain health conditions. © The Author(s) 2016.

  11. Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts.

    PubMed

    Löfmark, Anna; Mårtensson, Gunilla

    2017-03-01

    The aim of the present study was to establish the validity of the tool Assessment of Clinical Education (AssCE). The tool is widely used in Sweden and some Nordic countries for assessing nursing students' performance in clinical education. It is important that the tools in use be subjected to regular audit and critical reviews. The validation process, performed in two stages, was concluded with a high level of congruence. In the first stage, Delphi technique was used to elaborate the AssCE tool using a group of 35 clinical nurse lecturers. After three rounds, we reached consensus. In the second stage, a group of 46 clinical nurse lecturers representing 12 universities in Sweden and Norway audited the revised version of the AssCE in relation to learning outcomes from the last clinical course at their respective institutions. Validation of the revised AssCE was established with high congruence between the factors in the AssCE and examined learning outcomes. The revised AssCE tool seems to meet its objective to be a validated assessment tool for use in clinical nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

  12. Validation of screening tools to assess appetite among geriatric patients.

    PubMed

    Hanisah, R; Suzana, S; Lee, F S

    2012-07-01

    Poor appetite is one of the main contributing factors of poor nutritional status among elderly individuals. Recognizing the importance of assessment of appetite, a cross sectional study was conducted to determine the validity of appetite screening tools namely, the Council on Nutrition Appetite questionnaire (CNAQ) and the simplified nutritional appetite questionnaire (SNAQ) against the appetite, hunger and sensory perception questionnaire (AHSPQ), measures of nutritional status and food intake among geriatric patients at the main general hospital in Malaysia. Nutritional status was assessed using the subjective global assessment (SGA) while food intake was measured using the dietary history questionnaire (DHQ). Anthropometric parameters included weight, height, body mass index (BMI), calf circumference (CC) and mid upper arm circumference (MUAC). A total of 145 subjects aged 60 to 86 years (68.3 ± 5.8 years) with 31.7% men and 68.3% women were recruited from outpatients (35 subjects) and inpatients (110 subjects) of Kuala Lumpur Hospital of Malaysia. As assessed by SGA, most subjects were classified as mild to moderately malnourished (50.4%), followed by normal (38.6%) and severely malnourished (11.0%). A total of 79.3% and 57.2% subjects were classified as having poor appetite according to CNAQ and SNAQ, respectively. CNAQ (80.9%) had a higher sensitivity than SNAQ (69.7%) when validated against nutritional status as assessed using SGA. However, the specificity of SNAQ (62.5%) was higher than CNAQ (23.2%). Positive predictive value for CNAQ and SNAQ were 62.6% and 74.7%, respectively. Cronbach's alpha for CNAQ and SNAQ were 0.546 and 0.578, respectively. History of weight loss over the past one year (Adjusted odds ratio 2.49) (p < 0.01) and thiamine intake less than the recommended nutrient intake (RNI) (Adjusted odds ratio 3.04) (p < 0.05) were risk factors for poor appetite among subjects. In conclusion, malnutrition and poor appetite were prevalent among the

  13. A Turkish Version of the Critical-Care Pain Observation Tool: Reliability and Validity Assessment.

    PubMed

    Aktaş, Yeşim Yaman; Karabulut, Neziha

    2017-08-01

    The study aim was to evaluate the validity and reliability of the Critical-Care Pain Observation Tool in critically ill patients. A repeated measures design was used for the study. A convenience sample of 66 patients who had undergone open-heart surgery in the cardiovascular surgery intensive care unit in Ordu, Turkey, was recruited for the study. The patients were evaluated by using the Critical-Care Pain Observation Tool at rest, during a nociceptive procedure (suctioning), and 20 minutes after the procedure while they were conscious and intubated after surgery. The Turkish version of the Critical-Care Pain Observation Tool has shown statistically acceptable levels of validity and reliability. Inter-rater reliability was supported by moderate-to-high-weighted κ coefficients (weighted κ coefficient = 0.55 to 1.00). For concurrent validity, significant associations were found between the scores on the Critical-Care Pain Observation Tool and the Behavioral Pain Scale scores. Discriminant validity was also supported by higher scores during suctioning (a nociceptive procedure) versus non-nociceptive procedures. The internal consistency of the Critical-Care Pain Observation Tool was 0.72 during a nociceptive procedure and 0.71 during a non-nociceptive procedure. The validity and reliability of the Turkish version of the Critical-Care Pain Observation Tool was determined to be acceptable for pain assessment in critical care, especially for patients who cannot communicate verbally. Copyright © 2016 American Society of PeriAnesthesia Nurses. Published by Elsevier Inc. All rights reserved.

  14. Cross-cultural validation of Cancer Communication Assessment Tool in Korea.

    PubMed

    Shin, Dong Wook; Shin, Jooyeon; Kim, So Young; Park, Boram; Yang, Hyung-Kook; Cho, Juhee; Lee, Eun Sook; Kim, Jong Heun; Park, Jong-Hyock

    2015-02-01

    Communication between cancer patients and caregivers is often suboptimal. The Cancer Communication Assessment Tool for Patient and Families (CCAT-PF) is a unique tool developed to measure congruence in patient-family caregiver communication employing a dyadic approach. We aimed to examine the cross-cultural applicability of the CCAT in the Korean healthcare setting. Linguistic validation of the CCAT-PF was performed through a standard forward-backward translation process. Psychometric validation was performed with 990 patient-caregiver dyads recruited from 10 cancer centers. Mean scores of CCAT-P and CCAT-F were similar at 44.8 for both scales. Mean CCAT-PF score was 23.7 (8.66). Concordance of each items between patients and caregivers was low (weighted kappa values <0.20 for all items and Spearman's rho <0.18 for scale scores). Scale scores did not differ significantly across a variety of cancer types and stages. The CCAT-P or CCAT-F score was weakly associated with mental health and quality of life outcomes. The CCAT-PF was correlated weakly with both patient-perceived and caregiver-perceived family avoidance of cancer care scales. The CCAT-PF Korean version showed similar psychometric properties to the original English version in the assessment of communication congruence between cancer patient and family caregivers. Copyright © 2014 John Wiley & Sons, Ltd.

  15. Validity and applicability of a video-based animated tool to assess mobility in elderly Latin American populations

    PubMed Central

    Guerra, Ricardo Oliveira; Oliveira, Bruna Silva; Alvarado, Beatriz Eugenia; Curcio, Carmen Lucia; Rejeski, W Jack; Marsh, Anthony P; Ip, Edward H; Barnard, Ryan T; Guralnik, Jack M; Zunzunegui, Maria Victoria

    2016-01-01

    Aim To assess the reliability and the validity of Portuguese- and Spanish-translated versions of the video-based short-form Mobility Assessment Tool in assessing self-reported mobility, and to provide evidence for the applicability of these videos in elderly Latin American populations as a complement to physical performance measures. Methods The sample consisted of 300 elderly participants (150 from Brazil, 150 from Colombia) recruited at neighborhood social centers. Mobility was assessed with the Mobility Assessment Tool, and compared with the Short Physical Performance Battery score and self-reported functional limitations. Reliability was calculated using intraclass correlation coefficients. Multiple linear regression analyses were used to assess associations among mobility assessment tools and health, and sociodemographic variables. Results A significant gradient of increasing Mobility Assessment Tool score with better physical function was observed for both self-reported and objective measures, and in each city. Associations between self-reported mobility and health were strong, and significant. Mobility Assessment Tool scores were lower in women at both sites. Intraclass correlation coefficients of the Mobility Assessment Tool were 0.94 (95% confidence interval 0.90–0.97) in Brazil and 0.81 (95% confidence interval 0.66–0.91) in Colombia. Mobility Assessment Tool scores were lower in Manizales than in Natal after adjustment by Short Physical Performance Battery, self-rated health and sex. Conclusions These results provide evidence for high reliability and good validity of the Mobility Assessment Tool in its Spanish and Portuguese versions used in Latin American populations. In addition, the Mobility Assessment Tool can detect mobility differences related to environmental features that cannot be captured by objective perfor mance measures. PMID:24666718

  16. Validity and applicability of a video-based animated tool to assess mobility in elderly Latin American populations.

    PubMed

    Guerra, Ricardo Oliveira; Oliveira, Bruna Silva; Alvarado, Beatriz Eugenia; Curcio, Carmen Lucia; Rejeski, W Jack; Marsh, Anthony P; Ip, Edward H; Barnard, Ryan T; Guralnik, Jack M; Zunzunegui, Maria Victoria

    2014-10-01

    To assess the reliability and the validity of Portuguese- and Spanish-translated versions of the video-based short-form Mobility Assessment Tool in assessing self-reported mobility, and to provide evidence for the applicability of these videos in elderly Latin American populations as a complement to physical performance measures. The sample consisted of 300 elderly participants (150 from Brazil, 150 from Colombia) recruited at neighborhood social centers. Mobility was assessed with the Mobility Assessment Tool, and compared with the Short Physical Performance Battery score and self-reported functional limitations. Reliability was calculated using intraclass correlation coefficients. Multiple linear regression analyses were used to assess associations among mobility assessment tools and health, and sociodemographic variables. A significant gradient of increasing Mobility Assessment Tool score with better physical function was observed for both self-reported and objective measures, and in each city. Associations between self-reported mobility and health were strong, and significant. Mobility Assessment Tool scores were lower in women at both sites. Intraclass correlation coefficients of the Mobility Assessment Tool were 0.94 (95% confidence interval 0.90-0.97) in Brazil and 0.81 (95% confidence interval 0.66-0.91) in Colombia. Mobility Assessment Tool scores were lower in Manizales than in Natal after adjustment by Short Physical Performance Battery, self-rated health and sex. These results provide evidence for high reliability and good validity of the Mobility Assessment Tool in its Spanish and Portuguese versions used in Latin American populations. In addition, the Mobility Assessment Tool can detect mobility differences related to environmental features that cannot be captured by objective performance measures. © 2013 Japan Geriatrics Society.

  17. The Johns Hopkins Fall Risk Assessment Tool: A Study of Reliability and Validity.

    PubMed

    Poe, Stephanie S; Dawson, Patricia B; Cvach, Maria; Burnett, Margaret; Kumble, Sowmya; Lewis, Maureen; Thompson, Carol B; Hill, Elizabeth E

    Patient falls and fall-related injury remain a safety concern. The Johns Hopkins Fall Risk Assessment Tool (JHFRAT) was developed to facilitate early detection of risk for anticipated physiologic falls in adult inpatients. Psychometric properties in acute care settings have not yet been fully established; this study sought to fill that gap. Results indicate that the JHFRAT is reliable, with high sensitivity and negative predictive validity. Specificity and positive predictive validity were lower than expected.

  18. Assessing teamwork performance in obstetrics: A systematic search and review of validated tools.

    PubMed

    Fransen, Annemarie F; de Boer, Liza; Kienhorst, Dieneke; Truijens, Sophie E; van Runnard Heimel, Pieter J; Oei, S Guid

    2017-09-01

    Teamwork performance is an essential component for the clinical efficiency of multi-professional teams in obstetric care. As patient safety is related to teamwork performance, it has become an important learning goal in simulation-based education. In order to improve teamwork performance, reliable assessment tools are required. These can be used to provide feedback during training courses, or to compare learning effects between different types of training courses. The aim of the current study is to (1) identify the available assessment tools to evaluate obstetric teamwork performance in a simulated environment, and (2) evaluate their psychometric properties in order to identify the most valuable tool(s) to use. We performed a systematic search in PubMed, MEDLINE, and EMBASE to identify articles describing assessment tools for the evaluation of obstetric teamwork performance in a simulated environment. In order to evaluate the quality of the identified assessment tools the standards and grading rules have been applied as recommended by the Accreditation Council for Graduate Medical Education (ACGME) Committee on Educational Outcomes. The included studies were also assessed according to the Oxford Centre for Evidence Based Medicine (OCEBM) levels of evidence. This search resulted in the inclusion of five articles describing the following six tools: Clinical Teamwork Scale, Human Factors Rating Scale, Global Rating Scale, Assessment of Obstetric Team Performance, Global Assessment of Obstetric Team Performance, and the Teamwork Measurement Tool. Based on the ACGME guidelines we assigned a Class 3, level C of evidence, to all tools. Regarding the OCEBM levels of evidence, a level 3b was assigned to two studies and a level 4 to four studies. The Clinical Teamwork Scale demonstrated the most comprehensive validation, and the Teamwork Measurement Tool demonstrated promising results, however it is recommended to further investigate its reliability. Copyright © 2017

  19. Development and psychometric validation of a scale to assess information needs in cardiac rehabilitation: the INCR Tool.

    PubMed

    Ghisi, Gabriela Lima de Melo; Grace, Sherry L; Thomas, Scott; Evans, Michael F; Oh, Paul

    2013-06-01

    To develop and psychometrically validate a tool to assess information needs in cardiac rehabilitation (CR) patients. After a literature search, 60 information items divided into 11 areas of needs were identified. To establish content validity, they were reviewed by an expert panel (N=10). Refined items were pilot-tested in 34 patients on a 5-point Likert-scale from 1 "really not helpful" to 5 "very important". A final version was generated and psychometrically tested in 203 CR patients. Test-retest reliability was assessed via the intraclass correlation coefficient (ICC), the internal consistency using Cronbach's alpha, and criterion validity was assessed with regard to patient's education and duration in CR. Five items were excluded after ICC analysis as well as one area of needs. All 10 areas were considered internally consistent (Cronbach's alpha>0.7). Criterion validity was supported by significant differences in mean scores by educational level (p<0.05) and duration in CR (p<0.001). The mean total score was 4.08 ± 0.53. Patients rated safety as their greatest information need. The INCR Tool was demonstrated to have good reliability and validity. This is an appropriate tool for application in clinical and research settings, assessing patients' needs during CR and as part of education programming. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

  20. The Development and Validation of a Rapid Assessment Tool of Primary Care in China

    PubMed Central

    Mei, Jie; Liang, Yuan; Shi, LeiYu; Zhao, JingGe; Wang, YuTan; Kuang, Li

    2016-01-01

    Introduction. With Chinese health care reform increasingly emphasizing the importance of primary care, the need for a tool to evaluate primary care performance and service delivery is clear. This study presents a methodology for a rapid assessment of primary care organizations and service delivery in China. Methods. The study translated and adapted the Primary Care Assessment Tool-Adult Edition (PCAT-AE) into a Chinese version to measure core dimensions of primary care, namely, first contact, continuity, comprehensiveness, and coordination. A cross-sectional survey was conducted to assess the validity and reliability of the Chinese Rapid Primary Care Assessment Tool (CR-PCAT). Eight community health centers in Guangdong province have been selected to participate in the survey. Results. A total of 1465 effective samples were included for data analysis. Eight items were eliminated following principal component analysis and reliability testing. The principal component analysis extracted five multiple-item scales (first contact utilization, first contact accessibility, ongoing care, comprehensiveness, and coordination). The tests of scaling assumptions were basically met. Conclusion. The standard psychometric evaluation indicates that the scales have achieved relatively good reliability and validity. The CR-PCAT provides a rapid and reliable measure of four core dimensions of primary care, which could be applied in various scenarios. PMID:26885509

  1. Cross-cultural validity of standardized motor development screening and assessment tools: a systematic review.

    PubMed

    Mendonça, Bianca; Sargent, Barbara; Fetters, Linda

    2016-12-01

    To investigate whether standardized motor development screening and assessment tools that are used to evaluate motor abilities of children aged 0 to 2 years are valid in cultures other than those in which the normative sample was established. This was a systematic review in which six databases were searched. Studies were selected based on inclusion/exclusion criteria and appraised for evidence level and quality. Study variables were extracted. Twenty-three studies representing six motor development screening and assessment tools in 16 cultural contexts met the inclusion criteria: Alberta Infant Motor Scale (n=7), Ages and Stages Questionnaire, 3rd edition (n=2), Bayley Scales of Infant and Toddler Development, 3rd edition (n=8), Denver Developmental Screening Test, 2nd edition (n=4), Harris Infant Neuromotor Test (n=1), and Peabody Developmental Motor Scales, 2nd edition (n=1). Thirteen studies found significant differences between the cultural context and normative sample. Two studies established reliability and/or validity of standardized motor development assessments in high-risk infants from different cultural contexts. Five studies established new population norms. Eight studies described the cross-cultural adaptation of a standardized motor development assessment. Standardized motor development assessments have limited validity in cultures other than that in which the normative sample was established. Their use can result in under- or over-referral for services. © 2016 Mac Keith Press.

  2. Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

    PubMed

    Breimer, Gerben E; Haji, Faizal A; Cinalli, Giuseppe; Hoving, Eelco W; Drake, James M

    2017-02-01

    Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training. The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity). Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise ( P = .04 and .002, respectively). This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training. Copyright © 2016 by the Congress of Neurological Surgeons

  3. Development and validation of the Learning Disabilities Needs Assessment Tool (LDNAT), a HoNOS-based needs assessment tool for use with people with intellectual disability.

    PubMed

    Painter, J; Trevithick, L; Hastings, R P; Ingham, B; Roy, A

    2016-12-01

    In meeting the needs of individuals with intellectual disabilities (ID) who access health services, a brief, holistic assessment of need is useful. This study outlines the development and testing of the Learning Disabilities Needs Assessment Tool (LDNAT), a tool intended for this purpose. An existing mental health (MH) tool was extended by a multidisciplinary group of ID practitioners. Additional scales were drafted to capture needs across six ID treatment domains that the group identified. LDNAT ratings were analysed for the following: item redundancy, relevance, construct validity and internal consistency (n = 1692); test-retest reliability (n = 27); and concurrent validity (n = 160). All LDNAT scales were deemed clinically relevant with little redundancy apparent. Principal component analysis indicated three components (developmental needs, challenging behaviour, MH and well-being). Internal consistency was good (Cronbach alpha 0.80). Individual item test-retest reliability was substantial-near perfect for 20 scales and slight-fair for three scales. Overall reliability was near perfect (intra-class correlation = 0.91). There were significant associations with five of six condition-specific measures, i.e. the Waisman Activities of Daily Living Scale (general ability/disability), Threshold Assessment Grid (risk), Behaviour Problems Inventory for Individuals with Intellectual Disabilities-Short Form (challenging behaviour) Social Communication Questionnaire (autism) and a bespoke physical health questionnaire. Additionally, the statistically significant correlations between these tools and the LDNAT components made sense clinically. There were no statistically significant correlations with the Psychiatric Assessment Schedules for Adults with Developmental Disabilities (a measure of MH symptoms in people with ID). The LDNAT had clinically utility when rating the needs of people with ID prior to condition-specific assessment(s). Analyses of internal

  4. Development and validation of a new assessment tool for suturing skills in medical students.

    PubMed

    Sundhagen, Henriette Pisani; Almeland, Stian Kreken; Hansson, Emma

    2018-01-01

    In recent years, emphasis has been put on that medical student should demonstrate pre-practice/pre-registration core procedural skills to ensure patient safety. Nonetheless, the formal teaching and training of basic suturing skills to medical students have received relatively little attention and there is no standard for what should be tested and how. The aim of this study was to develop and validate, using scientific methods, a tool for assessment of medical students' suturing skills, measuring both micro- and macrosurgical qualities. A tool was constructed and content, construct, concurrent validity, and inter-rater, inter-item, inter-test reliability were tested. Three groups were included: students with no training in suturing skills, students who have had training, plastic surgery. The results show promising reliability and validity when assessing novice medical students' suturing skills. Further studies are needed on implementation of the instrument. Moreover, how the instrument can be used to give formative feedback, evaluate if a required standard is met and for curriculum development needs further investigation.Level of Evidence: Not ratable.

  5. Reliability and Validity of the Korean Cancer Pain Assessment Tool (KCPAT)

    PubMed Central

    Kim, Jeong A; Lee, Juneyoung; Park, Jeanno; Lee, Myung Ah; Yeom, Chang Hwan; Jang, Se Kwon; Yoon, Duck Mi; Kim, Jun Suk

    2005-01-01

    The Korean Cancer Pain Assessment Tool (KCPAT), which was developed in 2003, consists of questions concerning the location of pain, the nature of pain, the present pain intensity, the symptoms associated with the pain, and psychosocial/spiritual pain assessments. This study was carried out to evaluate the reliability and validity of the KCPAT. A stratified, proportional-quota, clustered, systematic sampling procedure was used. The study population (903 cancer patients) was 1% of the target population (90,252 cancer patients). A total of 314 (34.8%) questionnaires were collected. The results showed that the average pain score (5 point on Likert scale) according to the cancer type and the at-present average pain score (VAS, 0-10) were correlated (r=0.56, p<0.0001), and showed moderate agreement (kappa=0.364). The mean satisfaction score was 3.8 (1-5). The average time to complete the questionnaire was 8.9 min. In conclusion, the KCPAT is a reliable and valid instrument for assessing cancer pain in Koreans. PMID:16224166

  6. AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

    PubMed

    Vemer, P; Corro Ramos, I; van Voorn, G A K; Al, M J; Feenstra, T L

    2016-04-01

    A trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool providing model users with a structured view into the validation status of HE decision models, to address this trade-off. A Delphi panel was organized, and was completed by a workshop during an international conference. The proposed tool was constructed iteratively based on comments from, and the discussion amongst, panellists. During the Delphi process, comments were solicited on the importance and feasibility of possible validation techniques for modellers, their relevance for decision makers, and the overall structure and formulation in the tool. The panel consisted of 47 experts in HE modelling and HE decision making from various professional and international backgrounds. In addition, 50 discussants actively engaged in the discussion at the conference workshop and returned 19 questionnaires with additional comments. The final version consists of 13 items covering all relevant aspects of HE decision models: the conceptual model, the input data, the implemented software program, and the model outcomes. Assessment of the Validation Status of Health-Economic decision models (AdViSHE) is a validation-assessment tool in which model developers report in a systematic way both on validation efforts performed and on their outcomes. Subsequently, model users can establish whether confidence in the model is justified or whether additional validation efforts should be undertaken. In this way, AdViSHE enhances transparency of the validation status of HE models and supports efficient model validation.

  7. The Development and Validation of the Contextualized Assessment Tool for Risk and Protection Management (CAT-RPM)

    ERIC Educational Resources Information Center

    Bower, Julie M.; Carroll, Annemaree; Ashman, Adrian

    2014-01-01

    Traditional resources for ascertaining risk and protection for disengaged youth are often unsuitable, due to the stamina and skill required to complete them. Many of these tools assess risk without considering participants' potential for personal growth. The present study outlines the development and initial validation of a tool titled the…

  8. Validation of an early childhood caries risk assessment tool in a low-income Hispanic population.

    PubMed

    Custodio-Lumsden, Christie L; Wolf, Randi L; Contento, Isobel R; Basch, Charles E; Zybert, Patricia A; Koch, Pamela A; Edelstein, Burton L

    2016-03-01

    There is a recognized need for valid risk assessment tools for use by both dental and nondental personnel to identify young children at risk for, or with, precavitated stages of early childhood caries (i.e., early stage decalcifications or white spot lesions).The aim of this study is to establish concurrent criterion validity of "MySmileBuddy" (MSB), a novel technology-assisted ECC risk assessment and behavioral intervention tool against four measures of ECC activity: semi-quantitative assays of salivary mutans streptococci levels, visible quantity of dental plaque, visual evidence of enamel decalcifications, and cavitation status (none, ECC, severe ECC). One hundred eight children 2-6 years of age presenting to a pediatric dental clinic were recruited from a predominantly Spanish-speaking, low-income, urban population. All children received a comprehensive oral examination and saliva culture for assessment of ECC indicators. Their caregivers completed the iPad-based MSB assessment in its entirety (15-20 minutes). MSB calculated both diet and comprehensive ECC risk scores. Associations between all variables were determined using ordinal logistic regression. MSB diet risk scores were significantly positively associated with salivary mutans (P < 0.05), and approached significance with visible plaque levels (P < 0.1). MSB comprehensive risk scores were significantly associated with both oral mutans and visible plaque (P < 0.05). Neither was associated with visually evident decalcifications or cavitations. Findings suggest that MSB may have clinical utility as a valid risk assessment tool for identifying children with early precursors of cavitations but does not add value in identifying children with extant lesions. © 2015 American Association of Public Health Dentistry.

  9. Procedure-specific assessment tool for flexible pharyngo-laryngoscopy: gathering validity evidence and setting pass-fail standards.

    PubMed

    Melchiors, Jacob; Petersen, K; Todsen, T; Bohr, A; Konge, Lars; von Buchwald, Christian

    2018-06-01

    The attainment of specific identifiable competencies is the primary measure of progress in the modern medical education system. The system, therefore, requires a method for accurately assessing competence to be feasible. Evidence of validity needs to be gathered before an assessment tool can be implemented in the training and assessment of physicians. This evidence of validity must according to the contemporary theory on validity be gathered from specific sources in a structured and rigorous manner. The flexible pharyngo-laryngoscopy (FPL) is central to the otorhinolaryngologist. We aim to evaluate the flexible pharyngo-laryngoscopy assessment tool (FLEXPAT) created in a previous study and to establish a pass-fail level for proficiency. Eighteen physicians with different levels of experience (novices, intermediates, and experienced) were recruited to the study. Each performed an FPL on two patients. These procedures were video recorded, blinded, and assessed by two specialists. The score was expressed as the percentage of a possible max score. Cronbach's α was used to analyze internal consistency of the data, and a generalizability analysis was performed. The scores of the three different groups were explored, and a pass-fail level was determined using the contrasting groups' standard setting method. Internal consistency was strong with a Cronbach's α of 0.86. We found a generalizability coefficient of 0.72 sufficient for moderate stakes assessment. We found a significant difference between the novice and experienced groups (p < 0.001) and strong correlation between experience and score (Pearson's r = 0.75). The pass/fail level was established at 72% of the maximum score. Applying this pass-fail level in the test population resulted in half of the intermediary group receiving a failing score. We gathered validity evidence for the FLEXPAT according to the contemporary framework as described by Messick. Our results support a claim of validity and are

  10. Development of the Music Therapy Assessment Tool for Advanced Huntington's Disease: A Pilot Validation Study.

    PubMed

    O'Kelly, Julian; Bodak, Rebeka

    2016-01-01

    Case studies of people with Huntington's disease (HD) report that music therapy provides a range of benefits that may improve quality of life; however, no robust music therapy assessment tools exist for this population. Develop and conduct preliminary psychometric testing of a music therapy assessment tool for patients with advanced HD. First, we established content and face validity of the Music Therapy Assessment Tool for Advanced HD (MATA-HD) through focus groups and field testing. Second, we examined psychometric properties of the resulting MATA-HD in terms of its construct validity, internal consistency, and inter-rater and intra-rater reliability over 10 group music therapy sessions with 19 patients. The resulting MATA-HD included a total of 15 items across six subscales (Arousal/Attention, Physical Presentation, Communication, Musical, Cognition, and Psychological/Behavioral). We found good construct validity (r ≥ 0.7) for Mood, Communication Level, Communication Effectiveness, Choice, Social Behavior, Arousal, and Attention items. Cronbach's α of 0.825 indicated good internal consistency across 11 items with a common focus of engagement in therapy. The inter-rater reliability (IRR) Intra-Class Coefficient (ICC) scores averaged 0.65, and a mean intra-rater ICC reliability of 0.68 was obtained. Further training and retesting provided a mean of IRR ICC of 0.7. Preliminary data indicate that the MATA-HD is a promising tool for measuring patient responses to music therapy interventions across psychological, physical, social, and communication domains of functioning in patients with advanced HD. © the American Music Therapy Association 2016. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

  11. Development, Evaluation, and Validation of Environmental Assessment Tools to Evaluate the College Nutrition Environment

    ERIC Educational Resources Information Center

    Freedman, Marjorie R.

    2010-01-01

    Objective: To develop, evaluate, and validate 2 nutrition environment assessment tools (surveys), for specific use in combating overweight on college/university campuses. Participants and Methods: Invitations to complete surveys were e-mailed to food service and health center directors at 47 universities, Winter 2008. Overall response rate was…

  12. Overcoming redundancies in bedside nursing assessments by validating a parsimonious meta-tool: findings from a methodological exercise study.

    PubMed

    Palese, Alvisa; Marini, Eva; Guarnier, Annamaria; Barelli, Paolo; Zambiasi, Paola; Allegrini, Elisabetta; Bazoli, Letizia; Casson, Paola; Marin, Meri; Padovan, Marisa; Picogna, Michele; Taddia, Patrizia; Chiari, Paolo; Salmaso, Daniele; Marognolli, Oliva; Canzan, Federica; Ambrosi, Elisa; Saiani, Luisa; Grassetti, Luca

    2016-10-01

    There is growing interest in validating tools aimed at supporting the clinical decision-making process and research. However, an increased bureaucratization of clinical practice and redundancies in the measures collected have been reported by clinicians. Redundancies in clinical assessments affect negatively both patients and nurses. To validate a meta-tool measuring the risks/problems currently estimated by multiple tools used in daily practice. A secondary analysis of a database was performed, using a cross-validation and a longitudinal study designs. In total, 1464 patients admitted to 12 medical units in 2012 were assessed at admission with the Brass, Barthel, Conley and Braden tools. Pertinent outcomes such as the occurrence of post-discharge need for resources and functional decline at discharge, as well as falls and pressure sores, were measured. Explorative factor analysis of each tool, inter-tool correlations and a conceptual evaluation of the redundant/similar items across tools were performed. Therefore, the validation of the meta-tool was performed through explorative factor analysis, confirmatory factor analysis and the structural equation model to establish the ability of the meta-tool to predict the outcomes estimated by the original tools. High correlations between the tools have emerged (from r 0.428 to 0.867) with a common variance from 18.3% to 75.1%. Through a conceptual evaluation and explorative factor analysis, the items were reduced from 42 to 20, and the three factors that emerged were confirmed by confirmatory factor analysis. According to the structural equation model results, two out of three emerged factors predicted the outcomes. From the initial 42 items, the meta-tool is composed of 20 items capable of predicting the outcomes as with the original tools. © 2016 John Wiley & Sons, Ltd.

  13. Validating Appetite Assessment Tools among Patients Receiving Hemodialysis

    PubMed Central

    Molfino, Alessio; Kaysen, George A.; Chertow, Glenn M.; Doyle, Julie; Delgado, Cynthia; Dwyer, Tjien; Laviano, Alessandro; Fanelli, Filippo Rossi; Johansen, Kirsten L.

    2016-01-01

    Objective To test the performance of appetite assessment tools among patients receiving hemodialysis. Design Cross-sectional. Setting Seven dialysis facilities in Northern California. Subjects 221 patients receiving hemodialysis. Intervention We assessed five appetite assessment tools [self-assessment of appetite, subjective assessment of appetite, visual analogue scale (VAS), Functional Assessment of Anorexia/Cachexia Therapy (FAACT) score and the Anorexia Questionnaire (AQ)]. Main outcome measures Reported food intake, normalized protein catabolic rate (nPCR), and change in body weight were used as criterion measures, and we assessed associations among the appetite tools and biomarkers associated with nutrition and inflammation. Patients were asked to report their appetite and the percentage of food eaten (from 0% to 100%) during the last meal compared to usual intake. Results Fifty-eight (26%) patients reported food intake ≤50% (defined as poor appetite). The prevalence of anorexia was 12% by self-assessment of appetite, 6% by subjective assessment of appetite, 24% by VAS, 17% by FAACT score, and 12% by AQ. All tools were significantly associated with food intake ≤50% (p<0.001), except self-assessment of appetite. The FAACT score and the VAS had the strongest association with food intake ≤50% (c-statistic 0.80 and 0.76). Patients with food intake ≤50% reported weight loss more frequently than patients without low intake (36% vs 22%) and weight gain less frequently (19% vs 35%; p=0.03). nPCR was lower among anorexic patients based on the VAS (1.1 ± 0.3 vs 1.2 ± 0.3, p=0.03). Ln IL-6 correlated inversely with food intake (p=0.03), but neither IL-6 nor CRP correlated with any of the appetite tools. Furthermore, only the self-assessment of appetite was significantly associated with serum albumin (p=0.02), prealbumin (p=0.02) and adiponectin concentrations (p=0.03). Conclusions Alternative appetite assessment tools yielded widely different estimates of the

  14. Validating Appetite Assessment Tools Among Patients Receiving Hemodialysis.

    PubMed

    Molfino, Alessio; Kaysen, George A; Chertow, Glenn M; Doyle, Julie; Delgado, Cynthia; Dwyer, Tjien; Laviano, Alessandro; Rossi Fanelli, Filippo; Johansen, Kirsten L

    2016-03-01

    To test the performance of appetite assessment tools among patients receiving hemodialysis (HD). Cross-sectional. Two hundred twenty-one patients receiving HD enrolled in seven dialysis facilities in Northern California. We assessed 5 appetite assessment tools (self-assessment of appetite, subjective assessment of appetite, visual analog scale [VAS], Functional Assessment of Anorexia/Cachexia Therapy [FAACT] score, and the Anorexia Questionnaire [AQ]). Reported food intake, normalized protein catabolic rate, and change in body weight were used as criterion measures, and we assessed associations among the appetite tools and biomarkers associated with nutrition and inflammation. Patients were asked to report their appetite and the percentage of food eaten (from 0% to 100%) during the last meal compared to usual intake. Fifty-eight (26%) patients reported food intake ≤ 50% (defined as poor appetite). The prevalence of anorexia was 12% by self-assessment of appetite, 6% by subjective assessment of appetite, 24% by VAS, 17% by FAACT score, and 12% by AQ. All the tools were significantly associated with food intake ≤ 50% (P < .001), except self-assessment of appetite. The FAACT score and the VAS had the strongest association with food intake ≤ 50% (C-statistic 0.80 and 0.76). Patients with food intake ≤ 50% reported weight loss more frequently than patients without low intake (36% vs 22%) and weight gain less frequently (19% vs 35%; P = .03). Normalized protein catabolic rate was lower among anorexic patients based on the VAS (1.1 ± 0.3 vs 1.2 ± 0.3, P = .03). Ln interleukin-6 correlated inversely with food intake (P = .03), but neither interleukin-6 nor C-reactive protein correlated with any of the appetite tools. Furthermore, only the self-assessment of appetite was significantly associated with serum albumin (P = .02), prealbumin (P = .02) and adiponectin concentrations (P = .03). Alternative appetite assessment tools yielded widely different estimates of

  15. Assessing children’s competence to consent in research by a standardized tool: a validity study

    PubMed Central

    2012-01-01

    Background Currently over 50% of drugs prescribed to children have not been evaluated properly for use in their age group. One key reason why children have been excluded from clinical trials is that they are not considered able to exercise meaningful autonomy over the decision to participate. Dutch law states that competence to consent can be presumed present at the age of 12 and above; however, in pediatric practice children’s competence is not that clearly presented and the transition from assent to active consent is gradual. A gold standard for competence assessment in children does not exist. In this article we describe a study protocol on the development of a standardized tool for assessing competence to consent in research in children and adolescents. Methods/design In this study we modified the MacCAT-CR, the best evaluated competence assessment tool for adults, for use in children and adolescents. We will administer the tool prospectively to a cohort of pediatric patients from 6 to18 years during the selection stages of ongoing clinical trials. The outcomes of the MacCAT-CR interviews will be compared to a reference standard, established by the judgments of clinical investigators, and an expert panel consisting of child psychiatrists, child psychologists and medical ethicists. The reliability, criterion-related validity and reproducibility of the tool will be determined. As MacCAT-CR is a multi-item scale consisting of 13 items, power was justified at 130–190 subjects, providing a minimum of 10–15 observations per item. MacCAT-CR outcomes will be correlated with age, life experience, IQ, ethnicity, socio-economic status and competence judgment of the parent(s). It is anticipated that 160 participants will be recruited over 2 years to complete enrollment. Discussion A validity study on an assessment tool of competence to consent is strongly needed in research practice, particularly in the child and adolescent population. In this study we will establish

  16. Reliability and criterion-related validity testing (construct) of the Endotracheal Suction Assessment Tool (ESAT©).

    PubMed

    Davies, Kylie; Bulsara, Max K; Ramelet, Anne-Sylvie; Monterosso, Leanne

    2018-05-01

    To establish criterion-related construct validity and test-retest reliability for the Endotracheal Suction Assessment Tool© (ESAT©). Endotracheal tube suction performed in children can significantly affect clinical stability. Previously identified clinical indicators for endotracheal tube suction were used as criteria when designing the ESAT©. Content validity was reported previously. The final stages of psychometric testing are presented. Observational testing was used to measure construct validity and determine whether the ESAT© could guide "inexperienced" paediatric intensive care nurses' decision-making regarding endotracheal tube suction. Test-retest reliability of the ESAT© was performed at two time points. The researchers and paediatric intensive care nurse "experts" developed 10 hypothetical clinical scenarios with predetermined endotracheal tube suction outcomes. "Experienced" (n = 12) and "inexperienced" (n = 14) paediatric intensive care nurses were presented with the scenarios and the ESAT© guiding decision-making about whether to perform endotracheal tube suction for each scenario. Outcomes were compared with those predetermined by the "experts" (n = 9). Test-retest reliability of the ESAT© was measured at two consecutive time points (4 weeks apart) with "experienced" and "inexperienced" paediatric intensive care nurses using the same scenarios and tool to guide decision-making. No differences were observed between endotracheal tube suction decisions made by "experts" (n = 9), "inexperienced" (n = 14) and "experienced" (n = 12) nurses confirming the tool's construct validity. No differences were observed between groups for endotracheal tube suction decisions at T1 and T2. Criterion-related construct validity and test-retest reliability of the ESAT© were demonstrated. Further testing is recommended to confirm reliability in the clinical setting with the "inexperienced" nurse to guide decision-making related to endotracheal tube

  17. Validating a work group climate assessment tool for improving the performance of public health organizations

    PubMed Central

    Perry, Cary; LeMay, Nancy; Rodway, Greg; Tracy, Allison; Galer, Joan

    2005-01-01

    Background This article describes the validation of an instrument to measure work group climate in public health organizations in developing countries. The instrument, the Work Group Climate Assessment Tool (WCA), was applied in Brazil, Mozambique, and Guinea to assess the intermediate outcomes of a program to develop leadership for performance improvement. Data were collected from 305 individuals in 42 work groups, who completed a self-administered questionnaire. Methods The WCA was initially validated using Cronbach's alpha reliability coefficient and exploratory factor analysis. This article presents the results of a second validation study to refine the initial analyses to account for nested data, to provide item-level psychometrics, and to establish construct validity. Analyses included eigenvalue decomposition analysis, confirmatory factor analysis, and validity and reliability analyses. Results This study confirmed the validity and reliability of the WCA across work groups with different demographic characteristics (gender, education, management level, and geographical location). The study showed that there is agreement between the theoretical construct of work climate and the items in the WCA tool across different populations. The WCA captures a single perception of climate rather than individual sub-scales of clarity, support, and challenge. Conclusion The WCA is useful for comparing the climates of different work groups, tracking the changes in climate in a single work group over time, or examining differences among individuals' perceptions of their work group climate. Application of the WCA before and after a leadership development process can help work groups hold a discussion about current climate and select a target for improvement. The WCA provides work groups with a tool to take ownership of their own group climate through a process that is simple and objective and that protects individual confidentiality. PMID:16223447

  18. Assessing peristomal skin changes in ostomy patients: validation of the Ostomy Skin Tool.

    PubMed

    Jemec, G B; Martins, L; Claessens, I; Ayello, E A; Hansen, A S; Poulsen, L H; Sibbald, R G

    2011-02-01

    Peristomal skin problems are common and are treated by a variety of health professionals. Clear and consistent communication among these professionals is therefore particularly important. The Ostomy Skin Tool (OST) is a new assessment instrument for the extent and severity of peristomal skin conditions. Formal tests of reliability and validity are necessary for its use in clinical practice, research, and education. To estimate inter- and intra nurse assessment variability of the OST and validity by comparison to a 'gold standard' (GS) defined by an expert panel. Thirty photographs of peristomal skin were presented twice to 20 ostomy care nurses--10 from Denmark (DK) and 10 from Spain (ES)--to determine intra- and inter nurse assessment variability. The same photographs were presented to an international group of experts (dermatologist and ostomy care nurses), to establish a GS for comparison and validation of the results. A high intra-nurse assessment agreement, κ=0·84, was found with no differences in the intra-nurse assessments from the two groups of nurses (DK and ES). The inter-nurse assessment agreement was 'moderate to good', κ=0·54, with the agreement between the experts higher, κ=0·70. A high correlation between the scores from the nurses and the GS were seen in the lower part of the two scales [Discoloration, Erosion, Tissue overgrowth (DET) score<7)]. The study supported the validity of the OST. It is suggested that a categorical scale can be used to illustrate the severity of the DET scores. © 2011 The Authors. BJD © 2011 British Association of Dermatologists.

  19. Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

    PubMed

    Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

    2009-09-01

    The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

  20. The live donor assessment tool: a psychosocial assessment tool for live organ donors.

    PubMed

    Iacoviello, Brian M; Shenoy, Akhil; Braoude, Jenna; Jennings, Tiane; Vaidya, Swapna; Brouwer, Julianna; Haydel, Brandy; Arroyo, Hansel; Thakur, Devendra; Leinwand, Joseph; Rudow, Dianne LaPointe

    2015-01-01

    Psychosocial evaluation is an important part of the live organ donor evaluation process, yet it is not standardized across institutions, and although tools exist for the psychosocial evaluation of organ recipients, none exist to assess donors. We set out to develop a semistructured psychosocial evaluation tool (the Live Donor Assessment Tool, LDAT) to assess potential live organ donors and to conduct preliminary analyses of the tool's reliability and validity. Review of the literature on the psychosocial variables associated with treatment adherence, quality of life, live organ donation outcome, and resilience, as well as review of the procedures for psychosocial evaluation at our center and other centers around the country, identified 9 domains to address; these domains were distilled into several items each, in collaboration with colleagues at transplant centers across the country, for a total of 29 items. Four raters were trained to use the LDAT, and they retrospectively scored 99 psychosocial evaluations conducted on live organ donor candidates. Reliability of the LDAT was assessed by calculating the internal consistency of the items in the scale and interrater reliability between raters; validity was estimated by comparing LDAT scores between those with a "positive" evaluation outcome and "negative" outcome. The LDAT was found to have good internal consistency, inter-rater reliability, and showed signs of validity: LDAT scores differentiated the positive vs. negative outcome groups. The LDAT demonstrated good reliability and validity, but future research on the LDAT and the ability to implement the LDAT prospectively is warranted. Copyright © 2015 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.

  1. Multicentre study for validation of the French addictovigilance network reports assessment tool

    PubMed Central

    Hardouin, Jean Benoit; Rousselet, Morgane; Gerardin, Marie; Guerlais, Marylène; Guillou, Morgane; Bronnec, Marie; Sébille, Véronique; Jolliet, Pascale

    2016-01-01

    Aims The French health authority (ANSM) is responsible for monitoring medicinal and other drug dependencies. To support these activities, the ANSM manages a network of 13 drug dependence evaluation and information centres (Centres d'Evaluation et d'Information sur la Pharmacodépendance ‐ Addictovigilance ‐ CEIP‐A) throughout France. In 2006, the Nantes CEIP‐A created a new tool called the EGAP (Echelle de GrAvité de la Pharmacodépendance‐ drug dependence severity scale) based on DSM IV criteria. This tool allows the creation of a substance use profile that enables the drug dependence severity to be homogeneously quantified by assigning a score to each substance indicated in the reports from health professionals. This article describes the validation and psychometric properties of the drug dependence severity score obtained from the scale ( Clinicaltrials.gov NCT01052675). Method The validity of the EGAP construct, the concurrent validity and the discriminative ability of the EGAP score, the consistency of answers to EGAP items, the internal consistency and inter rater reliability of the EGAP score were assessed using statistical methods that are generally used for psychometric tests. Results The total EGAP score was a reliable and precise measure for evaluating drug dependence (Cronbach alpha = 0.84; ASI correlation = 0.70; global ICC = 0.92). In addition to its good psychometric properties, the EGAP is a simple and efficient tool that can be easily specified on the official ANSM notification form. Conclusion The good psychometric properties of the total EGAP score justify its use for evaluating the severity of drug dependence. PMID:27302554

  2. Development of a Content-Valid Standardized Orthopedic Assessment Tool (SOAT)

    ERIC Educational Resources Information Center

    Lafave, Mark; Katz, Larry; Butterwick, Dale

    2008-01-01

    Content validation of an instrument that measures student performance in OSCE-type practical examinations is a critical step in a tool's overall validity and reliability [Hopkins (1998), "Educational and Psychological Measurement and Evaluation" (8th ed.). Toronto: Allyn & Bacon]. The purpose of the paper is to outline the process…

  3. Power Plant Model Validation Tool

    DOE Office of Scientific and Technical Information (OSTI.GOV)

    The PPMV is used to validate generator model using disturbance recordings. The PPMV tool contains a collection of power plant models and model validation studies, as well as disturbance recordings from a number of historic grid events. The user can import data from a new disturbance into the database, which converts PMU and SCADA data into GE PSLF format, and then run the tool to validate (or invalidate) the model for a specific power plant against its actual performance. The PNNL PPMV tool enables the automation of the process of power plant model validation using disturbance recordings. The tool usesmore » PMU and SCADA measurements as input information. The tool automatically adjusts all required EPCL scripts and interacts with GE PSLF in the batch mode. The main tool features includes: The tool interacts with GE PSLF; The tool uses GE PSLF Play-In Function for generator model validation; Database of projects (model validation studies); Database of the historic events; Database of the power plant; The tool has advanced visualization capabilities; and The tool automatically generates reports« less

  4. The Readiness to Change for Bariatric Surgery Assessment Tool: Validity, Factor Structure, and Reliability.

    PubMed

    Carlson, Susan M S

    2017-11-01

    Currently, there is no guideline or standard of practice for performing the psychiatric/psychological evaluation that is a requirement for approval for bariatric surgery. The Readiness to Change for Bariatric Surgery Assessment Tool (RCB-SAT) establishes a means for psychiatric evaluators to objectively assess the patient's cognition, beliefs, and motivation around the bariatric diet and lifestyle changes. Development of a clinical decision-making tool for assessing readiness to change in bariatric patients will be useful regarding The Strategic Plan for NIH Obesity Research. The strategic plan outlines 6 overarching themes, with the last 3 centering around creation of such a clinical decision-making tool to assess a bariatric patient's readiness to change: evaluate promising strategies for obesity prevention and treatment in realworld settings and diverse populations, harness technology and tools to advance obesity research and improve health care delivery, and facilitate integration of research results into community programs and medical practice (National Institutes of Health, 2011). The pilot tool was administered to 153 potential bariatric patients, with 61 patients completing the survey a second time. Face and content validity of the items were established through an expert review process. Principle axis factoring by means of varimax rotation with Kaiser normalization identified 15 items loading on 3 factors associated with Prochaska and DiClemente's transtheoretical model of health behavior change: precontemplation, contemplation, and action (DiClemente & Prochaska, 1998). Test-retest reliability was also established for the tool. The proposed RCB-SAT demonstrates potential for assessing a patient's readiness to change regarding the bariatric diet and lifestyle.

  5. Assessing physiotherapists' communication skills for promoting patient autonomy for self-management: reliability and validity of the communication evaluation in rehabilitation tool.

    PubMed

    Murray, Aileen; Hall, Amanda; Williams, Geoffrey C; McDonough, Suzanne M; Ntoumanis, Nikos; Taylor, Ian; Jackson, Ben; Copsey, Bethan; Hurley, Deirdre A; Matthews, James

    2018-02-27

    To assess the inter-rater reliability and concurrent validity of the Communication Evaluation in Rehabilitation Tool, which aims to externally assess physiotherapists competency in using Self-Determination Theory-based communication strategies in practice. Audio recordings of initial consultations between 24 physiotherapists and 24 patients with chronic low back pain in four hospitals in Ireland were obtained as part of a larger randomised controlled trial. Three raters, all of whom had Ph.Ds in psychology and expertise in motivation and physical activity, independently listened to the 24 audio recordings and completed the 18-item Communication Evaluation in Rehabilitation Tool. Inter-rater reliability between all three raters was assessed using intraclass correlation coefficients. Concurrent validity was assessed using Pearson's r correlations with a reference standard, the Health Care Climate Questionnaire. The total score for the Communication Evaluation in Rehabilitation Tool is an average of all 18 items. Total scores demonstrated good inter-rater reliability (Intraclass Correlation Coefficient (ICC) = 0.8) and concurrent validity with the Health Care Climate Questionnaire total score (range: r = 0.7-0.88). Item-level scores of the Communication Evaluation in Rehabilitation Tool identified five items that need improvement. Results provide preliminary evidence to support future use and testing of the Communication Evaluation in Rehabilitation Tool. Implications for Rehabilitation Promoting patient autonomy is a learned skill and while interventions exist to train clinicians in these skills there are no tools to assess how well clinicians use these skills when interacting with a patient. The lack of robust assessment has severe implications regarding both the fidelity of clinician training packages and resulting outcomes for promoting patient autonomy. This study has developed a novel measurement tool Communication Evaluation in Rehabilitation Tool and a

  6. Validation of a scenario-based assessment of critical thinking using an externally validated tool.

    PubMed

    Buur, Jennifer L; Schmidt, Peggy; Smylie, Dean; Irizarry, Kris; Crocker, Carlos; Tyler, John; Barr, Margaret

    2012-01-01

    With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.

  7. Neighborhood Quality and Attachment: Validation of the Revised Residential Environment Assessment Tool.

    PubMed

    Poortinga, Wouter; Calve, Tatiana; Jones, Nikki; Lannon, Simon; Rees, Tabitha; Rodgers, Sarah E; Lyons, Ronan A; Johnson, Rhodri

    2017-04-01

    Various studies have shown that neighborhood quality is linked to neighborhood attachment and satisfaction. However, most have relied upon residents' own perceptions rather than independent observations of the neighborhood environment. This study examines the reliability and validity of the revised Residential Environment Assessment Tool (REAT 2.0), an audit instrument covering both public and private spaces of the neighborhood environment. The research shows that REAT 2.0 is a reliable, easy-to-use instrument and that most underlying constructs can be validated against residents' own neighborhood perceptions. The convergent validity of the instrument, which was tested against digital map data, can be improved for a number of miscellaneous urban form items. The research further found that neighborhood attachment was significantly associated with the overall REAT 2.0 score. This association can mainly be attributed to the property-level neighborhood quality and natural elements components. The research demonstrates the importance of private spaces in the outlook of the neighborhood environment.

  8. [Spanish validation of the MacArthur Competence Assessment Tool for Treatment interview to assess patients competence to consent treatment].

    PubMed

    Alvarez Marrodán, Ignacio; Baón Pérez, Beatriz; Navío Acosta, Mercedes; López-Antón, Raul; Lobo Escolar, Elena; Ventura Faci, Tirso

    2014-09-09

    To validate the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) Spanish version, which assesses the mental capacity of patients to consent treatment, by examining 4 areas (Understanding, Appreciation, Reasoning and Expressing a choice). 160 subjects (80 Internal Medicine inpatients, 40 Psychiatric inpatients and 40 healthy controls). MacCAT-T, Mini-Mental Status Examination (MMSE). Feasibility study, reliability and validity calculations (against to gold standard of clinical expert). Mean duration of the MacCAT-T interview was 18min. Inter-rater reliability: Intraclass correlation coefficient for Understanding=0.98, Appreciation=0.97, Reasoning=0.98, Expressing a choice=0.91. Internal consistency (Cronbach's alpha): Understanding=0.87, for Appreciation=0.76, for Reasoning=0.86. Patients considered to be incapable (gold standard) scored lower in all the MacCAT-T areas. Poor performance on the MacCAT-T was related to cognitive impairment assessed by MMSE. Spanish version of the MacCAT-T is feasible, reliable, and valid for assessing the capacity of patients to consent treatment. Copyright © 2013 Elsevier España, S.L. All rights reserved.

  9. Obstetric care providers are able to assess psychosocial risks, identify and refer high-risk pregnant women: validation of a short assessment tool - the KINDEX Greek version.

    PubMed

    Spyridou, Andria; Schauer, Maggie; Ruf-Leuschner, Martina

    2015-02-21

    Prenatal assessment for psychosocial risk factors and prevention and intervention is scarce and, in most cases, nonexistent in obstetrical care. In this study we aimed to evaluate if the KINDEX, a short instrument developed in Germany, is a useful tool in the hands of non-trained medical staff, in order to identify and refer women in psychosocial risk to the adequate mental health and social services. We also examined the criterion-related concurrent validity of the tool through a validation interview carried out by an expert clinical psychologist. Our final objective was to achieve the cultural adaptation of the KINDEX Greek Version and to offer a valid tool for the psychosocial risk assessment to the obstetric care providers. Two obstetricians and five midwives carried out 93 KINDEX interviews (duration 20 minutes) with pregnant women to assess psychosocial risk factors present during pregnancy. Afterwards they referred women who they identified having two or more psychosocial risk factors to the mental health attention unit of the hospital. During the validation procedure an expert clinical psychologist carried out diagnostic interviews with a randomized subsample of 50 pregnant women based on established diagnostic instruments for stress and psychopathology, like the PSS-14, ESI, PDS, HSCL-25. Significant correlations between the results obtained through the assessment using the KINDEX and the risk areas of stress, psychopathology and trauma load assessed in the validation interview demonstrate the criterion-related concurrent validity of the KINDEX. The referral accuracy of the medical staff is confirmed through comparisons between pregnant women who have and have not been referred to the mental health attention unit. Prenatal screenings for psychosocial risks like the KINDEX are feasible in public health settings in Greece. In addition, validity was confirmed in high correlations between the KINDEX results and the results of the validation interviews. The

  10. Improving Escalation of Care: Development and Validation of the Quality of Information Transfer Tool.

    PubMed

    Johnston, Maximilian J; Arora, Sonal; Pucher, Philip H; Reissis, Yannis; Hull, Louise; Huddy, Jeremy R; King, Dominic; Darzi, Ara

    2016-03-01

    To develop and provide validity and feasibility evidence for the QUality of Information Transfer (QUIT) tool. Prompt escalation of care in the setting of patient deterioration can prevent further harm. Escalation and information transfer skills are not currently measured in surgery. This study comprised 3 phases: the development (phase 1), validation (phase 2), and feasibility analysis (phase 3) of the QUIT tool. Phase 1 involved identification of core skills needed for successful escalation of care through literature review and 33 semistructured interviews with stakeholders. Phase 2 involved the generation of validity evidence for the tool using a simulated setting. Thirty surgeons assessed a deteriorating postoperative patient in a simulated ward and escalated their care to a senior colleague. The face and content validity were assessed using a survey. Construct and concurrent validity of the tool were determined by comparing performance scores using the QUIT tool with those measured using the Situation-Background-Assessment-Recommendation (SBAR) tool. Phase 3 was conducted using direct observation of escalation scenarios on surgical wards in 2 hospitals. A 7-category assessment tool was developed from phase 1 consisting of 24 items. Twenty-one of 24 items had excellent content validity (content validity index >0.8). All 7 categories and 18 of 24 (P < 0.05) items demonstrated construct validity. The correlation between the QUIT and SBAR tools used was strong indicating concurrent validity (r = 0.694, P < 0.001). Real-time scoring of escalation referrals was feasible and indicated that doctors currently have better information transfer skills than nurses when faced with a deteriorating patient. A validated tool to assess information transfer for deteriorating surgical patients was developed and tested using simulation and real-time clinical scenarios. It may improve the quality and safety of patient care on the surgical ward.

  11. Reliable and valid assessment of Lichtenstein hernia repair skills.

    PubMed

    Carlsen, C G; Lindorff-Larsen, K; Funch-Jensen, P; Lund, L; Charles, P; Konge, L

    2014-08-01

    Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia repair, (four experts, three intermediates, and three novices). The videos were blindly and individually assessed by three raters (surgical consultants) using the assessment tool. Based on these assessments, validity and reliability were explored. The internal consistency of the items was high (Cronbach's alpha = 0.97). The inter-rater reliability was very good with an intra-class correlation coefficient (ICC) = 0.93. Generalizability analysis showed a coefficient above 0.8 even with one rater. The coefficient improved to 0.92 if three raters were used. One-way analysis of variance found a significant difference between the three groups which indicates construct validity, p < 0.001. Lichtenstein hernia repair skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment of trainees performing Lichtenstein hernia repair to ensure that the objectives of competency-based surgical training are met.

  12. The FORGE AHEAD clinical readiness consultation tool: a validated tool to assess clinical readiness for chronic disease care mobilization in Canada's First Nations.

    PubMed

    Hayward, Mariam Naqshbandi; Mequanint, Selam; Paquette-Warren, Jann; Bailie, Ross; Chirila, Alexandra; Dyck, Roland; Green, Michael; Hanley, Anthony; Tompkins, Jordan; Harris, Stewart

    2017-03-23

    Given the astounding rates of diabetes and related complications, and the barriers to providing care present in Indigenous communities in Canada, intervention strategies that take into account contextual factors such as readiness to mobilize are needed to maximize improvements and increase the likelihood of success and sustainment. As part of the national FORGE AHEAD Program, we sought to develop, test and validate a clinical readiness consultation tool aimed at assessing the readiness of clinical teams working on-reserve in First Nations communities to participate in quality improvement (QI) to enhance diabetes care in Canada. A literature review was conducted to identify existing readiness tools. The ABCD - SAT was adapted using a consensus approach that emphasized a community-based participatory approach and prioritized the knowledge and wisdom held by community members. The tool was piloted with a group of 16 people from 7 provinces and 11 partnering communities to assess language use, clarity, relevance, format, and ease of completion using examples. Internal reliability analysis and convergence validity were conducted with data from 53 clinical team members from 11 First Nations communities (3-5 per community) who have participated in the FORGE AHEAD program. The 27-page Clinical Readiness Consultation Tool (CRCT) consists of five main components, 21 sub-components, and 74 items that are aligned with the Expanded Chronic Care Model. Five-point Likert scale feedback from the pilot ranged from 3.25 to 4.5. Length of the tool was reported as a drawback but respondents noted that all the items were needed to provide a comprehensive picture of the healthcare system. Results for internal consistency showed that all sub-components except for two were within acceptable ranges (0.77-0.93). The Team Structure and Function sub-component scale had a moderately significant positive correlation with the validated Team Climate Inventory, r = 0.45, p < 0.05. The

  13. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

    PubMed

    Shea, Beverley J; Hamel, Candyce; Wells, George A; Bouter, Lex M; Kristjansson, Elizabeth; Grimshaw, Jeremy; Henry, David A; Boers, Maarten

    2009-10-01

    Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR). We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times. The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete. AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.

  14. Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study.

    PubMed

    Morris, Christopher; Hoogenes, Jen; Shayegan, Bobby; Matsumoto, Edward D

    2017-01-01

    As urology training shifts toward competency-based frameworks, the need for tools for high stakes assessment of trainees is crucial. Validated assessment metrics are lacking for many robot-assisted radical prostatectomy (RARP). As it is quickly becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment tool for training is timely. We recruited 13 expert RARP surgeons from the United States and Canada to serve as our Delphi panel. Using an initial inventory developed via a modified Delphi process with urology residents, fellows, and staff at our institution, panelists iteratively rated each step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited for each item to determine proper step placement, wording, and suggestions. Panelist's responses were compiled and the inventory was edited through three iterations, after which 100% consensus was achieved. The initial inventory steps were decreased by 13% and a skip pattern was incorporated. The final RARP stepwise inventory was comprised of 13 critical steps with 52 sub-steps. There was no attrition throughout the Delphi process. Our Delphi study resulted in a comprehensive inventory of intraoperative RARP steps with excellent consensus. This final inventory will be used to develop a valid and psychometrically sound intraoperative assessment tool for use during RARP training and evaluation, with the aim of increasing competency of all trainees. Copyright® by the International Brazilian Journal of Urology.

  15. Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study

    PubMed Central

    Morris, Christopher; Hoogenes, Jen; Shayegan, Bobby; Matsumoto, Edward D.

    2017-01-01

    ABSTRACT Introduction As urology training shifts toward competency-based frameworks, the need for tools for high stakes assessment of trainees is crucial. Validated assessment metrics are lacking for many robot-assisted radical prostatectomy (RARP). As it is quickly becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment tool for training is timely. Materials and methods We recruited 13 expert RARP surgeons from the United States and Canada to serve as our Delphi panel. Using an initial inventory developed via a modified Delphi process with urology residents, fellows, and staff at our institution, panelists iteratively rated each step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited for each item to determine proper step placement, wording, and suggestions. Results Panelist’s responses were compiled and the inventory was edited through three iterations, after which 100% consensus was achieved. The initial inventory steps were decreased by 13% and a skip pattern was incorporated. The final RARP stepwise inventory was comprised of 13 critical steps with 52 sub-steps. There was no attrition throughout the Delphi process. Conclusions Our Delphi study resulted in a comprehensive inventory of intraoperative RARP steps with excellent consensus. This final inventory will be used to develop a valid and psychometrically sound intraoperative assessment tool for use during RARP training and evaluation, with the aim of increasing competency of all trainees. PMID:28379668

  16. Validation and Cultural Adaptation of the Arabic Version of the Eating Assessment Tool (EAT-10).

    PubMed

    Farahat, Mohamed; Mesallam, Tamer A

    2015-01-01

    The Eating Assessment Tool (EAT-10) is a 10-item self-administered questionnaire. It is a noninvasive tool to measure patients' perception of their swallowing problems. The purposes of the present study were to develop an Arabic version of the EAT-10 and to evaluate its validity, consistency, and reliability in the Arabic-speaking population with oropharyngeal dysphagia. This was a prospective study carried out at the Communication and Swallowing Disorders Unit, King Saud University, Riyadh, Saudi Arabia. The Arabic EAT-10 was administered to 138 patients with oropharyngeal dysphagia and 83 control subjects. Internal consistency and test-retest reliability were evaluated. Content and clinical validity were studied, and the EAT-10 results were compared across patients and control groups. The Arabic EAT-10 showed excellent internal consistency (Cronbach's α = 0.92). Also, good test-retest reliability was found for the total scores of the Arabic EAT-10 (intraclass correlation = 0.73). There was a significant difference in Arabic EAT-10 scores between the oropharyngeal dysphagia group and the control group (p < 0.001). This study demonstrated that the Arabic EAT-10 is a valid tool that can be used for screening of dysphagia-related problems in an Arabic-speaking population. © 2016 S. Karger AG, Basel.

  17. Measuring couple relationship quality in a rural African population: Validation of a Couple Functionality Assessment Tool in Malawi.

    PubMed

    Ruark, Allison; Chase, Rachel; Hembling, John; Davis, Valerie Rhoe; Perrin, Paul Clayton; Brewster-Lee, Dorothy

    2017-01-01

    Available data suggest that individual and family well-being are linked to the quality of women's and men's couple relationships, but few tools exist to assess couple relationship functioning in low- and middle-income countries. In response to this gap, Catholic Relief Services has developed a Couple Functionality Assessment Tool (CFAT) to capture valid and reliable data on various domains of relationship quality. This tool is designed to be used by interventions which aim to improve couple and family well-being as a means of measuring the effectiveness of these interventions, particularly related to couple relationship quality. We carried out a validation study of the CFAT among 401 married and cohabiting adults (203 women and 198 men) in rural Chikhwawa District, Malawi. Using psychometric scales, the CFAT addressed six domains of couple relationship quality (intimacy, partner support, sexual satisfaction, gender roles, decision-making, and communication and conflict management), and included questions on intimate partner violence. We used exploratory factor analysis to assess scale performance of each domain and produce a shortened Relationship Quality Index (RQI) composed of items from five relationship quality domains. This article reports the performance of the RQI. Internal reliability and validity of the RQI were found to be good. Regression analyses examined the relationship of the RQI to outcomes important to health and development: intra-household cooperation, positive health behaviors, intimate partner violence, and gender-equitable norms. We found many significant correlations between RQI scores and these couple- and family-level development issues. There is a need to further validate the tool with use in other populations as well as to continue to explore whether the observed linkages between couple functionality and development outcomes are causal relationships.

  18. Measuring couple relationship quality in a rural African population: Validation of a Couple Functionality Assessment Tool in Malawi

    PubMed Central

    Chase, Rachel; Hembling, John; Davis, Valerie Rhoe; Perrin, Paul Clayton; Brewster-Lee, Dorothy

    2017-01-01

    Available data suggest that individual and family well-being are linked to the quality of women’s and men’s couple relationships, but few tools exist to assess couple relationship functioning in low- and middle-income countries. In response to this gap, Catholic Relief Services has developed a Couple Functionality Assessment Tool (CFAT) to capture valid and reliable data on various domains of relationship quality. This tool is designed to be used by interventions which aim to improve couple and family well-being as a means of measuring the effectiveness of these interventions, particularly related to couple relationship quality. We carried out a validation study of the CFAT among 401 married and cohabiting adults (203 women and 198 men) in rural Chikhwawa District, Malawi. Using psychometric scales, the CFAT addressed six domains of couple relationship quality (intimacy, partner support, sexual satisfaction, gender roles, decision-making, and communication and conflict management), and included questions on intimate partner violence. We used exploratory factor analysis to assess scale performance of each domain and produce a shortened Relationship Quality Index (RQI) composed of items from five relationship quality domains. This article reports the performance of the RQI. Internal reliability and validity of the RQI were found to be good. Regression analyses examined the relationship of the RQI to outcomes important to health and development: intra-household cooperation, positive health behaviors, intimate partner violence, and gender-equitable norms. We found many significant correlations between RQI scores and these couple- and family-level development issues. There is a need to further validate the tool with use in other populations as well as to continue to explore whether the observed linkages between couple functionality and development outcomes are causal relationships. PMID:29190769

  19. Assessing Procedural Competence: Validity Considerations.

    PubMed

    Pugh, Debra M; Wood, Timothy J; Boulet, John R

    2015-10-01

    Simulation-based medical education (SBME) offers opportunities for trainees to learn how to perform procedures and to be assessed in a safe environment. However, SBME research studies often lack robust evidence to support the validity of the interpretation of the results obtained from tools used to assess trainees' skills. The purpose of this paper is to describe how a validity framework can be applied when reporting and interpreting the results of a simulation-based assessment of skills related to performing procedures. The authors discuss various sources of validity evidence because they relate to SBME. A case study is presented.

  20. Validity, efficacy and reliability of 3 nutritional screening tools regarding the nutritional assessment in different social and health areas.

    PubMed

    Castro-Vega, Iciar; Veses Martín, Silvia; Cantero Llorca, Juana; Barrios Marta, Cristina; Bañuls, Celia; Hernández-Mijares, Antonio

    2018-03-09

    Nutritional screening allows for the detection of nutritional risk. Validated tools should be implemented, and their usefulness should be contrasted with a gold standard. The aim of this study is to discover the validity, efficacy and reliability of 3 nutritional screening tools in relation to complete nutritional assessment. A sub-analysis of a cross-sectional and descriptive study on the prevalence of disease-related malnutrition. The sample was selected from outpatients, hospitalized and institutionalized patients. MUST, MNAsf and MST screening were employed. A nutritional assessment of all the patients was undertaken. The SENPE-SEDOM consensus was used for the diagnosis. In the outpatients, both MUST and MNAsf have a similar validity in relation to the nutritional assessment (AUC 0.871 and 0.883, respectively). In the institutionalized patients, the MUST screening method is the one that shows the greatest validity (AUC 0.815), whereas in the hospitalized patients, the most valid methods are both MUST and MST (AUC 0.868 and 0.853, respectively). It is essential to use nutritional screening to invest the available resources wisely. Based on our results, MUST is the most suitable screening method in hospitalized and institutionalized patients. Copyright © 2017 Elsevier España, S.L.U. All rights reserved.

  1. Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men.

    PubMed

    Hussein, Ahmed A; Sexton, Kevin J; May, Paul R; Meng, Maxwell V; Hosseini, Abolfazl; Eun, Daniel D; Daneshmand, Siamak; Bochner, Bernard H; Peabody, James O; Abaza, Ronney; Skinner, Eila C; Hautmann, Richard E; Guru, Khurshid A

    2018-04-13

    We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men. A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation. The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains. We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.

  2. Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills.

    PubMed

    Havemann, Maria Cecilie; Dalsgaard, Torur; Sørensen, Jette Led; Røssaak, Kristin; Brisling, Steffen; Mosgaard, Berit Jul; Høgdall, Claus; Bjerrum, Flemming

    2018-05-14

    Increasing focus on patient safety makes it important to ensure surgical competency among surgeons before operating on patients. The objective was to gather validity evidence for a virtual-reality simulator test for robotic surgical skills and evaluate its potential as a training tool. Surgeons with varying experience in robotic surgery were recruited: novices (zero procedures), intermediates (1-50), experienced (> 50). Five experienced surgeons rated five exercises on the da Vinci Skills Simulator. Participants were tested using the five exercises. Participants were invited back 3 times and completed a total of 10 attempts per exercise. The outcome was the average simulator performance score for the 5 exercises. 32 participants from 5 surgical specialties were included. 38 participants completed all 4 sessions. A moderate correlation between the average total score and robotic experience was identified for the first attempt (Spearman r = 0.58; p = 0.0004). A difference in average total score was observed between novices and intermediates [median score 61% (IQR 52-66) vs. 83% (IQR 75-91), adjusted p < 0.0001], as well as novices and experienced [median score 61% (IQR 52-66) vs. 80 (IQR 69-85), adjusted p = 0.002]. All three groups improved their performance between the 1st and 10th attempts (p < 0.00). This study describes validity evidence for a virtual-reality simulator for basic robotic surgical skills, which can be used for assessment of basic competency and as a training tool. However, more validity evidence is needed before it can be used for certification or high-stakes assessment.

  3. A web-based team-oriented medical error communication assessment tool: development, preliminary reliability, validity, and user ratings.

    PubMed

    Kim, Sara; Brock, Doug; Prouty, Carolyn D; Odegard, Peggy Soule; Shannon, Sarah E; Robins, Lynne; Boggs, Jim G; Clark, Fiona J; Gallagher, Thomas

    2011-01-01

    Multiple-choice exams are not well suited for assessing communication skills. Standardized patient assessments are costly and patient and peer assessments are often biased. Web-based assessment using video content offers the possibility of reliable, valid, and cost-efficient means for measuring complex communication skills, including interprofessional communication. We report development of the Web-based Team-Oriented Medical Error Communication Assessment Tool, which uses videotaped cases for assessing skills in error disclosure and team communication. Steps in development included (a) defining communication behaviors, (b) creating scenarios, (c) developing scripts, (d) filming video with professional actors, and (e) writing assessment questions targeting team communication during planning and error disclosure. Using valid data from 78 participants in the intervention group, coefficient alpha estimates of internal consistency were calculated based on the Likert-scale questions and ranged from α=.79 to α=.89 for each set of 7 Likert-type discussion/planning items and from α=.70 to α=.86 for each set of 8 Likert-type disclosure items. The preliminary test-retest Pearson correlation based on the scores of the intervention group was r=.59 for discussion/planning and r=.25 for error disclosure sections, respectively. Content validity was established through reliance on empirically driven published principles of effective disclosure as well as integration of expert views across all aspects of the development process. In addition, data from 122 medicine and surgical physicians and nurses showed high ratings for video quality (4.3 of 5.0), acting (4.3), and case content (4.5). Web assessment of communication skills appears promising. Physicians and nurses across specialties respond favorably to the tool.

  4. Validating an Environmental Education Field Day Observation Tool

    ERIC Educational Resources Information Center

    Carlson, Stephan P.; Heimlich, Joe E.; Storksdieck, Martin

    2011-01-01

    Environmental Field Days (EFD) are held throughout the country and provide a unique opportunity to involve students in real world science. A study to assess the validity of an observation tool for EFD programs was conducted at the Metro Water Festival with fifth grade students. Items from the observation tool were mapped to students' evaluation…

  5. Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review.

    PubMed

    Zendejas, Benjamin; Ruparel, Raaj K; Cook, David A

    2016-02-01

    The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.

  6. Assessing Reliability and Validity of the "GroPromo" Audit Tool for Evaluation of Grocery Store Marketing and Promotional Environments

    ERIC Educational Resources Information Center

    Kerr, Jacqueline; Sallis, James F.; Bromby, Erica; Glanz, Karen

    2012-01-01

    Objective: To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Methods: Trained observers used the "GroPromo" instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity,…

  7. Developing and Validating a Tool to Assess Ethical Decision-Making Ability of Nursing Students, Using Rubrics

    ERIC Educational Resources Information Center

    Indhraratana, Apinya; Kaemkate, Wannee

    2012-01-01

    The aim of this paper is to develop a reliable and valid tool to assess ethical decision-making ability of nursing students using rubrics. A proposed ethical decision making process, from reviewing related literature was used as a framework for developing the rubrics. Participants included purposive sample of 86 nursing students from the Royal…

  8. [Validation of a nutritional screening tool for hospitalized pediatric patients].

    PubMed

    Lama More, R A; Moráis López, A; Herrero Álvarez, M; Caraballo Chicano, S; Galera Martínez, R; López Ruzafa, E; Rodríguez Martínez, G; de la Mano Hernández, A; Rivero de la Rosa, M C

    2012-01-01

    Malnutrition among hospitalized patients has clinical implications, and interest has arisen to find screening tools able to identify subjects under risk. At present, there is no consensus about the most suitable nutrition screening tool for pediatric patients. To validate STAMP (Screening Tool for the Assessment of Malnutrition in Pediatrics) pediatric screening tool in Spain. Descriptive cross-sectional study of patients admitted to a 3rd level children's hospital with both medical and surgical specialities. During the first 24 hours of admission, STAMP screening tool was applied. For its validation, results were compared with those obtained from a nutritional assessment performed by specialist staff, which included clinical, anthropometric and body composition data. A sample of 250 children was studied. Nutritional assessment identified 64 patients (25.6%) under risk, 40 of whom were malnourished (16%). STAMP classified 48.4% of the patients as being under nutritional risk. This tool showed 75% sensitivity and 60.8% specificity when identifying patients under risk according to nutritional assessment. It showed 90% sensitivity and 59.5% specificity when identifying malnourished patients. Malnutrition was less frequent than that reported in other European countries, although diagnosis technique was different. STAMP is a simple and useful tool for nutritional screening, avoiding the need to assess all patients on admission in order to identify those under nutritional risk.

  9. Issues in developing valid assessments of speech pathology students' performance in the workplace.

    PubMed

    McAllister, Sue; Lincoln, Michelle; Ferguson, Alison; McAllister, Lindy

    2010-01-01

    Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described. A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process. This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools. Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the

  10. Clinical assessment of dysphagia in neurodegeneration (CADN): development, validity and reliability of a bedside tool for dysphagia assessment.

    PubMed

    Vogel, Adam P; Rommel, Natalie; Sauer, Carina; Horger, Marius; Krumm, Patrick; Himmelbach, Marc; Synofzik, Matthis

    2017-06-01

    Screening assessments for dysphagia are essential in neurodegenerative disease. Yet there are no purpose-built tools to quantify swallowing deficits at bedside or in clinical trials. A quantifiable, brief, easy to administer assessment that measures the impact of dysphagia and predicts the presence or absence of aspiration is needed. The Clinical Assessment of Dysphagia in Neurodegeneration (CADN) was designed by a multidisciplinary team (neurology, neuropsychology, speech pathology) validated against strict methodological criteria in two neurodegenerative diseases, Parkinson's disease (PD) and degenerative ataxia (DA). CADN comprises two parts, an anamnesis (part one) and consumption (part two). Two-thirds of patients were assessed using reference tests, the SWAL-QOL symptoms subscale (part one) and videofluoroscopic assessment of swallowing (part two). CADN has 11 items and can be administered and scored in an average of 7 min. Test-retest reliability was established using correlation and Bland-Altman plots. 125 patients with a neurodegenerative disease were recruited; 60 PD and 65 DA. Validity was established using ROC graphs and correlations. CADN has sensitivity of 79 and 84% and specificity 71 and 69% for parts one and two, respectively. Significant correlations with disease severity were also observed (p < 0.001) for PD with small to large associations between disease severity and CADN scores for DA. Cutoff scores were identified that signal the presence of clinically meaningful dysphagia symptomatology and risk of aspiration. The CADN is a reliable, valid, brief, quantifiable, and easily deployed assessment of swallowing in neurodegenerative disease. It is thus ideally suited for both clinical bedside assessment and future multicentre clinical trials in neurodegenerative disease.

  11. The Behavior Pain Assessment Tool for critically ill adults: a validation study in 28 countries.

    PubMed

    Gélinas, Céline; Puntillo, Kathleen A; Levin, Pavel; Azoulay, Elie

    2017-05-01

    Many critically ill adults are unable to communicate their pain through self-report. The study purpose was to validate the use of the 8-item Behavior Pain Assessment Tool (BPAT) in patients hospitalized in 192 intensive care units from 28 countries. A total of 4812 procedures in 3851 patients were included in data analysis. Patients were assessed with the BPAT before and during procedures by 2 different raters (mostly nurses and physicians). Those who were able to self-report were asked to rate their pain intensity and pain distress on 0 to 10 numeric rating scales. Interrater reliability of behavioral observations was supported by moderate (0.43-0.60) to excellent (>0.60) kappa coefficients. Mixed effects multilevel logistic regression models showed that most behaviors were more likely to be present during the procedure than before and in less sedated patients, demonstrating discriminant validation of the tool use. Regarding criterion validation, moderate positive correlations were found during procedures between the mean BPAT scores and the mean pain intensity (r = 0.54) and pain distress (r = 0.49) scores (P < 0.001). Regression models showed that all behaviors were significant predictors of pain intensity and pain distress, accounting for 35% and 29% of their total variance, respectively. A BPAT cut-point score >3.5 could classify patients with or without severe levels (≥8) of pain intensity and distress with sensitivity and specificity findings ranging from 61.8% to 75.1%. The BPAT was found to be reliable and valid. Its feasibility for use in practice and the effect of its clinical implementation on patient pain and intensive care unit outcomes need further research.

  12. Cognitive assessment tools in Asia: a systematic review.

    PubMed

    Rosli, Roshaslina; Tan, Maw Pin; Gray, William Keith; Subramanian, Pathmawathi; Chin, Ai-Vyrn

    2016-02-01

    The prevalence of dementia is increasing in Asia than in any other continent. However, the applicability of the existing cognitive assessment tools is limited by differences in educational and cultural factors in this setting. We conducted a systematic review of published studies on cognitive assessments tools in Asia. We aimed to rationalize the results of available studies which evaluated the validity of cognitive tools for the detection of cognitive impairment and to identify the issues surrounding the available cognitive impairment screening tools in Asia. Five electronic databases (CINAHL, MEDLINE, Embase, Cochrane Library, and Science Direct) were searched using the keywords dementia Or Alzheimer Or cognitive impairment And screen Or measure Or test Or tool Or instrument Or assessment, and 2,381 articles were obtained. Thirty-eight articles, evaluating 28 tools in seven Asian languages, were included. Twenty-nine (76%) of the studies had been conducted in East Asia with only four studies conducted in South Asia and no study from northern, western, or central Asia or Indochina. Local language translations of the Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) were assessed in 15 and six studies respectively. Only three tools (the Korean Dementia Screening Questionnaire, the Picture-based Memory Intelligence Scale, and the revised Hasegawa Dementia Screen) were derived de novo from Asian populations. These tools were assessed in five studies. Highly variable cut-offs were reported for the MMSE (17-29/30) and MoCA (21-26/30), with 13/19 (68%) of studies reporting educational bias. Few cognitive assessment tools have been validated in Asia, with no published validation studies for many Asian nations and languages. In addition, many available tools display educational bias. Future research should include concerted efforts to develop culturally appropriate tools with minimal educational bias.

  13. Consequential Validity and the Transformation of Tests from Measurement Tools to Policy Tools

    ERIC Educational Resources Information Center

    Welner, Kevin G.

    2013-01-01

    Background/Context: Recent U.S. policy has brought a shift in assessment use, from measurement tools to policy levers. In particular, testing has become a core part of teacher evaluation policies in many states, with test results becoming akin to a job evaluation. Purpose: To explore the notion of consequential validity in assessment use and…

  14. Validation of Caregiver-Centered Delirium Detection Tools: A Systematic Review.

    PubMed

    Rosgen, Brianna; Krewulak, Karla; Demiantschuk, Danielle; Ely, E Wesley; Davidson, Judy E; Stelfox, Henry T; Fiest, Kirsten M

    2018-04-18

    To summarize the validity of caregiver-centered delirium detection tools in hospitalized adults and assess associated patient and caregiver outcomes. Systematic review. We searched MEDLINE, EMBASE, PsycINFO, CINAHL, and Scopus from inception to May 15, 2017. Hospitalized adults. Caregiver-centered delirium detection tools. We drafted a protocol from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Two reviewers independently completed abstract and full-text review, data extraction, and quality assessment. We summarized findings using descriptive statistics including mean, median, standard deviation, range, frequencies (percentages), and Cohen's kappa. Studies that reported on the validity of caregiver-centered delirium detection tools or associated patient and caregiver outcomes and were cohort or cross-sectional in design were included. We reviewed 6,056 titles and abstracts, included 6 articles, and identified 6 caregiver-centered tools. All tools were designed to be administered in several minutes or less and had 11 items or fewer. Three tools were caregiver administered (completed independently by caregivers): Family Confusion Assessment Method (FAM-CAM), Informant Assessment of Geriatric Delirium (I-AGeD), and Sour Seven. Three tools were caregiver informed (administered by a healthcare professional using caregiver input): Single Question in Delirium (SQiD), Single Screening Question Delirium (SSQ-Delirium), and Stressful Caregiving Response to Experiences of Dying. Caregiver-administered tools had better psychometric properties (FAM-CAM sensitivity 75%, 95% confidence interval (CI)=35-95%, specificity 91%, 95% CI=74-97%; Sour Seven positive predictive value 89.5%, negative predictive value 90%) than caregiver-informed tools (SQiD: sensitivity 80%, 95% CI=28.4-99.5%; specificity 71%, 95% CI=41.9-91.6%; SSQ-Delirium sensitivity 79.6%, specificity 56.1%). Delirium detection is essential for appropriate delirium management

  15. Technical skills assessment toolbox: a review using the unitary framework of validity.

    PubMed

    Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

    2015-02-01

    The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

  16. Spanish validation of the Person-centered Care Assessment Tool (P-CAT).

    PubMed

    Martínez, Teresa; Suárez-Álvarez, Javier; Yanguas, Javier; Muñiz, José

    2016-01-01

    Person-centered Care (PCC) is an innovative approach which seeks to improve the quality of care services given to the care-dependent elderly. At present there are no Spanish language instruments for the evaluation of PCC delivered by elderly care services. The aim of this work is the adaptation and validation of the Person-centered Care Assessment Tool (P-CAT) for a Spanish population. The P-CAT was translated and adapted into Spanish, then given to a sample of 1339 front-line care professionals from 56 residential elderly care homes. The reliability and validity of the P-CAT were analyzed, within the frameworks of Classical Test Theory and Item Response Theory models. The Spanish P-CAT demonstrated good reliability, with an alpha coefficient of .88 and a test-retest reliability coefficient of .79. The P-CAT information function indicates that the test measures with good precision for the majority of levels of the measured variables (θ values between -2 and +1). The factorial structure of the test is essentially one-dimensional and the item discrimination indices are high, with values between .26 and .61. In terms of predictive validity, the correlations which stand out are between the P-CAT and organizational climate (r = .689), and the burnout factors; personal accomplishment (r = .382), and emotional exhaustion (r = - .510). The Spanish version of the P-CAT demonstrates good psychometric properties for its use in the evaluation of elderly care homes both professionally and in research.

  17. Assessing Households Preparedness for Earthquakes: An Exploratory Study in the Development of a Valid and Reliable Persian-version Tool.

    PubMed

    Ardalan, Ali; Sohrabizadeh, Sanaz

    2016-02-25

    Iran is placed among countries suffering from the highest number of earthquake casualties. Household preparedness, as one component of risk reduction efforts, is often supported in quake-prone areas. In Iran, lack of a valid and reliable household preparedness tool was reported by previous disaster studies. This study is aimed to fill this gap by developing a valid and reliable tool for assessing household preparedness in the event of an earthquake.  This survey was conducted through three phases including literature review and focus group discussions with the participation of eight key informants, validity measurements and reliability measurements. Field investigation was completed with the participation of 450 households within three provinces of Iran. Content validity, construct validity, the use of factor analysis; internal consistency using Cronbach's alpha coefficient, and test-retest reliability were carried out to develop the tool.  Based on the CVIs, ranging from 0.80 to 0.100, and exploratory factor analysis with factor loading of more than 0.5, all items were valid. The amount of Cronbach's alpha (0.7) and test-retest examination by Spearman correlations indicated that the scale was also reliable. The final instrument consisted of six categories and 18 questions including actions at the time of earthquakes, nonstructural safety, structural safety, hazard map, communications, drill, and safety skills.  Using a Persian-version tool that is adjusted to the socio-cultural determinants and native language may result in more trustful information on earthquake preparedness. It is suggested that disaster managers and researchers apply this tool in their future household preparedness projects. Further research is needed to make effective policies and plans for transforming preparedness knowledge into behavior.

  18. Development and validation of a fine-motor assessment tool for use with young children in a Chinese population.

    PubMed

    Siu, Andrew M H; Lai, Cynthia Y Y; Chiu, Amy S M; Yip, Calvin C K

    2011-01-01

    Most of the fine-motor assessment tools used in Hong Kong have been designed in Western countries, so there is a need to develop a standardized assessment which is relevant to the culture and daily living tasks of the local (that is, Chinese) population. This study aimed to (1) develop a fine-motor assessment tool (the Hong Kong Preschool Fine-Motor Developmental Assessment [HK-PFMDA]) for use with young children in a Chinese population and (2) examine the HK-PFMDA's psychometric properties. The HK-PFMDA was developed by a group of occupational therapists specializing in the area of developmental disabilities in Hong Kong. A panel of 21 experts reviewed the content validity of the instrument. Rasch item analysis was used to examine the model fit of items against the rating scale model, and to explore the dimensionality of the test. Intra- and interrater reliability, convergent validity, and criterion-related validity were examined. The participants included 783 children without disabilities, 45 with autistic spectrum disorder, and 35 with developmental delay. The Rasch analysis suggested that the 87-item HK-PFMDA had a unidimensional structure, as the items explained most (91.6%) of the variance. The HK-PFMDA demonstrated excellent intra- (ICC = .99) and interrater reliability (ICC = .99), and internal consistency (α ranging from .83 to .92). In terms of validity, the HK-PFMDA had significant positive correlations with both age and the convergent measures of the Peabody Developmental Motor Scales (PDMS-2). A set of normative data for local children aged from birth to 6 years was established. The HK-PFMDA has shown excellent psychometric properties and is suitable for clinical application by occupational therapists in the assessment of fine-motor skills development of young children in Chinese populations. Copyright © 2010 Elsevier Ltd. All rights reserved.

  19. Teamwork Assessment Tools in Obstetric Emergencies: A Systematic Review.

    PubMed

    Onwochei, Desire N; Halpern, Stephen; Balki, Mrinalini

    2017-06-01

    Team-based training and simulation can improve patient safety, by improving communication, decision making, and performance of team members. Currently, there is no general consensus on whether or not a specific assessment tool is better adapted to evaluate teamwork in obstetric emergencies. The purpose of this qualitative systematic review was to find the tools available to assess team effectiveness in obstetric emergencies. We searched Embase, Medline, PubMed, Web of Science, PsycINFO, CINAHL, and Google Scholar for prospective studies that evaluated nontechnical skills in multidisciplinary teams involving obstetric emergencies. The search included studies from 1944 until January 11, 2016. Data on reliability and validity measures were collected and used for interpretation. A descriptive analysis was performed on the data. Thirteen studies were included in the final qualitative synthesis. All the studies assessed teams in the context of obstetric simulation scenarios, but only six included anesthetists in the simulations. One study evaluated their teamwork tool using just validity measures, five using just reliability measures, and one used both. The most reliable tools identified were the Clinical Teamwork Scale, the Global Assessment of Obstetric Team Performance, and the Global Rating Scale of performance. However, they were still lacking in terms of quality and validity. More work needs to be conducted to establish the validity of teamwork tools for nontechnical skills, and the development of an ideal tool is warranted. Further studies are required to assess how outcomes, such as performance and patient safety, are influenced when using these tools.

  20. The development and testing of a skin tear risk assessment tool.

    PubMed

    Newall, Nelly; Lewin, Gill F; Bulsara, Max K; Carville, Keryln J; Leslie, Gavin D; Roberts, Pam A

    2017-02-01

    The aim of the present study is to develop a reliable and valid skin tear risk assessment tool. The six characteristics identified in a previous case control study as constituting the best risk model for skin tear development were used to construct a risk assessment tool. The ability of the tool to predict skin tear development was then tested in a prospective study. Between August 2012 and September 2013, 1466 tertiary hospital patients were assessed at admission and followed up for 10 days to see if they developed a skin tear. The predictive validity of the tool was assessed using receiver operating characteristic (ROC) analysis. When the tool was found not to have performed as well as hoped, secondary analyses were performed to determine whether a potentially better performing risk model could be identified. The tool was found to have high sensitivity but low specificity and therefore have inadequate predictive validity. Secondary analysis of the combined data from this and the previous case control study identified an alternative better performing risk model. The tool developed and tested in this study was found to have inadequate predictive validity. The predictive validity of an alternative, more parsimonious model now needs to be tested. © 2015 Medicalhelplines.com Inc and John Wiley & Sons Ltd.

  1. Development and Initial Validation of the NyTid Test: A Movement Assessment Tool for Compulsory School Pupils

    ERIC Educational Resources Information Center

    Tidén, Anna; Lundqvist, Carolina; Nyberg, Marie

    2015-01-01

    This study presents the development process and initial validation of the NyTid test, a process-oriented movement assessment tool for compulsory school pupils. A sample of 1,260 (627 girls and 633 boys; mean age of 14.39) Swedish school children participated in the study. In the first step, exploratory factor analyses (EFAs) were performed in…

  2. Psychometric and cognitive validation of a social capital measurement tool in Peru and Vietnam.

    PubMed

    De Silva, Mary J; Harpham, Trudy; Tuan, Tran; Bartolini, Rosario; Penny, Mary E; Huttly, Sharon R

    2006-02-01

    Social capital is a relatively new concept which has attracted significant attention in recent years. No consensus has yet been reached on how to measure social capital, resulting in a large number of different tools available. While psychometric validation methods such as factor analysis have been used by a few studies to assess the internal validity of some tools, these techniques rely on data already collected by the tool and are therefore not capable of eliciting what the questions are actually measuring. The Young Lives (YL) study includes quantitative measures of caregiver's social capital in four countries (Vietnam, Peru, Ethiopia, and India) using a short version of the Adapted Social Capital Assessment Tool (SASCAT). A range of different psychometric methods including factor analysis were used to evaluate the construct validity of SASCAT in Peru and Vietnam. In addition, qualitative cognitive interviews with 20 respondents from Peru and 24 respondents from Vietnam were conducted to explore what each question is actually measuring. We argue that psychometric validation techniques alone are not sufficient to adequately validate multi-faceted social capital tools for use in different cultural settings. Psychometric techniques show SASCAT to be a valid tool reflecting known constructs and displaying postulated links with other variables. However, results from the cognitive interviews present a more mixed picture with some questions being appropriately interpreted by respondents, and others displaying significant differences between what the researchers intended them to measure and what they actually do. Using evidence from a range of methods of assessing validity has enabled the modification of an existing instrument into a valid and low cost tool designed to measure social capital within larger surveys in Peru and Vietnam, with the potential for use in other developing countries following local piloting and cultural adaptation of the tool.

  3. Pain assessment for people with dementia: a systematic review of systematic reviews of pain assessment tools.

    PubMed

    Lichtner, Valentina; Dowding, Dawn; Esterhuizen, Philip; Closs, S José; Long, Andrew F; Corbett, Anne; Briggs, Michelle

    2014-12-17

    There is evidence of under-detection and poor management of pain in patients with dementia, in both long-term and acute care. Accurate assessment of pain in people with dementia is challenging and pain assessment tools have received considerable attention over the years, with an increasing number of tools made available. Systematic reviews on the evidence of their validity and utility mostly compare different sets of tools. This review of systematic reviews analyses and summarises evidence concerning the psychometric properties and clinical utility of pain assessment tools in adults with dementia or cognitive impairment. We searched for systematic reviews of pain assessment tools providing evidence of reliability, validity and clinical utility. Two reviewers independently assessed each review and extracted data from them, with a third reviewer mediating when consensus was not reached. Analysis of the data was carried out collaboratively. The reviews were synthesised using a narrative synthesis approach. We retrieved 441 potentially eligible reviews, 23 met the criteria for inclusion and 8 provided data for extraction. Each review evaluated between 8 and 13 tools, in aggregate providing evidence on a total of 28 tools. The quality of the reviews varied and the reporting often lacked sufficient methodological detail for quality assessment. The 28 tools appear to have been studied in a variety of settings and with varied types of patients. The reviews identified several methodological limitations across the original studies. The lack of a 'gold standard' significantly hinders the evaluation of tools' validity. Most importantly, the samples were small providing limited evidence for use of any of the tools across settings or populations. There are a considerable number of pain assessment tools available for use with the elderly cognitive impaired population. However there is limited evidence about their reliability, validity and clinical utility. On the basis of this

  4. Tool for Evaluating the Ways Nurses Assess Pain (TENAP): psychometric properties assessment.

    PubMed

    Ng, Siok Qi; Brammer, Jillian; Creedy, Debra K; Klainin-Yobas, Piyanee

    2014-12-01

    Elderly people with cognitive impairment are at risk for under-treatment of pain due to their inability to communicate. Poor knowledge and attitudes of nurses toward pain in this population may result in inadequate pain assessment. This study used a descriptive correlational design to develop and validate a tool to assess nurses' knowledge, attitudes, and reported practice of pain assessment in cognitively impaired elderly patients in acute care settings. The Tool for Evaluating the ways Nurses Assess Pain (TENAP) has two sections: (1) nurses' knowledge and attitudes about pain assessment and management and (2) two vignettes to assess reported practice. Content validity was established by an expert panel of three geriatric-trained nurse clinicians, and pilot tested with a convenience sample of 10 nurses. The psychometric properties were tested with a sample of 263 Registered and Enrolled nurses working in medical wards of two public hospitals in Singapore. The final version of TENAP comprised 29 items. Content validity index ranged from 0.84 to 1.00. The scale took 10 to 15 minutes to complete and items were easily understood. Results from the factor analysis suggested that Section A demonstrated one factor (13 items) while Section B had two distinct factors (16 items), one for each vignette, supporting construct validity of the scale. Cronbach's alphas for all factors were acceptable. TENAP was feasible, valid, and reliable for assessing nurses' knowledge, attitudes, and reported practice of pain assessment in cognitively-impaired elderly patients. Further testing of the tool with a larger sample of nurses in other practice contexts is needed. Copyright © 2014 American Society for Pain Management Nursing. Published by Elsevier Inc. All rights reserved.

  5. Development and validation of a collaborative behaviors objective assessment tool for end-of-life communication.

    PubMed

    Blackhall, Leslie J; Erickson, Jeanne; Brashers, Valentina; Owen, John; Thomas, Shannon

    2014-01-01

    Helping families make end-of-life decisions requires close collaboration between physicians and nurses. However, medical and nursing students have little formal training in how to collaborate in this task, and few instruments are available to measure collaborative behaviors. The objective of this project was to develop and validate observational assessment tools to measure specific interprofessional competencies in medical and nursing students related to end-of-life discussions. A literature search for evidence-based guidelines and competencies and focus groups with an expert panel of nurses and physicians were used to outline best collaborative practice behaviors for nurses and physicians in an end-of-life decision making simulation. The panel used these practice-behavior checklists to rate videotaped student scenarios and then refined the checklists for validity and clarity until the tools had acceptable inter-rater reliability. The setting was a workshop teaching end-of-life communication to third-year nursing and medical students. Inter-rater reliability was measured using percent agreement and kappa; internal consistency was measured using Cronbach's alpha. Collaborative behaviors objective assessment tools (CBOATs) for nursing and medical students were developed. For the medical CBOAT we found 85% agreement between raters, with an overall kappa of 0.744 and Cronbach's alpha of 0.806. For the nursing CBOAT there was 81% agreement, with a kappa of 0.686 and Cronbach's alpha of 0.845. Development of an end-of-life CBOAT clarified the important collaborative behaviors needed by physician and nurse. The resulting instruments provide a helpful guide for teaching interprofessional sessions related to the end of life and measuring student outcomes using an objective strategy.

  6. Validation of Social Cognition Rating Tools in Indian Setting (SOCRATIS): A new test-battery to assess social cognition.

    PubMed

    Mehta, Urvakhsh M; Thirthalli, Jagadisha; Naveen Kumar, C; Mahadevaiah, Mahesh; Rao, Kiran; Subbakrishna, Doddaballapura K; Gangadhar, Bangalore N; Keshavan, Matcheri S

    2011-09-01

    Social cognition is a cognitive domain that is under substantial cultural influence. There are no culturally appropriate standardized tools in India to comprehensively test social cognition. This study describes validation of tools for three social cognition constructs: theory of mind, social perception and attributional bias. Theory of mind tests included adaptations of, (a) two first order tasks [Sally-Anne and Smarties task], (b) two second order tasks [Ice cream van and Missing cookies story], (c) two metaphor-irony tasks and (d) the faux pas recognition test. Internal, Personal, and Situational Attributions Questionnaire (IPSAQ) and Social Cue Recognition Test were adapted to assess attributional bias and social perception, respectively. These tests were first modified to suit the Indian cultural context without changing the constructs to be tested. A panel of experts then rated the tests on likert scales as to (1) whether the modified tasks tested the same construct as in the original and (2) whether they were culturally appropriate. The modified tests were then administered to groups of actively symptomatic and remitted schizophrenia patients as well as healthy comparison subjects. All tests of the Social Cognition Rating Tools in Indian Setting had good content validity and known groups validity. In addition, the social cure recognition test in Indian setting had good internal consistency and concurrent validity. Copyright © 2011 Elsevier B.V. All rights reserved.

  7. Nutrition Environment Food Pantry Assessment Tool (NEFPAT): Development and Evaluation.

    PubMed

    Nikolaus, Cassandra J; Laurent, Emily; Loehmer, Emily; An, Ruopeng; Khan, Naiman; McCaffrey, Jennifer

    2018-04-24

    To develop and evaluate a nutrition environment assessment tool to assess the consumer nutrition environment and use of recommended practices in food pantries. The Nutrition Environment Food Pantry Assessment Tool (NEFPAT) was developed based on a literature review and guidance from professionals working with food pantries. The tool was pilot-tested at 9 food pantries, an expert panel assessed content validity, and interrater reliability was evaluated by pairs in 3 pantries. After revisions, the NEFPAT was used in 27 pantries. Pilot tests indicated positive appraisal for the NEFPAT and recommendations were addressed. The NEFPAT's 6 objectives and the overall tool were rated as content valid by experts, with an average section rating of 3.85 ± 0.10. Intraclass correlation coefficients for interrater reliability were >0.90. The NEFPAT is content valid with high interrater reliability. It provides baseline data that could be valuable for interventions within the nutrition environment of food pantries. Published by Elsevier Inc.

  8. Comparison of seven fall risk assessment tools in community-dwelling Korean older women.

    PubMed

    Kim, Taekyoung; Xiong, Shuping

    2017-03-01

    This study aimed to compare seven widely used fall risk assessment tools in terms of validity and practicality, and to provide a guideline for choosing appropriate fall risk assessment tools for elderly Koreans. Sixty community-dwelling Korean older women (30 fallers and 30 matched non-fallers) were evaluated. Performance measures of all tools were compared between the faller and non-faller groups through two sample t-tests. Receiver Operating Characteristic curves were generated with odds ratios for discriminant analysis. Results showed that four tools had significant discriminative power, and the shortened version of Falls Efficacy Scale (SFES) showed excellent discriminant validity, followed by Berg Balance Scale (BBS) with acceptable discriminant validity. The Mini Balance Evaluation System Test and Timed Up and Go, however, had limited discriminant validities. In terms of practicality, SFES was also excellent. These findings suggest that SFES is the most suitable tool for assessing the fall risks of community-dwelling Korean older women, followed by BBS. Practitioner Summary: There is no general guideline on which fall risk assessment tools are suitable for community-dwelling Korean older women. This study compared seven widely used assessment tools in terms of validity and practicality. Results suggested that the short Falls Efficacy Scale is the most suitable tool, followed by Berg Balance Scale.

  9. Evaluation of the validity of osteoporosis and fracture risk assessment tools (IOF One Minute Test, SCORE, and FRAX) in postmenopausal Palestinian women.

    PubMed

    Kharroubi, Akram; Saba, Elias; Ghannam, Ibrahim; Darwish, Hisham

    2017-12-01

    The need for simple self-assessment tools is necessary to predict women at high risk for developing osteoporosis. In this study, tools like the IOF One Minute Test, Fracture Risk Assessment Tool (FRAX), and Simple Calculated Osteoporosis Risk Estimation (SCORE) were found to be valid for Palestinian women. The threshold for predicting women at risk for each tool was estimated. The purpose of this study is to evaluate the validity of the updated IOF (International Osteoporosis Foundation) One Minute Osteoporosis Risk Assessment Test, FRAX, SCORE as well as age alone to detect the risk of developing osteoporosis in postmenopausal Palestinian women. Three hundred eighty-two women 45 years and older were recruited including 131 women with osteoporosis and 251 controls following bone mineral density (BMD) measurement, 287 completed questionnaires of the different risk assessment tools. Receiver operating characteristic (ROC) curves were evaluated for each tool using bone BMD as the gold standard for osteoporosis. The area under the ROC curve (AUC) was the highest for FRAX calculated with BMD for predicting hip fractures (0.897) followed by FRAX for major fractures (0.826) with cut-off values ˃1.5 and ˃7.8%, respectively. The IOF One Minute Test AUC (0.629) was the lowest compared to other tested tools but with sufficient accuracy for predicting the risk of developing osteoporosis with a cut-off value ˃4 total yes questions out of 18. SCORE test and age alone were also as good predictors of risk for developing osteoporosis. According to the ROC curve for age, women ≥64 years had a higher risk of developing osteoporosis. Higher percentage of women with low BMD (T-score ≤-1.5) or osteoporosis (T-score ≤-2.5) was found among women who were not exposed to the sun, who had menopause before the age of 45 years, or had lower body mass index (BMI) compared to controls. Women who often fall had lower BMI and approximately 27% of the recruited postmenopausal

  10. The predictive validity of common risk assessment tools in men with intellectual disabilities and problematic sexual behaviors.

    PubMed

    Fedoroff, J Paul; Richards, Deborah; Ranger, Rebekah; Curry, Susan

    2016-10-01

    This CIHR-funded study examined whether certain current risk assessment tools were effective in appraising risk of recidivism in a sample of sex offenders with intellectual disabilities (ID). Fifty men with ID who had engaged in problematic sexual behavior (PSB) were followed for an average of 2.5 years. Recidivism was defined and measured as any illegal or problematic behavior, as well as any problematic but not necessarily illegal behavior. At the beginning of the study, each participant was rated on two risk assessment tools: the Violence Risk Appraisal Guide (VRAG) and the Sex Offender Risk Appraisal Guide (SORAG). During each month of follow-up, participants were also rated on the Short-Dynamic Risk Scale (SDRS), an assessment tool intended to measure the risk of future problematic behaviors. Data was analyzed using t-tests, Cohen's d and area under the curve (AUC) to test predictive validity of the assessment tools. Using the AUC, results showed that the VRAG was predictive of sexual (AUC=0.74), sexual and/or violent (AUC=0.71) and of any criminally chargeable event (AUC=0.69). The SORAG was only significantly predictive of sexual events (AUC=0.70) and the SDRS was predictive of violent events (AUC=0.71). The t-test and Cohen's d analyses, which are less robust to deviations from the assumptions of normal and continuous distribution than AUC, did not yield significant results in each category, and therefore, while the results of this study suggest that the VRAG and the SORAG may be effective tools in measuring the short term risk of sexual recidivism; and the VRAG and SDRS may be effective tools in appraising long term risk of sexual and/or violent recidivism in this population, it should be used with caution. Regardless of the assessment tool used, risk assessments should take into account the differences between sex offenders with and without ID to ensure effective measurement. Copyright © 2016. Published by Elsevier Ltd.

  11. Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools.

    PubMed

    Griesinger, Claudius; Desprez, Bertrand; Coecke, Sandra; Casey, Warren; Zuang, Valérie

    This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the

  12. The virtual reality simulator dV-Trainer(®) is a valid assessment tool for robotic surgical skills.

    PubMed

    Perrenot, Cyril; Perez, Manuela; Tran, Nguyen; Jehl, Jean-Philippe; Felblinger, Jacques; Bresler, Laurent; Hubert, Jacques

    2012-09-01

    Exponential development of minimally invasive techniques, such as robotic-assisted devices, raises the question of how to assess robotic surgery skills. Early development of virtual simulators has provided efficient tools for laparoscopic skills certification based on objective scoring, high availability, and lower cost. However, similar evaluation is lacking for robotic training. The purpose of this study was to assess several criteria, such as reliability, face, content, construct, and concurrent validity of a new virtual robotic surgery simulator. This prospective study was conducted from December 2009 to April 2010 using three simulators dV-Trainers(®) (MIMIC Technologies(®)) and one Da Vinci S(®) (Intuitive Surgical(®)). Seventy-five subjects, divided into five groups according to their initial surgical training, were evaluated based on five representative exercises of robotic specific skills: 3D perception, clutching, visual force feedback, EndoWrist(®) manipulation, and camera control. Analysis was extracted from (1) questionnaires (realism and interest), (2) automatically generated data from simulators, and (3) subjective scoring by two experts of depersonalized videos of similar exercises with robot. Face and content validity were generally considered high (77 %). Five levels of ability were clearly identified by the simulator (ANOVA; p = 0.0024). There was a strong correlation between automatic data from dV-Trainer and subjective evaluation with robot (r = 0.822). Reliability of scoring was high (r = 0.851). The most relevant criteria were time and economy of motion. The most relevant exercises were Pick and Place and Ring and Rail. The dV-Trainer(®) simulator proves to be a valid tool to assess basic skills of robotic surgery.

  13. Development of Reliable and Validated Tools to Evaluate Technical Resuscitation Skills in a Pediatric Simulation Setting: Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics.

    PubMed

    Faudeux, Camille; Tran, Antoine; Dupont, Audrey; Desmontils, Jonathan; Montaudié, Isabelle; Bréaud, Jean; Braun, Marc; Fournier, Jean-Paul; Bérard, Etienne; Berlengi, Noémie; Schweitzer, Cyril; Haas, Hervé; Caci, Hervé; Gatin, Amélie; Giovannini-Chami, Lisa

    2017-09-01

    To develop a reliable and validated tool to evaluate technical resuscitation skills in a pediatric simulation setting. Four Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics (RESCAPE) evaluation tools were created, following international guidelines: intraosseous needle insertion, bag mask ventilation, endotracheal intubation, and cardiac massage. We applied a modified Delphi methodology evaluation to binary rating items. Reliability was assessed comparing the ratings of 2 observers (1 in real time and 1 after a video-recorded review). The tools were assessed for content, construct, and criterion validity, and for sensitivity to change. Inter-rater reliability, evaluated with Cohen kappa coefficients, was perfect or near-perfect (>0.8) for 92.5% of items and each Cronbach alpha coefficient was ≥0.91. Principal component analyses showed that all 4 tools were unidimensional. Significant increases in median scores with increasing levels of medical expertise were demonstrated for RESCAPE-intraosseous needle insertion (P = .0002), RESCAPE-bag mask ventilation (P = .0002), RESCAPE-endotracheal intubation (P = .0001), and RESCAPE-cardiac massage (P = .0037). Significantly increased median scores over time were also demonstrated during a simulation-based educational program. RESCAPE tools are reliable and validated tools for the evaluation of technical resuscitation skills in pediatric settings during simulation-based educational programs. They might also be used for medical practice performance evaluations. Copyright © 2017 Elsevier Inc. All rights reserved.

  14. Methodological challenges of validating a clinical decision-making tool in the practice environment.

    PubMed

    Brennan, Caitlin W; Daly, Barbara J

    2015-04-01

    Validating a measurement tool intended for use in the practice environment poses challenges that may not be present when validating a tool intended solely for research purposes. The aim of this article is to describe the methodological challenges of validating a clinical decision-making tool, the Oncology Acuity Tool, which nurses use to make nurse assignment and staffing decisions prospectively each shift. Data were derived from a larger validation study, during which several methodological challenges arose. Revisions to the tool, including conducting iterative feedback cycles with end users, were necessary before the validation study was initiated. The "true" value of patient acuity is unknown, and thus, two approaches to inter-rater reliability assessment were used. Discordant perspectives existed between experts and end users. Balancing psychometric rigor with clinical relevance may be achieved through establishing research-practice partnerships, seeking active and continuous feedback with end users, and weighing traditional statistical rules of thumb with practical considerations. © The Author(s) 2014.

  15. Validation of vocational assessment tool for persons with substance use disorders.

    PubMed

    Sethuraman, Lakshmanan; Subodh, B N; Murthy, Pratima

    2016-01-01

    Work-related problems are a serious concern among persons with substance use but due to lack of a standardized tool to measure it; these problems are neither systematically assessed nor appropriately addressed. Most existing measures of work performance cater to the needs of the workplace rather than focusing on the workers' perception of the difficulties at work. To develop a standardized instrument to measure work-related problems in persons with substance use disorders. Qualitative data obtained from interviews with substance users were used to develop a scale. The refined list of the scale was circulated among an expert panel for content validation. The modified scale was administered to 150 cases, and 50 cases completed the scale twice at the interval of 2 weeks for test-retest reliability. Items with a test-retest reliability kappa coefficient of 0.4 or greater were retained and subjected to factor analysis. The final 45-item scale has a five-factor structure. The value of Cronbach's alpha of the final version of the scale was 0.91. This self-report questionnaire, which can be completed in 10 min, may help us in making a baseline assessment of the work-related impairment among persons with substance use and the impact of substance use on work.

  16. The letter knowledge assessment tool.

    PubMed

    Pedro, Cassandra; Lousada, Marisa; Pereira, Rita; Hall, Andreia; Jesus, Luis M T

    2017-10-10

    There is a need to develop letter knowledge assessment tools to characterise the letter knowledge in Portuguese pre-schoolers and to compare it with pre-schoolers from other countries, but there are no tools for this purpose in Portugal. The aim of this paper is to describe the development and validation procedures of the Prova de Avaliação de Competências de Pré-Literacia (PACPL), which assesses letter knowledge. This study includes data that has been gathered in two phases: pilot and main study. In the pilot study, an expert panel of six speech and language pathologists analysed the instrument. Children (n = 216) aged 5;0-7;11 participated in the main study that reports data related to the psychometric characteristics of the PACPL. Content validity, internal consistency, reliability and contributing factors to performance were examined statistically. A modified Bland-Altman method revealed good agreement amongst evaluators. The main study showed that the PACPL has a very good internal consistency and high inter-rater (96.2% of agreement and a Cohen's k value of 0.92) and intra-rater (95.6% of agreement and a Cohen's k value of 0.91) agreement. Construct validity of the PCAPL was also assured (Cronbach's α of 0.982). Significant differences were found between age groups with children increasing their letter knowledge with age. In addition, they were better at identifying than at producing both letter names and letter sounds. The PACPL is a valid and reliable instrument to assess letter knowledge in Portuguese children.

  17. A scoping review of assessment tools for laparoscopic suturing.

    PubMed

    Bilgic, Elif; Endo, Satoshi; Lebedeva, Ekaterina; Takao, Madoka; McKendy, Katherine M; Watanabe, Yusuke; Feldman, Liane S; Vassiliou, Melina C

    2018-05-03

    A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions. Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills. Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences. Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.

  18. Assessing reliability and validity of the GroPromo audit tool for evaluation of grocery store marketing and promotional environments.

    PubMed

    Kerr, Jacqueline; Sallis, James F; Bromby, Erica; Glanz, Karen

    2012-01-01

    To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Trained observers used the GroPromo instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity, customers' receipts were coded for percentage of food purchases in each of the categories. Of the 22 categories tested, 21 demonstrated moderate to high interrater reliability (intraclass correlation ≥ 0.61). When more unhealthy items were placed in prominent locations, a higher percentage of money was spent on less-healthy items, and a lower percentage of food dollars were spent on fruits and vegetables. The prominence of locations was more important than the number of locations. The GroPromo tool can be used to assess promotional practices in stores. Data may help advocates campaign for more healthy food items in key promotional locations. Copyright © 2012 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

  19. Children's Physical Activity While Gardening: Development of a Valid and Reliable Direct Observation Tool.

    PubMed

    Myers, Beth M; Wells, Nancy M

    2015-04-01

    Gardens are a promising intervention to promote physical activity (PA) and foster health. However, because of the unique characteristics of gardening, no extant tool can capture PA, postures, and motions that take place in a garden. The Physical Activity Research and Assessment tool for Garden Observation (PARAGON) was developed to assess children's PA levels, tasks, postures, and motions, associations, and interactions while gardening. PARAGON uses momentary time sampling in which a trained observer watches a focal child for 15 seconds and then records behavior for 15 seconds. Sixty-five children (38 girls, 27 boys) at 4 elementary schools in New York State were observed over 8 days. During the observation, children simultaneously wore Actigraph GT3X+ accelerometers. The overall interrater reliability was 88% agreement, and Ebel was .97. Percent agreement values for activity level (93%), garden tasks (93%), motions (80%), associations (95%), and interactions (91%) also met acceptable criteria. Validity was established by previously validated PA codes and by expected convergent validity with accelerometry. PARAGON is a valid and reliable observation tool for assessing children's PA in the context of gardening.

  20. The THINC-Integrated Tool (THINC-it) Screening Assessment for Cognitive Dysfunction: Validation in Patients With Major Depressive Disorder.

    PubMed

    McIntyre, Roger S; Best, Michael W; Bowie, Christopher R; Carmona, Nicole E; Cha, Danielle S; Lee, Yena; Subramaniapillai, Mehala; Mansur, Rodrigo B; Barry, Harry; Baune, Bernhard T; Culpepper, Larry; Fossati, Philippe; Greer, Tracy L; Harmer, Catherine; Klag, Esther; Lam, Raymond W; Wittchen, Hans-Ulrich; Harrison, John

    2017-07-01

    To validate the THINC-integrated tool (THINC-it)-a freely available, patient-administered, computerized screening tool integrating subjective and objective measures of cognitive function in adults with major depressive disorder (MDD). Subjects aged 18 to 65 years (n = 100) with recurrent MDD experiencing a major depressive episode of at least moderate severity were evaluated and compared to age-, sex-, and education-matched healthy controls (n = 100). Between January and June 2016, subjects completed the THINC-it, which includes variants of the Choice Reaction Time Identification Task (IDN), One-Back Test, Digit Symbol Substitution Test, Trail Making Test-Part B, and the Perceived Deficits Questionnaire for Depression-5-item (PDQ-5-D). The THINC-it required approximately 10 to 15 minutes for administration and was capable of detecting cognitive deficits in adults with MDD. A total of 44.4% of adults with MDD exhibited cognitive performance at ≥ 1.0 SD below that of healthy controls on standardized mean scores of the THINC-it. Concurrent validity of the overall tool, based on a calculated composite score, was acceptable (r = 0.539, P < .001). Concurrent validity of the component tests ranged from -0.083 (IDN) to 0.929 (PDQ-5-D). Qualitative survey results indicated that there was a high level of satisfaction and perceived value in administering the THINC-it regarding its impact on the appropriateness and quality of care being received. The THINC-it is a valid and sensitive tool for detecting cognitive dysfunction in adults with MDD that is free, easy to use, and rapidly administered. The THINC-it should be incorporated into the assessment and measurement of all patients with MDD, particularly among those with enduring functional impairment. ClinicalTrials.gov identifier: NCT02508493. © Copyright 2017 Physicians Postgraduate Press, Inc.

  1. Development and validation of an observation tool for the assessment of nursing pain management practices in intensive care unit in a standardized clinical simulation setting.

    PubMed

    Gosselin, Emilie; Bourgault, Patricia; Lavoie, Stephan; Coleman, Robin-Marie; Méziat-Burdin, Anne

    2014-12-01

    Pain management in the intensive care unit is often inadequate. There is no tool available to assess nursing pain management practices. The aim of this study was to develop and validate a measuring tool to assess nursing pain management in the intensive care unit during standardized clinical simulation. A literature review was performed to identify relevant components demonstrating optimal pain management in adult intensive care units and to integrate them in an observation tool. This tool was submitted to an expert panel and pretested. It was then used to assess pain management practice during 26 discrete standardized clinical simulation sessions with intensive care nurses. The Nursing Observation Tool for Pain Management (NOTPaM) contains 28 statements grouped into 8 categories, which are grouped into 4 dimensions: subjective assessment, objective assessment, interventions, and reassessment. The tool's internal consistency was calculated at a Cronbach's alpha of 0.436 for the whole tool; the alpha varies from 0.328 to 0.518 for each dimension. To evaluate the inter-rater reliability, intra-class correlation coefficient was used, which was calculated at 0.751 (p < .001) for the whole tool, with variations from 0.619 to 0.920 (p < .01) between dimensions. The expert panel was satisfied with the content and face validity of the tool. The psychometric qualities of the NOTPaM developed in this study are satisfactory. However, the tool could be improved with slight modifications. Nevertheless, it was useful in assessing intensive care nurses' pain management in a standardized clinical simulation. The NOTPaM is the first tool created for this purpose. Copyright © 2014 American Society for Pain Management Nursing. Published by Elsevier Inc. All rights reserved.

  2. Validation of the educational needs assessment tool as a generic instrument for rheumatic diseases in seven European countries.

    PubMed

    Ndosi, Mwidimi; Bremander, Ann; Hamnes, Bente; Horton, Mike; Kukkurainen, Marja Leena; Machado, Pedro; Marques, Andrea; Meesters, Jorit; Stamm, Tanja A; Tennant, Alan; de la Torre-Aboki, Jenny; Vliet Vlieland, Theodora P M; Zangi, Heidi A; Hill, Jackie

    2014-12-01

    To validate the educational needs assessment tool (ENAT) as a generic tool for assessing the educational needs of patients with rheumatic diseases in European Countries. A convenience sample of patients from seven European countries was included comprising the following diagnostic groups: ankylosing spondylitis, psoriatic arthritis, systemic sclerosis, systemic lupus erythematosus, osteoarthritis (OA) and fibromyalgia syndrome. Translated versions of the ENAT were completed through surveys in each country. Rasch analysis was used to assess the construct validity of the adapted ENATs including differential item functioning by culture (cross-cultural DIF). Initially, the data from each country and diagnostic group were fitted to the Rasch model separately, and then the pooled data from each diagnostic group. The sample comprised 3015 patients; the majority, 1996 (66.2%), were women. Patient characteristics (stratified by diagnostic group) were comparable across countries except the educational background, which was variable. In most occasions, the 39-item ENAT deviated significantly from the Rasch model expectations (item-trait interaction χ(2) p<0.05). After correction for local dependency (grouping the items into seven domains and analysing them as 'testlets'), fit to the model was satisfied (item-trait interaction χ(2) p>0.18) in all pooled disease group datasets except OA (χ(2)=99.91; p=0.002). The internal consistency in each group was high (Person Separation Index above 0.90). There was no significant DIF by person characteristics. Cross-cultural DIF was found in some items, which required adjustments. Subsequently, interval-level scales were calibrated to enable transformation of ENAT scores when required. The adapted ENAT is a valid tool with high internal consistency providing accurate estimation of the educational needs of people with rheumatic diseases. Cross-cultural comparison of educational needs is now possible. Published by the BMJ Publishing Group

  3. The Democritos Movement Screening Tool for Preschool Children (DEMOST-PRE©): development and factorial validity.

    PubMed

    Kambas, Antonis; Venetsanou, Fotini

    2014-07-01

    The aim of this study was (a) to develop an assessment tool (the Democritos Movement Screening Tool for Preschool Children - DEMOST-PRE), designed to provide preschool educators, clinicians and researchers with information about assessment and screening of the motor proficiency of children aged 4-6 years, as well as the development and control of movement programmes and (b) to assess its factorial validity. First, tool's content and face validity were established and its final structure was determined. Then, the DEMOST-PRE was administered to 435 children (197 girls) aged 48-71 months (M=60.48 months, SD=6.98). The factor analysis conducted revealed two distinct components. Present evidence combined with the DEMOST-PRE administrative traits make it promising for preschool aged children's assessment. Copyright © 2014 Elsevier Ltd. All rights reserved.

  4. Evaluation of Knowla: An Online Assessment and Learning Tool

    ERIC Educational Resources Information Center

    Thompson, Meredith Myra; Braude, Eric John

    2016-01-01

    The assessment of learning in large online courses requires tools that are valid, reliable, easy to administer, and can be automatically scored. We have evaluated an online assessment and learning tool called Knowledge Assembly, or Knowla. Knowla measures a student's knowledge in a particular subject by having the student assemble a set of…

  5. Validity evidence as a key marker of quality of technical skill assessment in OTL-HNS.

    PubMed

    Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

    2018-01-13

    Quality monitoring of assessment practices should be a priority in all residency programs. Validity evidence is one of the main hallmarks of assessment quality and should be collected to support the interpretation and use of assessment data. Our objective was to identify, synthesize, and present the validity evidence reported supporting different technical skill assessment tools in otolaryngology-head and neck surgery (OTL-HNS). We performed a secondary analysis of data generated through a systematic review of all published tools for assessing technical skills in OTL-HNS (n = 16). For each tool, we coded validity evidence according to the five types of evidence described by the American Educational Research Association's interpretation of Messick's validity framework. Descriptive statistical analyses were conducted. All 16 tools included in our analysis were supported by internal structure and relationship to variables validity evidence. Eleven articles presented evidence supporting content. Response process was discussed only in one article, and no study reported on evidence exploring consequences. We present the validity evidence reported for 16 rater-based tools that could be used for work-based assessment of OTL-HNS residents in the operating room. The articles included in our review were consistently deficient in evidence for response process and consequences. Rater-based assessment tools that support high-stakes decisions that impact the learner and programs should include several sources of validity evidence. Thus, use of any assessment should be done with careful consideration of the context-specific validity evidence supporting score interpretation, and we encourage deliberate continual assessment quality-monitoring. NA. Laryngoscope, 2018. © 2018 The American Laryngological, Rhinological and Otological Society, Inc.

  6. Assessing functional mobility in survivors of lower-extremity sarcoma: reliability and validity of a new assessment tool.

    PubMed

    Marchese, Victoria G; Rai, Shesh N; Carlson, Claire A; Hinds, Pamela S; Spearing, Elena M; Zhang, Lijun; Callaway, Lulie; Neel, Michael D; Rao, Bhaskar N; Ginsberg, Jill P

    2007-08-01

    Reliability and validity of a new tool, Functional Mobility Assessment (FMA), were examined in patients with lower-extremity sarcoma. FMA requires the patients to physically perform the functional mobility measures, unlike patient self-report or clinician administered measures. A sample of 114 subjects participated, 20 healthy volunteers and 94 patients with lower-extremity sarcoma after amputation, limb-sparing, or rotationplasty surgery. Reliability of the FMA was examined by three raters testing 20 healthy volunteers and 23 subjects with lower-extremity sarcoma. Concurrent validity was examined using data from 94 subjects with lower-extremity sarcoma who completed the FMA, Musculoskeletal Tumor Society (MSTS), Short-Form 36 (SF-36v2), and Toronto Extremity Salvage Scale (TESS) scores. Construct validity was measured by the ability of the FMA to discriminate between subjects with and without functional mobility deficits. FMA demonstrated excellent reliability (ICC [2,1] >or=0.97). Moderate correlations were found between FMA and SF-36v2 (r = 0.60, P < 0.01), FMA and MSTS (r = 0.68, P < 0.01), and FMA and TESS (r = 0.62, P < 0.01). The patients with lower-extremity sarcoma scored lower on the FMA as compared to healthy controls (P < 0.01). The FMA is a reliable and valid functional outcome measure for patients with lower-extremity sarcoma. This study supports the ability of the FMA to discriminate between patients with varying functional abilities and supports the need to include measures of objective functional mobility in examination of patients with lower-extremity sarcoma.

  7. Validation of vocational assessment tool for persons with substance use disorders

    PubMed Central

    Sethuraman, Lakshmanan; Subodh, B. N.; Murthy, Pratima

    2016-01-01

    Background: Work-related problems are a serious concern among persons with substance use but due to lack of a standardized tool to measure it; these problems are neither systematically assessed nor appropriately addressed. Most existing measures of work performance cater to the needs of the workplace rather than focusing on the workers' perception of the difficulties at work. Aim: To develop a standardized instrument to measure work-related problems in persons with substance use disorders. Methods: Qualitative data obtained from interviews with substance users were used to develop a scale. The refined list of the scale was circulated among an expert panel for content validation. The modified scale was administered to 150 cases, and 50 cases completed the scale twice at the interval of 2 weeks for test–retest reliability. Results: Items with a test–retest reliability kappa coefficient of 0.4 or greater were retained and subjected to factor analysis. The final 45-item scale has a five-factor structure. The value of Cronbach's alpha of the final version of the scale was 0.91. Conclusions: This self-report questionnaire, which can be completed in 10 min, may help us in making a baseline assessment of the work-related impairment among persons with substance use and the impact of substance use on work. PMID:28163409

  8. Validation of an organizational communication climate assessment toolkit.

    PubMed

    Wynia, Matthew K; Johnson, Megan; McCoy, Thomas P; Griffin, Leah Passmore; Osborn, Chandra Y

    2010-01-01

    Effective communication is critical to providing quality health care and can be affected by a number of modifiable organizational factors. The authors performed a prospective multisite validation study of an organizational communication climate assessment tool in 13 geographically and ethnically diverse health care organizations. Communication climate was measured across 9 discrete domains. Patient and staff surveys with matched items in each domain were developed using a national consensus process, which then underwent psychometric field testing and assessment of domain coherence. The authors found meaningful within-site and between-site performance score variability in all domains. In multivariable models, most communication domains were significant predictors of patient-reported quality of care and trust. The authors conclude that these assessment tools provide a valid empirical assessment of organizational communication climate in 9 domains. Assessment results may be useful to track organizational performance, to benchmark, and to inform tailored quality improvement interventions.

  9. Predictive validity of the post-enrolment English language assessment tool for commencing undergraduate nursing students.

    PubMed

    Glew, Paul J; Hillege, Sharon P; Salamonson, Yenna; Dixon, Kathleen; Good, Anthony; Lombardo, Lien

    2015-12-01

    Nursing students with English as an additional language (EAL) may underperform academically. The post-enrolment English language assessment (PELA) is used in literacy support, but its predictive validity in identifying those at risk of underperformance remains unknown. To validate a PELA, as a predictor of academic performance. Prospective survey design. The study was conducted at a university located in culturally and linguistically diverse areas of western Sydney, Australia. Commencing undergraduate nursing students who were Australian-born (n=1323, 49.6%) and born outside of Australia (n=1346, 50.4%) were recruited for this study. The 2669 (67% of 3957) participants provided consent and completed a first year nursing unit that focussed on developing literacy skills. Between 2010 and 2013, commencing students completed the PELA and English language acculturation scale (ELAS), a previously validated instrument. The grading levels of the PELA tool were: Level 1 (proficient), Level 2 (borderline), and Level 3 (poor, and requiring additional support). Participants with a PELA Level 2 or 3 were more likely to be: a) non-Australian-born (χ(2): 520.6, df: 2, p<0.001); b) spoke a language other than English at home (χ(2): 490.2, df: 2, p<0.001); and c) an international student (χ(2): 225.6, df: 2, p<0.001). There was an inverse relationship between participants' ELAS scores and PELA levels (r=-0.52, p<0.001), and those graded as 'proficient' with a PELA Level 1 were more likely to obtain higher scores in their: i) unit essay assessment (χ(2): 40.2, df: 2, p<0.001); ii) final unit mark (χ(2): 218.6, df: 2, p<0.001), and attain a higher GPA (χ(2): 100.8, df: 2, p<0.001). The PELA is a useful screening tool in identifying commencing nursing students who are at risk of academic underachievement. Crown Copyright © 2015. Published by Elsevier Ltd. All rights reserved.

  10. School-based behavioral assessment tools are reliable and valid for measurement of fruit and vegetable intake, physical activity, and television viewing in young children.

    PubMed

    Economos, Christina D; Sacheck, Jennifer M; Kwan Ho Chui, Kenneth; Irizarry, Laura; Irizzary, Laura; Guillemont, Juliette; Collins, Jessica J; Hyatt, Raymond R

    2008-04-01

    Interventions aiming to modify the dietary and physical activity behaviors of young children require precise and accurate measurement tools. As part of a larger community-based project, three school-based questionnaires were developed to assess (a) fruit and vegetable intake, (b) physical activity and television (TV) viewing, and (c) perceived parental support for diet and physical activity. Test-retest reliability was performed on all questionnaires and validity was measured for fruit and vegetable intake, physical activity, and TV viewing. Eighty-four school children (8.3+/-1.1 years) were studied. Test-retest reliability was performed by administering questionnaires twice, 1 to 2 hours apart. Validity of the fruit and vegetable questionnaire was measured by direct observation, while the physical activity and TV questionnaire was validated by a parent phone interview. All three questionnaires yielded excellent test-retest reliability (P<0.001). The majority of fruit and vegetable questions and the questions regarding specific physical activities and TV viewing were valid. Low validity scores were found for questions on watching TV during breakfast or dinner. These questionnaires are reliable and valid tools to assess fruit and vegetable intake, physical activity, and TV viewing behaviors in early elementary school-aged children. Methods for assessment of children's TV viewing during meals should be further investigated because of parent-child discrepancies.

  11. Screening and assessment tools for pediatric malnutrition.

    PubMed

    Huysentruyt, Koen; Vandenplas, Yvan; De Schepper, Jean

    2016-06-18

    The ideal measures for screening and assessing undernutrition in children remain a point of discussion in literature. This review aims to provide an overview of recent advances in the nutritional screening and assessment methods in children. This review focuses on two major topics that emerged in literature since 2015: the practical endorsement of the new definition for pediatric undernutrition, with a focus on anthropometric measurements and the search for a consensus on pediatric nutritional screening tools in different settings. Few analytical tools exist for the assessment of the nutritional status in children. The subjective global nutritional assessment has been validated by anthropometric as well as clinical outcome parameters. Nutritional screening can help in selecting patients that benefit the most from a full nutritional assessment. Two new screening tools have been developed for use in a general (mixed) hospital population, and one for a population of children with cancer. The value of screening tools in different disease-specific and outpatient pediatric populations remains to be proven.

  12. Validation of a 4-item Negative Symptom Assessment (NSA-4): a short, practical clinical tool for the assessment of negative symptoms in schizophrenia.

    PubMed

    Alphs, Larry; Morlock, Robert; Coon, Cheryl; Cazorla, Pilar; Szegedi, Armin; Panagides, John

    2011-06-01

    The 16-item Negative Symptom Assessment (NSA-16) scale is a validated tool for evaluating negative symptoms of schizophrenia. The psychometric properties and predictive power of a four-item version (NSA-4) were compared with the NSA-16. Baseline data from 561 patients with predominant negative symptoms of schizophrenia who participated in two identically designed clinical trials were evaluated. Ordered logistic regression analysis of ratings using NSA-4 and NSA-16 were compared with ratings using several other standard tools to determine predictive validity and construct validity. Internal consistency and test--retest reliability were also analyzed. NSA-16 and NSA-4 scores were both predictive of scores on the NSA global rating (odds ratio = 0.83-0.86) and the Clinical Global Impressions--Severity scale (odds ratio = 0.91-0.93). NSA-16 and NSA-4 showed high correlation with each other (Pearson r = 0.85), similar high correlation with other measures of negative symptoms (demonstrating convergent validity), and lesser correlations with measures of other forms of psychopathology (demonstrating divergent validity). NSA-16 and NSA-4 both showed acceptable internal consistency (Cronbach α, 0.85 and 0.64, respectively) and test--retest reliability (intraclass correlation coefficient, 0.87 and 0.82). This study demonstrates that NSA-4 offers accuracy comparable to the NSA-16 in rating negative symptoms in patients with schizophrenia. Copyright © 2011 John Wiley & Sons, Ltd.

  13. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.

    PubMed

    Shea, Beverley J; Grimshaw, Jeremy M; Wells, George A; Boers, Maarten; Andersson, Neil; Hamel, Candyce; Porter, Ashley C; Tugwell, Peter; Moher, David; Bouter, Lex M

    2007-02-15

    Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus. A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity. The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity. A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.

  14. The Development, Validation, and User Evaluation of Foodbook24: A Web-Based Dietary Assessment Tool Developed for the Irish Adult Population

    PubMed Central

    McNulty, Breige; Kehoe, Laura; Evans, Katie; Walton, Janette; Flynn, Albert

    2017-01-01

    Background The application of technology in the area of dietary assessment has resulted in the development of an array of tools, which are often specifically designed for a particular country or region. Objective The aim of this study was to describe the development, validation, and user evaluation of a Web-based dietary assessment tool “Foodbook24.” Methods Foodbook24 is a Web-based, dietary assessment tool consisting of a 24-hour dietary recall (24HDR) and food frequency questionnaire (FFQ) alongside supplementary questionnaires. Validity of the 24HDR component was assessed by 40 participants, who completed 3 nonconsecutive, self-administered 24HDR using Foodbook24 and a 4-day semi-weighed food diary at separate time points. Participants also provided fasted blood samples and 24-hour urine collections for the identification of biomarkers of nutrient and food group intake during each recording period. Statistical analyses on the nutrient and food group intake data derived from each method were performed in SPSS version 20.0 (SPSS Inc). Mean nutrient intakes (and standard deviations) recorded using each method of dietary assessment were calculated. Spearman and Pearson correlations, Wilcoxon Signed Rank and Paired t test were used to investigate the agreement and differences between the nutritional output from Foodbook24 (test method) and the 4-day semi-weighed food diary (reference method). Urinary and plasma biomarkers of nutrient intake were used as an objective validation of Foodbook24. To investigate the user acceptability of Foodbook24, participants from different studies involved with Foodbook24 were asked to complete an evaluation questionnaire. Results For nutrient intake, correlations between the dietary assessment methods were acceptable to very good in strength and statistically significant (range r=.32 to .75). There were some significant differences between reported mean intakes of micronutrients recorded by both methods; however, with the

  15. Validation of a measurement tool to assess awareness of breast cancer.

    PubMed

    Linsell, Louise; Forbes, Lindsay J L; Burgess, Caroline; Kapari, Marcia; Thurnham, Angela; Ramirez, Amanda J

    2010-05-01

    Until now, there has been no universally accepted and validated measure of breast cancer awareness. This study aimed to validate the new Breast Cancer Awareness Measure (BCAM) which assesses, using a self-complete questionnaire, knowledge of breast cancer symptoms and age-related risk, and frequency of breast checking. We measured the psychometric properties of the BCAM in 1035 women attending the NHS Breast Screening Programme: acceptability was assessed using a feedback questionnaire (n=292); sensitivity to change after an intervention promoting breast cancer awareness (n=576), and test-retest reliability (n=167). We also assessed readability, and construct validity using the 'known-groups' method. The readability of the BCAM was high. Over 90% of women found it acceptable. The BCAM was sensitive to change: there was an increase in the proportion of women obtaining the full score for breast cancer awareness one month after receiving the intervention promoting breast cancer awareness; this was greater among those who received a more intensive version (less intensive version (booklet): 9.3%, 95% confidence interval (CI): 4.5-14.1%; more intensive version (interaction with health professional plus booklet): 30%, 95% CI: 23.4-36.6%). Test-retest reliability of the BCAM was moderate to good for most items. Cancer experts had higher levels of cancer awareness than non-medical academics (50% versus 6%, p=0.001), indicating good construct validity. The BCAM is a valid and robust measure of breast cancer awareness suitable for use in surveys of breast cancer awareness in the general population and to evaluate the impact of awareness-raising interventions. Copyright (c) 2010 Elsevier Ltd. All rights reserved.

  16. Content Validity of a Tool Measuring Medication Errors.

    PubMed

    Tabassum, Nishat; Allana, Saleema; Saeed, Tanveer; Dias, Jacqueline Maria

    2015-08-01

    The objective of this study was to determine content and face validity of a tool measuring medication errors among nursing students in baccalaureate nursing education. Data was collected from the Aga Khan University School of Nursing and Midwifery (AKUSoNaM), Karachi, from March to August 2014. The tool was developed utilizing literature and the expertise of the team members, expert in different areas. The developed tool was then sent to five experts from all over Karachi for ensuring the content validity of the tool, which was measured on relevance and clarity of the questions. The Scale Content Validity Index (S-CVI) for clarity and relevance of the questions was found to be 0.94 and 0.98, respectively. The tool measuring medication errors has an excellent content validity. This tool should be used for future studies on medication errors, with different study populations such as medical students, doctors, and nurses.

  17. The risk of bias in systematic reviews tool showed fair reliability and good construct validity.

    PubMed

    Bühn, Stefanie; Mathes, Tim; Prengel, Peggy; Wegewitz, Uta; Ostermann, Thomas; Robens, Sibylle; Pieper, Dawid

    2017-11-01

    There is a movement from generic quality checklists toward a more domain-based approach in critical appraisal tools. This study aimed to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR), that is, the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability, and applicability. Validation study with four reviewers based on 16 systematic reviews in the field of occupational health. Interrater reliability (IRR) of all four raters was highest for domain 2 (Fleiss' kappa κ = 0.56) and lowest for domain 4 (κ = 0.04). For ROBIS, median IRR was κ = 0.52 (range 0.13-0.88) for the experienced pair of raters compared to κ = 0.32 (range 0.12-0.76) for the less experienced pair of raters. The percentage of "yes" scores of each review of ROBIS ratings was strongly correlated with the AMSTAR ratings (r s  = 0.76; P = 0.01). ROBIS has fair reliability and good construct validity to assess the risk of bias in systematic reviews. More validation studies are needed to investigate reliability and applicability, in particular. Copyright © 2017 Elsevier Inc. All rights reserved.

  18. The validity of a professional competence tool for physiotherapy students in simulation-based clinical education: a Rasch analysis.

    PubMed

    Judd, Belinda K; Scanlan, Justin N; Alison, Jennifer A; Waters, Donna; Gordon, Christopher J

    2016-08-05

    Despite the recent widespread adoption of simulation in clinical education in physiotherapy, there is a lack of validated tools for assessment in this setting. The Assessment of Physiotherapy Practice (APP) is a comprehensive tool used in clinical placement settings in Australia to measure professional competence of physiotherapy students. The aim of the study was to evaluate the validity of the APP for student assessment in simulation settings. A total of 1260 APPs were collected, 971 from students in simulation and 289 from students in clinical placements. Rasch analysis was used to examine the construct validity of the APP tool in three different simulation assessment formats: longitudinal assessment over 1 week of simulation; longitudinal assessment over 2 weeks; and a short-form (25 min) assessment of a single simulation scenario. Comparison with APPs from 5 week clinical placements in hospital and clinic-based settings were also conducted. The APP demonstrated acceptable fit to the expectations of the Rasch model for the 1 and 2 week clinical simulations, exhibiting unidimensional properties that were able to distinguish different levels of student performance. For the short-form simulation, nine of the 20 items recorded greater than 25 % of scores as 'not-assessed' by clinical educators which impacted on the suitability of the APP tool in this simulation format. The APP was a valid assessment tool when used in longitudinal simulation formats. A revised APP may be required for assessment in short-form simulation scenarios.

  19. A tool for assessing the quality of nursing handovers: a validation study.

    PubMed

    Ferrara, Paolo; Terzoni, Stefano; Davì, Salvatore; Bisesti, Alberto; Destrebecq, Anne

    2017-08-10

    Handover, in particular between two shifts, is a crucial aspect of nursing for patient safety, aimed at ensuring continuity of care. During this process, several factors can affect quality of care and cause errors. This study aimed to assess quality of handovers, by validating the Handoff CEX-Italian scale. The scale was translated from English into Italian and the content validity index was calculated and internal consistency assessed. The scale was used in several units of the San Paolo Teaching Hospital in Milan, Italy. A total of 48 reports were assessed (192 evaluations). The median score was 6, interquartile range (IQR) [5;7] and was not influenced by specific (p=0.21) or overall working experience (p=0.13). The domains showing the lowest median values (median=6, IQR [4;8]) were context, communication, and organisation. Night to morning handovers obtained the lowest scores. CVI-S was 0.96, Cronbach's alpha was 0.79. The Handoff CEX-Italian scale is valid and reliable and it can be used to assess the quality of nurse handovers.

  20. A Rapid Assessment Tool for affirming good practice in midwifery education programming.

    PubMed

    Fullerton, Judith T; Johnson, Peter; Lobe, Erika; Myint, Khine Haymar; Aung, Nan Nan; Moe, Thida; Linn, Nay Aung

    2016-03-01

    to design a criterion-referenced assessment tool that could be used globally in a rapid assessment of good practices and bottlenecks in midwifery education programs. a standard tool development process was followed, to generate standards and reference criteria; followed by external review and field testing to document psychometric properties. review of standards and scoring criteria were conducted by stakeholders around the globe. Field testing of the tool was conducted in Myanmar. eleven of Myanmar׳s 22 midwifery education programs participated in the assessment. the clinimetric tool was demonstrated to have content validity and high inter-rater reliability in use. a globally validated tool, and accompanying user guide and handbook are now available for conducting rapid assessments of compliance with good practice criteria in midwifery education programming. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

  1. Development and psychometric testing of Holistic Clinical Assessment Tool (HCAT) for undergraduate nursing students.

    PubMed

    Wu, Xi Vivien; Enskär, Karin; Pua, Lay Hoon; Heng, Doreen Gek Noi; Wang, Wenru

    2016-09-22

    A major focus in nursing education is on the judgement of clinical performance, and it is a complex process due to the diverse nature of nursing practice. A holistic approach in assessment of competency is advocated. Difficulties in the development of valid and reliable assessment measures in nursing competency have resulted in the development of assessment instruments with an increase in face and content validity, but few studies have tested these instruments psychometrically. It is essential to develop a holistic assessment tool to meet the needs of the clinical education. The study aims to develop a Holistic Clinical Assessment Tool (HCAT) and test its psychometric properties. The HCAT was developed based on the systematic literature review and the findings of qualitative studies. An expert panel was invited to evaluate the content validity of the tool. A total of 130 final-year nursing undergraduate students were recruited to evaluate the psychometric properties (i.e. factor structure, internal consistency and test-retest reliability) of the tool. The HCAT has good content validity with content validity index of .979. The exploratory factor analysis reveals a four-factor structure of the tool. The internal consistency and test-retest reliability of the HCAT are satisfactory with Cronbach alpha ranging from .789 to .965 and Intraclass Correlation Coefficient ranging from .881 to .979 for the four subscales and total scale. HCAT has the potential to be used as a valid measure to evaluate clinical competence in nursing students, and provide specific and ongoing feedback to enhance the holistic clinical learning experience. In addition, HCAT functions as a tool for self-reflection, peer-assessment and guides preceptors in clinical teaching and assessment.

  2. Content validation of an interprofessional learning video peer assessment tool.

    PubMed

    Nisbet, Gillian; Jorm, Christine; Roberts, Chris; Gordon, Christopher J; Chen, Timothy F

    2017-12-16

    Large scale models of interprofessional learning (IPL) where outcomes are assessed are rare within health professional curricula. To date, there is sparse research describing robust assessment strategies to support such activities. We describe the development of an IPL assessment task based on peer rating of a student generated video evidencing collaborative interprofessional practice. We provide content validation evidence of an assessment rubric in the context of large scale IPL. Two established approaches to scale development in an educational setting were combined. A literature review was undertaken to develop a conceptual model of the relevant domains and issues pertaining to assessment of student generated videos within IPL. Starting with a prototype rubric developed from the literature, a series of staff and student workshops were undertaken to integrate expert opinion and user perspectives. Participants assessed five-minute videos produced in a prior pilot IPL activity. Outcomes from each workshop informed the next version of the rubric until agreement was reached on anchoring statements and criteria. At this point the rubric was declared fit to be used in the upcoming mandatory large scale IPL activity. The assessment rubric consisted of four domains: patient issues, interprofessional negotiation; interprofessional management plan in action; and effective use of video medium to engage audience. The first three domains reflected topic content relevant to the underlying construct of interprofessional collaborative practice. The fourth domain was consistent with the broader video assessment literature calling for greater emphasis on creativity in education. We have provided evidence for the content validity of a video-based peer assessment task portraying interprofessional collaborative practice in the context of large-scale IPL activities for healthcare professional students. Further research is needed to establish the reliability of such a scale.

  3. Development of a Multi-Domain Assessment Tool for Quality Improvement Projects.

    PubMed

    Rosenbluth, Glenn; Burman, Natalie J; Ranji, Sumant R; Boscardin, Christy K

    2017-08-01

    Improving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education. We developed and validated a tool to assess QI projects with learner engagement across the educational continuum. After reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool. The resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance. The MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels.

  4. Older adult mistreatment risk screening: contribution to the validation of a screening tool in a domestic setting.

    PubMed

    Lindenbach, Jeannette M; Larocque, Sylvie; Lavoie, Anne-Marise; Garceau, Marie-Luce

    2012-06-01

    ABSTRACTThe hidden nature of older adult mistreatment renders its detection in the domestic setting particularly challenging. A validated screening instrument that can provide a systematic assessment of risk factors can facilitate this detection. One such instrument, the "expanded Indicators of Abuse" tool, has been previously validated in the Hebrew language in a hospital setting. The present study has contributed to the validation of the "e-IOA" in an English-speaking community setting in Ontario, Canada. It consisted of two phases: (a) a content validity review and adaptation of the instrument by experts throughout Ontario, and (b) an inter-rater reliability assessment by home visiting nurses. The adaptation, the "Mistreatment of Older Adult Risk Factors" tool, offers a comprehensive tool for screening in the home setting. This instrument is significant to professional practice as practitioners working with older adults will be better equipped to assess for risk of mistreatment.

  5. Validation of a novel venous duplex ultrasound objective structured assessment of technical skills for the assessment of venous reflux.

    PubMed

    Jaffer, Usman; Normahani, Pasha; Lackenby, Kimberly; Aslam, Mohammed; Standfield, Nigel J

    2015-01-01

    Duplex ultrasound measurement of reflux time is central to the diagnosis of venous incompetence. We have developed an assessment tool for Duplex measurement of venous reflux for both simulator and patient-based training. A novel assessment tool, Venous Duplex Ultrasound Assessment of Technical Skills (V-DUOSATS), was developed. A modified DUOSATS was used for simulator training. Participants of varying skill level were invited to viewed an instructional video and were allowed ample time to familiarize with the Duplex equipment. Attempts made by the participants were recorded and independently assessed by 3 expert assessors and 5 novice assessors using the modified V-DUOSATS. "Global" assessment was also done by expert assessors on a 4-point Likert scale. Content, construct, and concurrent validities as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. Receiver operator characteristic analysis-established cut points of 19/22 and 21/30 were most appropriate for simulator and patient-based assessment, respectively. We have validated a novel assessment tool for Duplex venous reflux measurement. Further work is required to establish transference validity of simulator training to improve skill in scanning patients. We have developed and validated V-DUOSATS for simulator training. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

  6. Investigating the Validity of Two Widely Used Quantitative Text Tools

    ERIC Educational Resources Information Center

    Cunningham, James W.; Hiebert, Elfrieda H.; Mesmer, Heidi Anne

    2018-01-01

    In recent years, readability formulas have gained new prominence as a basis for selecting texts for learning and assessment. Variables that quantitative tools count (e.g., word frequency, sentence length) provide valid measures of text complexity insofar as they accurately predict representative and high-quality criteria. The longstanding…

  7. The Shoulder Objective Practical Assessment Tool: Evaluation of a New Tool Assessing Residents Learning in Diagnostic Shoulder Arthroscopy.

    PubMed

    Talbot, Christopher L; Holt, Edward M; Gooding, Benjamin W T; Tennent, Thomas D; Foden, Philip

    2015-08-01

    To design and validate an objective practical assessment tool for diagnostic shoulder arthroscopy that would provide residents with a method to evaluate their progression in this field of surgery and to identify specific learning needs. We designed and evaluated the shoulder Objective Practical Assessment Tool (OPAT). The shoulder OPAT was designed by us, and scoring domains were created using a Delphi process. The shoulder OPAT was trialed by members of the British Elbow & Shoulder Society Education Committee for internal consistency and ease of use before being offered to other trainers and residents. Inter-rater reliability and intrarater reliability were calculated. One hundred forty orthopaedic residents, of varying seniority, within 5 training regions in the United Kingdom, were questioned regarding the tool. A pilot study of 6 residents was undertaken. Internal consistency was 0.77 (standardized Cronbach α). Inter-rater reliability was 0.60, and intrarater reliability was 0.82. The Spearman correlation coefficient (r) between the global summary score for the shoulder OPAT and the current assessment tool used in postgraduate training for orthopaedic residents undertaking diagnostic shoulder arthroscopy equaled 0.74. Of the residents, 82% agreed or strongly agreed when asked if the shoulder OPAT would be a useful tool in monitoring progression and 72% agreed or strongly agreed with the introduction of the shoulder OPAT within the orthopaedic domain. This study shows that the shoulder OPAT fulfills several aspects of reliability and validity when tested. Despite the inter-rater reliability being 0.60, we believe that the shoulder OPAT has the potential to play a role alongside the current assessment tool in the training of orthopaedic residents. The shoulder OPAT can be used to assess residents during shoulder arthroscopy and has the potential for use in medical education, as well as arthroscopic skills training in the operating theater. Copyright © 2015

  8. Different type 2 diabetes risk assessments predict dissimilar numbers at 'high risk': a retrospective analysis of diabetes risk-assessment tools.

    PubMed

    Gray, Benjamin J; Bracken, Richard M; Turner, Daniel; Morgan, Kerry; Thomas, Michael; Williams, Sally P; Williams, Meurig; Rice, Sam; Stephens, Jeffrey W

    2015-12-01

    Use of a validated risk-assessment tool to identify individuals at high risk of developing type 2 diabetes is currently recommended. It is under-reported, however, whether a different risk tool alters the predicted risk of an individual. This study explored any differences between commonly used validated risk-assessment tools for type 2 diabetes. Cross-sectional analysis of individuals who participated in a workplace-based risk assessment in Carmarthenshire, South Wales. Retrospective analysis of 676 individuals (389 females and 287 males) who participated in a workplace-based diabetes risk-assessment initiative. Ten-year risk of type 2 diabetes was predicted using the validated QDiabetes(®), Leicester Risk Assessment (LRA), FINDRISC, and Cambridge Risk Score (CRS) algorithms. Differences between the risk-assessment tools were apparent following retrospective analysis of individuals. CRS categorised the highest proportion (13.6%) of individuals at 'high risk' followed by FINDRISC (6.6%), QDiabetes (6.1%), and, finally, the LRA was the most conservative risk tool (3.1%). Following further analysis by sex, over one-quarter of males were categorised at high risk using CRS (25.4%), whereas a greater percentage of females were categorised as high risk using FINDRISC (7.8%). The adoption of a different valid risk-assessment tool can alter the predicted risk of an individual and caution should be used to identify those individuals who really are at high risk of type 2 diabetes. © British Journal of General Practice 2015.

  9. Refining and validating a two-stage and web-based cancer risk assessment tool for village doctors in China.

    PubMed

    Shen, Xing-Rong; Chai, Jing; Feng, Rui; Liu, Tong-Zhu; Tong, Gui-Xian; Cheng, Jing; Li, Kai-Chun; Xie, Shao-Yu; Shi, Yong; Wang, De-Bin

    2014-01-01

    The big gap between efficacy of population level prevention and expectations due to heterogeneity and complexity of cancer etiologic factors calls for selective yet personalized interventions based on effective risk assessment. This paper documents our research protocol aimed at refining and validating a two-stage and web- based cancer risk assessment tool, from a tentative one in use by an ongoing project, capable of identifying individuals at elevated risk for one or more types of the 80% leading cancers in rural China with adequate sensitivity and specificity and featuring low cost, easy application and cultural and technical sensitivity for farmers and village doctors. The protocol adopted a modified population-based case control design using 72, 000 non-patients as controls, 2, 200 cancer patients as cases, and another 600 patients as cases for external validation. Factors taken into account comprised 8 domains including diet and nutrition, risk behaviors, family history, precancerous diseases, related medical procedures, exposure to environment hazards, mood and feelings, physical activities and anthropologic and biologic factors. Modeling stresses explored various methodologies like empirical analysis, logistic regression, neuro-network analysis, decision theory and both internal and external validation using concordance statistics, predictive values, etc..

  10. Implementation of a digital preparation validation tool in dental skills laboratory training.

    PubMed

    Kozarovska, A; Larsson, C

    2018-05-01

    To describe the implementation of a digital tool for preparation validation and evaluate it as an aid in students' self-assessment. Students at the final semester of skills laboratory training were asked to use a digital preparation validation tool (PVT) when performing two different tasks; preparation of crowns for teeth 11 and 21. The students were divided into two groups. Group A self-assessed and scanned all three attempts at 21 ("prep-and-scan"). Group B self-assessed all attempts chose the best one and scanned it ("best-of-three"). The situation was reversed for 11. The students assessed five parameters of the preparation and marked them as approved (A) or failed (F). These marks were compared with the information from the PVT. The students also completed a questionnaire. Each question was rated from 1 to 5. Teachers' opinions were collected at staff meetings throughout the project. Most students in the "prep-and-scan" groups showed an increase in agreement between their self-assessment and the information from the PVT, whereas students in the "best-of-three" groups showed lower levels of agreement. All students rated the PVT positively. Most strongly agreed that the tool was helpful in developing skills (mean 4.15), easy to use (mean 4.23) and that it added benefits in comparison to existing assessment tools (mean 4.05). They did not however, fully agree that the tool is time efficient (mean 2.55), and they did not consider it a substitute for verbal teacher feedback. Teachers' feedback suggested advantages of the tool in the form of ease of use, visual aid and increasing interest and motivation during skills laboratory training however, they did not notice a reduction in need of verbal feedback. Within the limitations of the study, our conclusion is that a digital PVT may be a valuable adjunct to other assessment tools in skills laboratory training. © 2017 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

  11. The development and validation of a meta-tool for quality appraisal of public health evidence: Meta Quality Appraisal Tool (MetaQAT).

    PubMed

    Rosella, L; Bowman, C; Pach, B; Morgan, S; Fitzpatrick, T; Goel, V

    2016-07-01

    Most quality appraisal tools were developed for clinical medicine and tend to be study-specific with a strong emphasis on risk of bias. In order to be more relevant to public health, an appropriate quality appraisal tool needs to be less reliant on the evidence hierarchy and consider practice applicability. Given the broad range of study designs used in public health, the objective of this study was to develop and validate a meta-tool that combines public health-focused principles of appraisal coupled with a set of design-specific companion tools. Several design methods were used to develop and validate the tool including literature review, synthesis, and validation with a reference standard. A search of critical appraisal tools relevant to public health was conducted; core concepts were collated. The resulting framework was piloted during three feedback sessions with public health practitioners. Following subsequent revisions, the final meta-tool, the Meta Quality Appraisal Tool (MetaQAT), was then validated through a content analysis of appraisals conducted by two groups of experienced public health researchers (MetaQAT vs generic appraisal form). The MetaQAT framework consists of four domains: relevancy, reliability, validity, and applicability. In addition, a companion tool was assembled from existing critical appraisal tools to provide study design-specific guidance on validity appraisal. Content analysis showed similar methodological and generalizability concerns were raised by both groups; however, the MetaQAT appraisers commented more extensively on applicability to public health practice. Critical appraisal tools designed for clinical medicine have limitations for use in the context of public health. The meta-tool structure of the MetaQAT allows for rigorous appraisal, while allowing users to simultaneously appraise the multitude of study designs relevant to public health research and assess non-standard domains, such as applicability. Copyright © 2015

  12. Development and Validity Testing of the Worksite Health Index: An Assessment Tool to Help and Improve Korean Employees' Health-Related Outcome.

    PubMed

    Yun, Young Ho; Sim, Jin Ah; Lim, Ye Jin; Lim, Cheol Il; Kang, Sung-Choon; Kang, Joon-Ho; Park, Jun Dong; Noh, Dong Young

    2016-06-01

    The objective of this study was to develop the Worksite Health Index (WHI) and validate its psychometric properties. The development of the WHI questionnaire included item generation, item construction, and field testing. To assess the instrument's reliability and validity, we recruited 30 different Korean worksites. We developed the WHI questionnaire of 136 items categorized into five domains, namely Governance and Infrastructure, Need Assessment and Planning, Health Prevention and Promotion Program, Occupational Safety, and Monitoring and Feedback. All WHI domains demonstrated a high reliability with good internal consistency. The total WHI scores differentiated worksite groups effectively according to firm size. Each domain was associated significantly with employees' health status, absence, and financial outcome. The WHI can assess comprehensive worksite health programs. This tool is publicly available for addressing the growing need for worksite health programs.

  13. The SAFE-T assessment tool: derivation and validation of a web-based application for point-of-care evaluation of gastroenterology fellow performance in colonoscopy.

    PubMed

    Kumar, Navin L; Kugener, Guillaume; Perencevich, Molly L; Saltzman, John R

    2018-01-01

    Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise assessment tool to evaluate real-time fellow performance in colonoscopy administered via a web-based application. The Skill Assessment in Fellow Endoscopy Training (SAFE-T) tool was derived as a novel 5-question evaluation tool that captures both summative and formative feedback adapted into a web-based application. A prospective study of 15 gastroenterology fellows (5 fellows each from years 1 to 3 of training) was performed using the SAFE-T tool. An independent reviewer evaluated a subset of these procedures and completed the SAFE-T tool and Mayo Colonoscopy Skills Assessment Tool (MCSAT) for reliability testing. Twenty-six faculty completed 350 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score (year 1, 2.00; year 2, 3.84; year 3, 4.28) differentiated each sequential fellow year of training (P < .0001). The mean SAFE-T overall score decreased with increasing case complexity score, with straightforward cases compared with average cases (4.07 vs 3.50, P < .0001), and average cases compared with challenging cases (3.50 vs 3.08, P = .0134). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a kappa agreement statistic of 0.898 (P < .0001). Correlation of the SAFE-T overall score with the MCSAT overall hands-on and individual motor scores was excellent (each r > 0.90, P < .0001). We developed and validated the SAFE-T assessment tool, a concise and web-based means of assessing real-time gastroenterology fellow performance in colonoscopy. Copyright © 2018 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

  14. The Myotonometer: Not a Valid Measurement Tool for Active Hamstring Musculotendinous Stiffness.

    PubMed

    Pamukoff, Derek N; Bell, Sarah E; Ryan, Eric D; Blackburn, J Troy

    2016-05-01

    Hamstring musculotendinous stiffness (MTS) is associated with lower-extremity injury risk (ie, hamstring strain, anterior cruciate ligament injury) and is commonly assessed using the damped oscillatory technique. However, despite a preponderance of studies that measure MTS reliably in laboratory settings, there are no valid clinical measurement tools. A valid clinical measurement technique is needed to assess MTS and permit identification of individuals at heightened risk of injury and track rehabilitation progress. To determine the validity and reliability of the Myotonometer for measuring active hamstring MTS. Descriptive laboratory study. Laboratory. 33 healthy participants (15 men, age 21.33 ± 2.94 y, height 172.03 ± 16.36 cm, mass 74.21 ± 16.36 kg). Hamstring MTS was assessed using the damped oscillatory technique and the Myotonometer. Intraclass correlations were used to determine the intrasession, intersession, and interrater reliability of the Myotonometer. Criterion validity was assessed via Pearson product-moment correlation between MTS measures obtained from the Myotonometer and from the damped oscillatory technique. The Myotonometer demonstrated good intrasession (ICC3,1 = .807) and interrater reliability (ICC2,k = .830) and moderate intersession reliability (ICC2,k = .693). However, it did not provide a valid measurement of MTS compared with the damped oscillatory technique (r = .346, P = .061). The Myotonometer does not provide a valid measure of active hamstring MTS. Although the Myotonometer does not measure active MTS, it possesses good reliability and portability and could be used clinically to measure tissue compliance, muscle tone, or spasticity associated with multiple musculoskeletal disorders. Future research should focus on portable and clinically applicable tools to measure active hamstring MTS in efforts to prevent and monitor injuries.

  15. Teamwork Assessment Tools in Modern Surgical Practice: A Systematic Review

    PubMed Central

    Whittaker, George; Abboudi, Hamid; Khan, Muhammed Shamim; Dasgupta, Prokar; Ahmed, Kamran

    2015-01-01

    Introduction. Deficiencies in teamwork skills have been shown to contribute to the occurrence of adverse events during surgery. Consequently, several teamwork assessment tools have been developed to evaluate trainee nontechnical performance. This paper aims to provide an overview of these instruments and review the validity of each tool. Furthermore, the present paper aims to review the deficiencies surrounding training and propose several recommendations to address these issues. Methods. A systematic literature search was conducted to identify teamwork assessment tools using MEDLINE (1946 to August 2015), EMBASE (1974 to August 2015), and PsycINFO (1806 to August 2015) databases. Results. Eight assessment tools which encompass aspects of teamwork were identified. The Nontechnical Skills for Surgeons (NOTSS) assessment was found to possess the highest level of validity from a variety of sources; reliability and acceptability have also been established for this tool. Conclusions. Deficits in current surgical training pathways have prompted several recommendations to meet the evolving requirements of surgeons. Recommendations from the current paper include integration of teamwork training and assessment into medical school curricula, standardised formal training of assessors to ensure accurate evaluation of nontechnical skill acquisition, and integration of concurrent technical and nontechnical skills training throughout training. PMID:26425732

  16. Validation of the Individualized Numeric Rating Scale (INRS): a pain assessment tool for nonverbal children with intellectual disability.

    PubMed

    Solodiuk, Jean C; Scott-Sutherland, Jennifer; Meyers, Margie; Myette, Beth; Shusterman, Christine; Karian, Victoria E; Harris, Sion Kim; Curley, Martha A Q

    2010-08-01

    Clinical observations suggest that nonverbal children with severe intellectual disability exhibit pain in a wide variety yet uniquely individual ways. Here, we investigate the feasibility and describe the initial psychometrics properties of the Individualized Numeric Rating Scale (INRS), a personalized pain assessment tool for nonverbal children with intellectual disability based on the parent's knowledge of the child. Parents of 50 nonverbal children with severe intellectual disability scheduled for surgery were able to complete the task of describing then rank ordering their child's usual and pain indicators. The parent, bedside nurse and research assistant (RA) triad then simultaneously yet independently scored the patient's post-operative pain using the INRS for a maximum of two sets of pre/post paired observations. A total of 170 triad assessments were completed before (n=85) and after (n=85) an intervention to manage the child's pain. INRS inter-rater agreement between the parents and research nurse was high (ICC 0.82-0.87) across all ratings. Parent and bedside nurse agreement (ICC 0.65-0.74) and bedside nurse and research nurse agreement (ICC 0.74-0.80) also suggest good reliability. A moderate to strong correlation (0.63-0.73) between INRS ratings and NCCPC-PV total scores provides evidence of convergent validity. These results provide preliminary data that the INRS is a valid and reliable tool for assessing pain in nonverbal children with severe intellectual disability in an acute care setting. Copyright (c) 2010 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.

  17. Evaluation of the Validity and Response Burden of Patient Self-Report Measures of the Pain Assessment Screening Tool and Outcomes Registry (PASTOR).

    PubMed

    Cook, Karon F; Kallen, Michael A; Buckenmaier, Chester; Flynn, Diane M; Hanling, Steven R; Collins, Teresa S; Joltes, Kristin; Kwon, Kyung; Medina-Torne, Sheila; Nahavandi, Parisa; Suen, Joshua; Gershon, Richard

    2017-07-01

    In 2009, the Army Pain Management Task Force was chartered. On the basis of their findings, the Department of Defense recommended a comprehensive pain management strategy that included development of a standardized pain assessment system that would collect patient-reported outcomes data to inform the patient-provider clinical encounter. The result was the Pain Assessment Screening Tool and Outcomes Registry (PASTOR). The purpose of this study was to assess the validity and response burden of the patient-reported outcome measures in PASTOR. Data for analyses were collected from 681 individuals who completed PASTOR at baseline and follow-up as part of their routine clinical care. The survey tool included self-report measures of pain severity and pain interference (measured using the National Institutes of Health Patient-Reported Outcome Measurement Information System [PROMIS] and the Defense and Veterans Pain Rating scale). PROMIS measures of pain correlates also were administered. Validation analyses included estimation of score associations among measures, comparison of scores of known groups, responsiveness, ceiling and floor effects, and response burden. Results of psychometric testing provided substantial evidence for the validity of PASTOR self-report measures in this population. Expected associations among scores largely supported the concurrent validity of the measures. Scores effectively distinguished among respondents on the basis of their self-reported impressions of general health. PROMIS measures were administered using computer adaptive testing and each, on average, required less than 1 minute to administer. Statistical and graphical analyses demonstrated the responsiveness of PASTOR measures over time. Reprint & Copyright © 2017 Association of Military Surgeons of the U.S.

  18. Construction and validation of a tool to Assess the Use of Light Technologies at Intensive Care Units.

    PubMed

    Marinho, Pabliane Matias Lordelo; Campos, Maria Pontes de Aguiar; Rodrigues, Eliana Ofélia Llapa; Gois, Cristiane Franca Lisboa; Barreto, Ikaro Daniel de Carvalho

    2016-12-19

    to construct and validate a tool to assess the use of light technologies by the nursing team at Intensive Care Units. methodological study in which the tool was elaborated by means of the psychometric method for construction based on the categorization of health technologies by Merhy and Franco, from the National Humanization Policy, using the Nursing Intervention Classification taxonomy to categorize the domains of the tool. Agreement Percentages and Content Validity Indices were used for the purpose of validation. The result of the application of the Interrater Agreement Percentage exceeded the recommended level of 80%, highlighting the relevance for the proposed theme in the assessment, with an agreement rate of 99%. the tool was validated with four domains (Bond, Autonomy, Welcoming and Management) and nineteen items that assess the use of light technologies at Intensive Care Units. construir e validar um instrumento para avaliação do uso de tecnologias leves, pela equipe de enfermagem, em Unidades de Terapia Intensiva. estudo metodológico no qual o instrumento foi elaborado utilizando o método psicométrico para construção com base na categorização das tecnologias em saúde de Merhy e Franco, da Política Nacional de Humanização, utilizando-se a taxonomia Nursing Intervention Classification para categorizar os domínios do instrumento. Utilizou-se o Percentual de Concordância e o Índice de Validade de Conteúdo (IVC) para validação. o resultado da aplicação do Percentual de Concordância entre os juízes foi superior ao recomendado de 80%, havendo destaque na avaliação da pertinência ao tema proposto, apresentando um percentual de concordância de 99%. o instrumento foi validado com quatro domínios (Vínculo, Autonomia, Acolhimento e Gestão) e dezenove itens que avaliam o uso das tecnologias leves em Unidade de Terapia Intensiva. construir y validar un instrumento para evaluación del uso de tecnologías leves, por el equipo de enfermer

  19. A Student Assessment Tool for Standardized Patient Simulations (SAT-SPS): Psychometric analysis.

    PubMed

    Castro-Yuste, Cristina; García-Cabanillas, María José; Rodríguez-Cornejo, María Jesús; Carnicer-Fuentes, Concepción; Paloma-Castro, Olga; Moreno-Corral, Luis Javier

    2018-05-01

    The evaluation of the level of clinical competence acquired by the student is a complex process that must meet various requirements to ensure its quality. The psychometric analysis of the data collected by the assessment tools used is a fundamental aspect to guarantee the student's competence level. To conduct a psychometric analysis of an instrument which assesses clinical competence in nursing students at simulation stations with standardized patients in OSCE-format tests. The construct of clinical competence was operationalized as a set of observable and measurable behaviors, measured by the newly-created Student Assessment Tool for Standardized Patient Simulations (SAT-SPS), which was comprised of 27 items. The categories assigned to the items were 'incorrect or not performed' (0), 'acceptable' (1), and 'correct' (2). 499 nursing students. Data were collected by two independent observers during the assessment of the students' performance at a four-station OSCE with standardized patients. Descriptive statistics were used to summarize the variables. The difficulty levels and floor and ceiling effects were determined for each item. Reliability was analyzed using internal consistency and inter-observer reliability. The validity analysis was performed considering face validity, content and construct validity (through exploratory factor analysis), and criterion validity. Internal reliability and inter-observer reliability were higher than 0.80. The construct validity analysis suggested a three-factor model accounting for 37.1% of the variance. These three factors were named 'Nursing process', 'Communication skills', and 'Safe practice'. A significant correlation was found between the scores obtained and the students' grades in general, as well as with the grades obtained in subjects with clinical content. The assessment tool has proven to be sufficiently reliable and valid for the assessment of the clinical competence of nursing students using standardized patients

  20. Different type 2 diabetes risk assessments predict dissimilar numbers at ‘high risk’: a retrospective analysis of diabetes risk-assessment tools

    PubMed Central

    Gray, Benjamin J; Bracken, Richard M; Turner, Daniel; Morgan, Kerry; Thomas, Michael; Williams, Sally P; Williams, Meurig; Rice, Sam; Stephens, Jeffrey W

    2015-01-01

    Background Use of a validated risk-assessment tool to identify individuals at high risk of developing type 2 diabetes is currently recommended. It is under-reported, however, whether a different risk tool alters the predicted risk of an individual. Aim This study explored any differences between commonly used validated risk-assessment tools for type 2 diabetes. Design and setting Cross-sectional analysis of individuals who participated in a workplace-based risk assessment in Carmarthenshire, South Wales. Method Retrospective analysis of 676 individuals (389 females and 287 males) who participated in a workplace-based diabetes risk-assessment initiative. Ten-year risk of type 2 diabetes was predicted using the validated QDiabetes®, Leicester Risk Assessment (LRA), FINDRISC, and Cambridge Risk Score (CRS) algorithms. Results Differences between the risk-assessment tools were apparent following retrospective analysis of individuals. CRS categorised the highest proportion (13.6%) of individuals at ‘high risk’ followed by FINDRISC (6.6%), QDiabetes (6.1%), and, finally, the LRA was the most conservative risk tool (3.1%). Following further analysis by sex, over one-quarter of males were categorised at high risk using CRS (25.4%), whereas a greater percentage of females were categorised as high risk using FINDRISC (7.8%). Conclusion The adoption of a different valid risk-assessment tool can alter the predicted risk of an individual and caution should be used to identify those individuals who really are at high risk of type 2 diabetes. PMID:26541180

  1. An Integrative Review of Pediatric Fall Risk Assessment Tools.

    PubMed

    DiGerolamo, Kimberly; Davis, Katherine Finn

    Patient fall prevention begins with accurate risk assessment. However, sustained improvements in prevention and quality of care include use of validated fall risk assessment tools (FRATs). The goal of FRATs is to identify patients at highest risk. Adult FRATs are often borrowed from to create tools for pediatric patients. Though factors associated with pediatric falls in the hospital setting are similar to those in adults, such as mobility, medication use, and cognitive impairment, adult FRATs and the factors associated with them do not adequately assess risk in children. Articles were limited to English language, ages 0-21years, and publish date 2006-2015. The search yielded 22 articles. Ten were excluded as the population was primarily adult or lacked discussion of a FRAT. Critical appraisal and findings were synthesized using the Johns Hopkins Nursing evidence appraisal system. Twelve articles relevant to fall prevention in the pediatric hospital setting that discussed fall risk assessment and use of a FRAT were reviewed. Comparison between and accuracy of FRATs is challenged when different classifications, definitions, risk stratification, and inclusion criteria are used. Though there are several pediatric FRATs published in the literature, none have been found to be reliable and valid across institutions and diverse populations. This integrative review highlights the importance of choosing a FRAT based on an institution's identified risk factors and validating the tool for one's own patient population as well as using the tool in conjunction with nursing clinical judgment to guide interventions. Copyright © 2017 Elsevier Inc. All rights reserved.

  2. A Systematic Review of Tools Used to Assess Team Leadership in Health Care Action Teams.

    PubMed

    Rosenman, Elizabeth D; Ilgen, Jonathan S; Shandro, Jamie R; Harper, Amy L; Fernandez, Rosemarie

    2015-10-01

    To summarize the characteristics of tools used to assess leadership in health care action (HCA) teams. HCA teams are interdisciplinary teams performing complex, critical tasks under high-pressure conditions. The authors conducted a systematic review of the PubMed/MEDLINE, CINAHL, ERIC, EMBASE, PsycINFO, and Web of Science databases, key journals, and review articles published through March 2012 for English-language articles that applied leadership assessment tools to HCA teams in all specialties. Pairs of reviewers assessed identified articles for inclusion and exclusion criteria and abstracted data on study characteristics, tool characteristics, and validity evidence. Of the 9,913 abstracts screened, 83 studies were included. They described 61 team leadership assessment tools. Forty-nine tools (80%) provided behaviors, skills, or characteristics to define leadership. Forty-four tools (72%) assessed leadership as one component of a larger assessment, 13 tools (21%) identified leadership as the primary focus of the assessment, and 4 (7%) assessed leadership style. Fifty-three studies (64%) assessed leadership at the team level; 29 (35%) did so at the individual level. Assessments of simulated (n = 55) and live (n = 30) patient care events were performed. Validity evidence included content validity (n = 75), internal structure (n = 61), relationship to other variables (n = 44), and response process (n = 15). Leadership assessment tools applied to HCA teams are heterogeneous in content and application. Comparisons between tools are limited by study variability. A systematic approach to team leadership tool development, evaluation, and implementation will strengthen understanding of this important competency.

  3. Pain assessment tools: is the content appropriate for use in palliative care?

    PubMed

    Hølen, Jacob Chr; Hjermstad, Marianne Jensen; Loge, Jon Håvard; Fayers, Peter M; Caraceni, Augusto; De Conno, Franco; Forbes, Karen; Fürst, Carl Johan; Radbruch, Lukas; Kaasa, Stein

    2006-12-01

    Inadequate pain assessment prevents optimal treatment in palliative care. The content of pain assessment tools might limit their usefulness for proper pain assessment, but data on the content validity of the tools are scarce. The objective of this study was to examine the content of the existing pain assessment tools, and to evaluate the appropriateness of different dimensions and items for pain assessment in palliative care. A systematic search was performed to find pain assessment tools for patients with advanced cancer who were receiving palliative care. An ad hoc search with broader search criteria supplemented the systematic search. The items of the identified tools were allocated to appropriate dimensions. This was reviewed by an international panel of experts, who also evaluated the relevance of the different dimensions for pain assessment in palliative care. The systematic literature search generated 16 assessment tools while the ad hoc search generated 64. Ten pain dimensions containing 1,011 pain items were identified by the experts. The experts ranked intensity, temporal pattern, treatment and exacerbating/relieving factors, location, and interference with health-related quality of life as the most important dimensions. None of the assessment tools covered these dimensions satisfactorily. Most items were related to interference (231) and intensity (138). Temporal pattern (which includes breakthrough pain), ranked as the second most important dimension, was covered by 29 items only. Many tools include dimensions and items of limited relevance for patients with advanced cancer. This might reduce compliance and threaten the validity of the assessment. New tools should reflect the clinical relevance of different dimensions and be user-friendly.

  4. Reliability and Validity of the Chinese Version of FACIT-AI, a New Tool for Assessing Quality of Life in Patients with Malignant Ascites.

    PubMed

    Lou, Yanni; Lu, Linghui; Li, Yuan; Liu, Meng; Bredle, Jason M; Jia, Liqun

    2015-10-01

    The study objective was to determine the reliability and validity of the Chinese version of the Functional Assessment of Chronic Illness Therapy - Ascites Index (FACIT-AI). A forward-backward translation procedure was adopted to develop the Chinese version of the FACIT-AI, which was tested in 69 patients with malignant ascites. Cronbach's α, split-half reliability, and test-retest reliability were used to assess the reliability of the scale. The content validity index was used to assess the content validity, while factor analysis was used for construct validity and correlation analysis was used for criterion validity. The Cronbach's α was 0.772 for the total scale, and the split-half reliability was 0.693. The test-retest correlation was 0.972. The content validity index for the scale was 0.8-1.0. Four factors were extracted by factor analysis, and these contributed 63.51% of the total variance. Item-total correlations ranged from 0.591 to 0.897, and these were correlated with visual analog scale scores (correlation coefficient, 0.889; P<0.01). The Chinese version of the FACIT-AI has good reliability and validity and can be used as a tool to measure quality of life in Chinese patients with malignant ascites.

  5. Development and validation of a tool to assess the physical and social environment associated with physical activity among adults in Sri Lanka.

    PubMed

    De Silva Weliange, Shreenika H; Fernando, Dulitha; Gunatilake, Jagath

    2014-05-03

    Environmental characteristics are known to be associated with patterns of physical activity (PA). Although several validated tools exist, to measure the environment characteristics, these instruments are not necessarily suitable for application in all settings especially in a developing country. This study was carried out to develop and validate an instrument named the "Physical And Social Environment Scale--PASES" to assess the physical and social environmental factors associated with PA. This will enable identification of various physical and social environmental factors affecting PA in Sri Lanka, which will help in the development of more tailored intervention strategies for promoting higher PA levels in Sri Lanka. The PASES was developed using a scientific approach of defining the construct, item generation, analysis of content of items and item reduction. Both qualitative and quantitative methods of key informant interviews, in-depth interviews and rating of the items generated by experts were conducted. A cross sectional survey among 180 adults was carried out to assess the factor structure through principal component analysis. Another cross sectional survey among a different group of 180 adults was carried out to assess the construct validity through confirmatory factor analysis. Reliability was assessed with test re-test reliability and internal consistency using Spearman r and Cronbach's alpha respectively. Thirty six items were selected after the expert ratings and were developed into interviewer administered questions. Exploration of factor structure of the 34 items which were factorable through principal component analysis with Quartimax rotation extracted 8 factors. The 34 item instrument was assessed for construct validity with confirmatory factor analysis which confirmed an 8 factor model (x2 = 339.9, GFI = 0.90). The identified factors were infrastructure for walking, aesthetics and facilities for cycling, vehicular traffic safety, access

  6. Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany.

    PubMed

    Dreier, Maren; Borutta, Birgit; Stahmeyer, Jona; Krauth, Christian; Walter, Ulla

    2010-06-14

    HEALTH CARE POLICY BACKGROUND: Findings from scientific studies form the basis for evidence-based health policy decisions. Quality assessments to evaluate the credibility of study results are an essential part of health technology assessment reports and systematic reviews. Quality assessment tools (QAT) for assessing the study quality examine to what extent study results are systematically distorted by confounding or bias (internal validity). The tools can be divided into checklists, scales and component ratings. What QAT are available to assess the quality of interventional studies or studies in the field of health economics, how do they differ from each other and what conclusions can be drawn from these results for quality assessments? A systematic search of relevant databases from 1988 onwards is done, supplemented by screening of the references, of the HTA reports of the German Agency for Health Technology Assessment (DAHTA) and an internet search. The selection of relevant literature, the data extraction and the quality assessment are carried out by two independent reviewers. The substantive elements of the QAT are extracted using a modified criteria list consisting of items and domains specific to randomized trials, observational studies, diagnostic studies, systematic reviews and health economic studies. Based on the number of covered items and domains, more and less comprehensive QAT are distinguished. In order to exchange experiences regarding problems in the practical application of tools, a workshop is hosted. A total of eight systematic methodological reviews is identified as well as 147 QAT: 15 for systematic reviews, 80 for randomized trials, 30 for observational studies, 17 for diagnostic studies and 22 for health economic studies. The tools vary considerably with regard to the content, the performance and quality of operationalisation. Some tools do not only include the items of internal validity but also the items of quality of reporting and

  7. The script concordance test in radiation oncology: validation study of a new tool to assess clinical reasoning

    PubMed Central

    Lambert, Carole; Gagnon, Robert; Nguyen, David; Charlin, Bernard

    2009-01-01

    Background The Script Concordance test (SCT) is a reliable and valid tool to evaluate clinical reasoning in complex situations where experts' opinions may be divided. Scores reflect the degree of concordance between the performance of examinees and that of a reference panel of experienced physicians. The purpose of this study is to demonstrate SCT's usefulness in radiation oncology. Methods A 90 items radiation oncology SCT was administered to 155 participants. Three levels of experience were tested: medical students (n = 70), radiation oncology residents (n = 38) and radiation oncologists (n = 47). Statistical tests were performed to assess reliability and to document validity. Results After item optimization, the test comprised 30 cases and 70 questions. Cronbach alpha was 0.90. Mean scores were 51.62 (± 8.19) for students, 71.20 (± 9.45) for residents and 76.67 (± 6.14) for radiation oncologists. The difference between the three groups was statistically significant when compared by the Kruskall-Wallis test (p < 0.001). Conclusion The SCT is reliable and useful to discriminate among participants according to their level of experience in radiation oncology. It appears as a useful tool to document the progression of reasoning during residency training. PMID:19203358

  8. Measuring pain in patients undergoing hemodialysis: a review of pain assessment tools

    PubMed Central

    Upadhyay, Chandani; Cameron, Karen; Murphy, Laura; Battistella, Marisa

    2014-01-01

    Background Patients undergoing hemodialysis frequently report pain with multifactorial causes, not limited to that experienced directly from hemodialysis treatment. Their pain may be nociceptive, neuropathic, somatic or visceral in nature. Despite this, pain in this population remains under-recognized and under-treated. Although several tools have been used to measure pain in patients undergoing hemodialysis as reported in the literature, none of them have been validated specifically in this population. The objective for this review was to compare and contrast these pain assessment tools and discuss their clinical utility in this patient population. Methods To identify pain assessment tools studied in patients undergoing hemodialysis, a literature search was performed in PubMed and Medline. An expert panel of dialysis and pain clinicians reviewed each tool. Each pain assessment tool was assessed on how it is administered and scored, its psychometric properties such as reliability, validity and responsiveness to change, and its clinical utility in a hemodialysis population. Brief Pain Inventory, McGill Pain Questionnaire, Pain Management Index, Edmonton Symptom Assessment System, Visual Analogue Scale and Faces Pain Scale were evaluated and compared. Results This assessment will help clinicians practicing in nephrology to determine which of these pain assessment tools is best suited for use in their individual clinical practice. PMID:25852910

  9. Part 1: Pressure ulcer assessment - the development of Critical Care Pressure Ulcer Assessment Tool made Easy (CALCULATE).

    PubMed

    Richardson, Annette; Barrow, Isabel

    2015-11-01

    Critically ill patients are at high risk of developing pressure ulcers resulting in serious untoward patient and health care system outcomes. Pressure ulcer prevention is therefore an important patient safety priority and establishing a structured approach to pressure ulcer risk assessment to identify patients at risk is a critical first step. The literature was searched using three electronic databases from 2000 to 2011 to identify papers reporting on pressure ulcer risk factors and assessment in adult critical care. The review and appraisal of papers were conducted by two critical care nurses. Papers underwent detailed review if they met inclusion criteria where they identified pressure ulcer assessment scores, scales or risk factors and related to adult critical care patients Seven papers were reviewed. No single assessment tool was sufficiently validated for critically ill patients and seven key critical care risk factors were identified. These risk factors were: mechanical ventilation, impaired circulation, dialysis, long surgery, low protein and too unstable to turn. The tool Critical Care Pressure Ulcer Assessment Tool made Easy (CALCULATE) was developed utilizing the risk factors from the literature and expert critical care nursing consensus decision-making. In the absence of current consensus, valid assessment scales and limited evidence for the most appropriate pressure ulcer assessment for critically ill patients, this assessment tool offers an easy, appropriate alternative for critically ill patients than existing tools primarily validated for acute care wards. 'CALCULATE' offers an important contribution towards the advancement and development of critical care pressure ulcer risk assessment. Future research is needed to further enhance and inform pressure ulcer risk assessment of the critically ill patients. The identification of critical care risk factors may be an indicative method of assessing pressure ulcer risk in the critically ill patients.

  10. Time Domain Tool Validation Using ARES I-X Flight Data

    NASA Technical Reports Server (NTRS)

    Hough, Steven; Compton, James; Hannan, Mike; Brandon, Jay

    2011-01-01

    The ARES I-X vehicle was launched from NASA's Kennedy Space Center (KSC) on October 28, 2009 at approximately 11:30 EDT. ARES I-X was the first test flight for NASA s ARES I launch vehicle, and it was the first non-Shuttle launch vehicle designed and flown by NASA since Saturn. The ARES I-X had a 4-segment solid rocket booster (SRB) first stage and a dummy upper stage (US) to emulate the properties of the ARES I US. During ARES I-X pre-flight modeling and analysis, six (6) independent time domain simulation tools were developed and cross validated. Each tool represents an independent implementation of a common set of models and parameters in a different simulation framework and architecture. Post flight data and reconstructed models provide the means to validate a subset of the simulations against actual flight data and to assess the accuracy of pre-flight dispersion analysis. Post flight data consists of telemetered Operational Flight Instrumentation (OFI) data primarily focused on flight computer outputs and sensor measurements as well as Best Estimated Trajectory (BET) data that estimates vehicle state information from all available measurement sources. While pre-flight models were found to provide a reasonable prediction of the vehicle flight, reconstructed models were generated to better represent and simulate the ARES I-X flight. Post flight reconstructed models include: SRB propulsion model, thrust vector bias models, mass properties, base aerodynamics, and Meteorological Estimated Trajectory (wind and atmospheric data). The result of the effort is a set of independently developed, high fidelity, time-domain simulation tools that have been cross validated and validated against flight data. This paper presents the process and results of high fidelity aerospace modeling, simulation, analysis and tool validation in the time domain.

  11. Developing the Polish Educational Needs Assessment Tool (Pol-ENAT) in rheumatoid arthritis and systemic sclerosis: a cross-cultural validation study using Rasch analysis.

    PubMed

    Sierakowska, Matylda; Sierakowski, Stanisław; Sierakowska, Justyna; Horton, Mike; Ndosi, Mwidimi

    2015-03-01

    To undertake cross-cultural adaptation and validation of the educational needs assessment tool (ENAT) for use with people with rheumatoid arthritis (RA) and systemic sclerosis (SSc) in Poland. The study involved two main phases: (1) cross-cultural adaptation of the ENAT from English into Polish and (2) Cross-cultural validation of Polish Educational Needs Assessment Tool (Pol-ENAT). The first phase followed an established process of cross-cultural adaptation of self-report measures. The second phase involved completion of the Pol-ENAT by patients and subjecting the data to Rasch analysis to assess the construct validity, unidimensionality, internal consistency and cross-cultural invariance. An adequate conceptual equivalence was achieved following the adaptation process. The dataset for validation comprised a total of 278 patients, 237 (85.3 %) of which were female. In each disease group (145, RA and 133, SSc), the 7 domains of the Pol-ENAT were found to fit the Rasch model, X (2)(df) = 16.953(14), p = 0.259 and 8.132(14), p = 0.882 for RA and SSc, respectively. Internal consistency of the Pol-ENAT was high (patient separation index = 0.85 and 0.89 for SSc and RA, respectively), and unidimensionality was confirmed. Cross-cultural differential item functioning (DIF) was detected in some subscales, and DIF-adjusted conversion tables were calibrated to enable cross-cultural comparison of data between Poland and the UK. Using a standard process in cross-cultural adaptation, conceptual equivalence was achieved between the original (UK) ENAT and the adapted Pol-ENAT. Fit to the Rasch model, confirmed that the construct validity, unidimensionality and internal consistency of the ENAT have been preserved.

  12. Development of a Peer Teaching-Assessment Program and a Peer Observation and Evaluation Tool

    PubMed Central

    Trujillo, Jennifer M.; Barr, Judith; Gonyeau, Michael; Van Amburgh, Jenny A.; Matthews, S. James; Qualters, Donna

    2008-01-01

    Objectives To develop a formalized, comprehensive, peer-driven teaching assessment program and a valid and reliable assessment tool. Methods A volunteer taskforce was formed and a peer-assessment program was developed using a multistep, sequential approach and the Peer Observation and Evaluation Tool (POET). A pilot study was conducted to evaluate the efficiency and practicality of the process and to establish interrater reliability of the tool. Intra-class correlation coefficients (ICC) were calculated. Results ICCs for 8 separate lectures evaluated by 2-3 observers ranged from 0.66 to 0.97, indicating good interrater reliability of the tool. Conclusion Our peer assessment program for large classroom teaching, which includes a valid and reliable evaluation tool, is comprehensive, feasible, and can be adopted by other schools of pharmacy. PMID:19325963

  13. Patient simulation: a literary synthesis of assessment tools in anesthesiology.

    PubMed

    Edler, Alice A; Fanning, Ruth G; Chen, Michael I; Claure, Rebecca; Almazan, Dondee; Struyk, Brain; Seiden, Samuel C

    2009-12-20

    High-fidelity patient simulation (HFPS) has been hypothesized as a modality for assessing competency of knowledge and skill in patient simulation, but uniform methods for HFPS performance assessment (PA) have not yet been completely achieved. Anesthesiology as a field founded the HFPS discipline and also leads in its PA. This project reviews the types, quality, and designated purpose of HFPS PA tools in anesthesiology. We used the systematic review method and systematically reviewed anesthesiology literature referenced in PubMed to assess the quality and reliability of available PA tools in HFPS. Of 412 articles identified, 50 met our inclusion criteria. Seventy seven percent of studies have been published since 2000; more recent studies demonstrated higher quality. Investigators reported a variety of test construction and validation methods. The most commonly reported test construction methods included "modified Delphi Techniques" for item selection, reliability measurement using inter-rater agreement, and intra-class correlations between test items or subtests. Modern test theory, in particular generalizability theory, was used in nine (18%) of studies. Test score validity has been addressed in multiple investigations and shown a significant improvement in reporting accuracy. However the assessment of predicative has been low across the majority of studies. Usability and practicality of testing occasions and tools was only anecdotally reported. To more completely comply with the gold standards for PA design, both shared experience of experts and recognition of test construction standards, including reliability and validity measurements, instrument piloting, rater training, and explicit identification of the purpose and proposed use of the assessment tool, are required.

  14. The development and validation of the Youth Actuarial Care Needs Assessment Tool for Non-Offenders (Y-ACNAT-NO).

    PubMed

    Assink, Mark; van der Put, Claudia E; Oort, Frans J; Stams, Geert Jan J M

    2015-03-04

    In The Netherlands, police officers not only come into contact with juvenile offenders, but also with a large number of juveniles who were involved in a criminal offense, but not in the role of a suspect (i.e., juvenile non-offenders). Until now, no valid and reliable instrument was available that can be used by Dutch police officers for estimating the risk for future care needs of juvenile non-offenders. In the present study, the Youth Actuarial Care Needs Assessment Tool for Non-Offenders (Y-ACNAT-NO) was developed for predicting the risk for future care needs that consisted of (1) a future supervision order as imposed by a juvenile court judge and (2) future worrisome incidents involving child abuse, domestic violence/strife, and/or sexual offensive behavior at the juvenile's living address (i.e., problems in the child-rearing environment). Police records of 3,200 juveniles were retrieved from the Dutch police registration system after which the sample was randomly split in a construction (n = 1,549) and validation sample (n = 1,651). The Y-ACNAT-NO was developed by performing an Exhaustive CHAID analysis using the construction sample. The predictive validity of the instrument was examined in the validation sample by calculating several performance indicators that assess discrimination and calibration. The CHAID output yielded an instrument that consisted of six variables and eleven different risk groups. The risk for future care needs ranged from 0.06 in the lowest risk group to 0.83 in the highest risk group. The AUC value in the validation sample was .764 (95% CI [.743, .784]) and Sander's calibration score indicated an average assessment error of 3.74% in risk estimates per risk category. The Y-ACNAT-NO is the first instrument that can be used by Dutch police officers for estimating the risk for future care needs of juvenile non-offenders. The predictive validity of the Y-ACNAT-NO in terms of discrimination and calibration was sufficient to justify

  15. Appraising the risk matrix 2000 static sex offender risk assessment tool.

    PubMed

    Tully, Ruth J; Browne, Kevin D

    2015-02-01

    This critical appraisal explores the reliability and validity of the Risk Matrix 2000 static sex offender risk assessment tool that is widely used in the United Kingdom. The Risk Matrix 2000 has to some extent been empirically validated for use with adult male sex offenders; however, this review highlights that further research into the validity of this static tool with sex offender subgroups or types is necessary in order to improve practical utility. The Risk Matrix 2000 relies on static risk predictors, thus it is limited in scope. This article argues that the addition of dynamic items that have been shown to be predictive of sexual recidivism would further enhance the tool. The paper argues that adding dynamic risk items would fit better with a rehabilitative approach to sex offender risk management and assessment. This would also provide a means by which to effectively plan sex offender treatment and evaluate individual offenders' progress in treatment; however, difficulties remain in identifying and assessing dynamic risk factors of sexual offending and so further research is required. © The Author(s) 2013.

  16. The PRECIS-2 tool has good interrater reliability and modest discriminant validity.

    PubMed

    Loudon, Kirsty; Zwarenstein, Merrick; Sullivan, Frank M; Donnan, Peter T; Gágyor, Ildikó; Hobbelen, Hans J S M; Althabe, Fernando; Krishnan, Jerry A; Treweek, Shaun

    2017-08-01

    PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders. Copyright © 2017 Elsevier Inc. All rights reserved.

  17. Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

    PubMed

    Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

    2016-04-01

    History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy

  18. Developing a tool for assessing competency in root cause analysis.

    PubMed

    Gupta, Priyanka; Varkey, Prathibha

    2009-01-01

    Root cause analysis (RCA) is a tool for identifying the key cause(s) contributing to a sentinel event or near miss. Although training in RCA is gaining popularity in medical education, there is no published literature on valid or reliable methods for assessing competency in the same. A tool for assessing competency in RCA was pilot tested as part of an eight-station Objective Structured Clinical Examination that was conducted at the completion of a three-week quality improvement (QI) curriculum for the Mayo Clinic Preventive Medicine and Endocrinology fellowship programs. As part of the curriculum, fellows completed a QI project to enhance physician communication of the diagnosis and treatment plan at the end of a patient visit. They had a didactic session on RCA, followed by process mapping of the information flow at the project clinic, after which fellows conducted an actual RCA using the Ishikawa fishbone diagram. For the RCA competency assessment, fellows performed an RCA regarding a scenario describing an adverse medication event and provided possible solutions to prevent such errors in the future. All faculty strongly agreed or agreed that they were able to accurately assess competency in RCA using the tool. Interrater reliability for the global competency rating and checklist scoring were 0.96 and 0.85, respectively. Internal consistency (Cronbach's alpha) was 0.76. Six of eight of the fellows found the difficulty level of the test to be optimal. Assessment methods must accompany education programs to ensure that graduates are competent in QI methodologies and are able to apply them effectively in the workplace. The RCA assessment tool was found to be a valid, reliable, feasible, and acceptable method for assessing competency in RCA. Further research is needed to examine its predictive validity and generalizability.

  19. The Scarbase Duo(®): Intra-rater and inter-rater reliability and validity of a compact dual scar assessment tool.

    PubMed

    Fell, Matthew; Meirte, Jill; Anthonissen, Mieke; Maertens, Koen; Pleat, Jonathon; Moortgat, Peter

    2016-03-01

    Objective scar assessment tools were designed to help identify problematic scars and direct clinical management. Their use has been restricted by their measurement of a single scar property and the bulky size of equipment. The Scarbase Duo(®) was designed to assess both trans-epidermal water loss (TEWL) and colour of a burn scar whilst being compact and easy to use. Twenty patients with a burn scar were recruited and measurements taken using the Scarbase Duo(®) by two observers. The Scarbase Duo(®) measures TEWL via an open-chamber system and undertakes colorimetry via narrow-band spectrophotometry, producing values for relative erythema and melanin pigmentation. Validity was assessed by comparing the Scarbase Duo(®) against the Dermalab(®) and the Minolta Chromameter(®) respectively for TEWL and colorimetry measurements. The intra-class correlation coefficient (ICC) was used to assess reliability with standard error of measurement (SEM) used to assess reproducibility of measurements. The Pearson correlation coefficient (r) was used to assess the convergent validity. The Scarbase Duo(®) TEWL mode had excellent reliability when used on scars for both intra- (ICC=0.95) and inter-rater (ICC=0.96) measurements with moderate SEM values. The erythema component of the colorimetry mode showed good reliability for use on scars for both intra-(ICC=0.81) and inter-rater (ICC=0.83) measurements with low SEM values. Pigmentation values showed excellent reliability on scar tissue for both intra- (ICC=0.97) and inter-rater (ICC=0.97) with moderate SEM values. The Scarbase Duo(®) TEWL function had excellent correlation with the Dermalab(®) (r=0.93) whilst the colorimetry erythema value had moderate correlation with the Minolta Chromameter (r=0.72). The Scarbase Duo(®) is a reliable and objective scar assessment tool, which is specifically designed for burn scars. However, for clinical use, standardised measurement conditions are recommended. Copyright © 2015 Elsevier

  20. Validity and reliability of Internet-based physiotherapy assessment for musculoskeletal disorders: a systematic review.

    PubMed

    Mani, Suresh; Sharma, Shobha; Omar, Baharudin; Paungmali, Aatit; Joseph, Leonard

    2017-04-01

    Purpose The purpose of this review is to systematically explore and summarise the validity and reliability of telerehabilitation (TR)-based physiotherapy assessment for musculoskeletal disorders. Method A comprehensive systematic literature review was conducted using a number of electronic databases: PubMed, EMBASE, PsycINFO, Cochrane Library and CINAHL, published between January 2000 and May 2015. The studies examined the validity, inter- and intra-rater reliabilities of TR-based physiotherapy assessment for musculoskeletal conditions were included. Two independent reviewers used the Quality Appraisal Tool for studies of diagnostic Reliability (QAREL) and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool to assess the methodological quality of reliability and validity studies respectively. Results A total of 898 hits were achieved, of which 11 articles based on inclusion criteria were reviewed. Nine studies explored the concurrent validity, inter- and intra-rater reliabilities, while two studies examined only the concurrent validity. Reviewed studies were moderate to good in methodological quality. The physiotherapy assessments such as pain, swelling, range of motion, muscle strength, balance, gait and functional assessment demonstrated good concurrent validity. However, the reported concurrent validity of lumbar spine posture, special orthopaedic tests, neurodynamic tests and scar assessments ranged from low to moderate. Conclusion TR-based physiotherapy assessment was technically feasible with overall good concurrent validity and excellent reliability, except for lumbar spine posture, orthopaedic special tests, neurodynamic testa and scar assessment.

  1. Assessing patient-caregiver communication in cancer--a psychometric validation of the Cancer Communication Assessment Tool (CCAT-PF) in a German sample.

    PubMed

    Haun, Markus W; Sklenarova, Halina; Winkler, Eva C; Huber, Johannes; Thomas, Michael; Siminoff, Laura A; Woll, Michael; Brechtel, Anette; Herzog, Wolfgang; Hartmann, Mechthild

    2014-09-01

    The recently introduced Cancer Communication Assessment Tool (CCAT-PF) measures congruence in patient-caregiver communication and was initially validated in lung cancer patients. Contributing to a greater proportion of the variance in the conflict scores, primary caregivers were hypothesized to experience greater stress. For a detailed understanding of conflicting communication patterns of cancer-affected families, our study aimed for psychometric validation of the CCAT-PF in a sample covering heterogeneous tumor entities. Subsequent to a cross-sectional survey of 189 pairs of cancer patients (31 % gastrointestinal, 34 % lung, and 35 % urological) and their caregivers' exploratory factor analysis with principal component condensation and varimax rotation was conducted (response rate, 74.2 %). Reliability and construct validity were assessed calculating Cronbach's α and Pearson correlation coefficients for CCAT-P and CCAT-F scales and related constructs, respectively. Cancer-related communication according to the CCAT-PF can be subdivided into four factors including the scales Disclosure, Limitation of treatment, Family involvement in treatment decisions, and Continuing treatment. Reliability ranged from α = .51-.68. The Disclosure scale, describing poor cancer-related communication of the patient, was correlated with patient's distress (QSC-R10: r = .30, p < .0001), unmet needs in several areas (SCNS-SF-34: r = .25-.32, p < .001), and negatively with social/family well-being (FACT: r = -0.31, p < .0001). Higher scores on the scale were significantly associated with considerable decrements in emotional well-being especially for caregivers perceiving patients' disclosure as problematic. The Disclosure scale originating from the CCAT-PF emerged as a short, valid, and reliable stand-alone instrument for identifying conflicting communication in patient-caregiver-dyads at risk.

  2. Initial Steps in Creating a Developmentally Valid Tool for Observing/Assessing Rope Jumping

    ERIC Educational Resources Information Center

    Roberton, Mary Ann; Thompson, Gregory; Langendorfer, Stephen J.

    2017-01-01

    Background: Valid motor development sequences show the various behaviors that children display as they progress toward competence in specific motor skills. Teachers can use these sequences to observe informally or formally assess their students. While longitudinal study is ultimately required to validate developmental sequences, there are earlier,…

  3. Development and Validation of a Cross-Country Hospital Patient Quality of Care Assessment Tool in Europe.

    PubMed

    Villiers-Tuthill, Amanda; Doulougeri, Karolina; McGee, Hannah; Montgomery, Anthony; Panagopoulou, Efharis; Morgan, Karen

    2017-12-01

    Patient perceptions of quality of care (QoC) are directly linked with patient safety and clinical effectiveness. We need patient-designed QoC instruments that work across languages and countries to optimise studies across systems in this area. Few QoC measurement tools exist that assess all aspects of QoC from the patient perspective. This paper describes the development and validation of a comprehensive measure to assess patient perceptions of QoC that incorporates technical and interpersonal aspects of care and is grounded in the established Institute of Medicine (IOM) QoC framework. We conducted a multi-country cross-sectional study. Following a literature review and patient focus groups, an expert panel generated questionnaire items. Following a pilot study, item numbers were reduced. The final questionnaire consisted of three sections: demographics, perceived QoC and one open-ended question. Data was collected from patients (n = 531) discharged from hospitals across seven countries in South East Europe (languages: Turkish, Greek, Portuguese, Romanian, Croatian, Macedonian and Bulgarian). Reliability and validity of the measure were assessed. Confirmatory factor analysis was used to compare various factor models of patient-perceived QoC. Good model fit was demonstrated for a two-factor model: communication and interpersonal care, and hospital facilities. The ORCAB (Improving quality and safety in the hospital: The link between organisational culture, burnout and quality of care) Patient QoC questionnaire has been collaboratively and exhaustively developed between healthcare professionals and patients. It enables patient QoC data to be assessed in the context of the IOM pillars of quality, considering both technical and interpersonal dimensions of care. It represents an important first step in including the patient perspective.

  4. Redesign and Validation of Sisom, an Interactive Assessment and Communication Tool for Children With Cancer

    PubMed Central

    2016-01-01

    Background Children with cancer undergo intensive and long treatment periods that expose them and their families to a number of difficult physical, mental, and social challenges. Empowering children by actively involving them in their care can help them to cope with these challenges. It can, however, be difficult for children to be involved and talk about their illness experiences in a “traditional” conversation with health care professionals, especially for younger children. Sisom (Norwegian acronym “Si det som det er” or “Tell it how it is”) is an interactive computer-based assessment and communication tool to give children (aged 6-12 years) with cancer a “voice” in their care. Because of technological advances and widespread use of mobile devices Sisom had to be redesigned to better meet the needs of children of today. Objective To redesign Sisom for use on mobile devices and to validate and adapt it for use in a Swedish population of children with cancer. Methods A user-experience design was used. Content adaptation included forward-backward translation by Swedish and Norwegian translators. Healthy children (n=5), children with experiences of cancer treatment (n=5) and their parents (n=5), and pediatric nurses (n=2) were then involved in culturally adapting Sisom to the Swedish context. The iterative low- and high-fidelity evaluation was supported by a think aloud method, semistructured interviews, and drawings to capture children’s views of Sisom. The redesign and evaluation continued until no further changes or improvements were identified by the participants or the researchers. Results Children, parents, and pediatric nurses offered many suggestions for improvements to the original version in terms of content, aesthetics, and usability of Sisom. The most significant change that emerged through user input was a modification that entailed not using problem-focused statements in the assessment items. The parents and pediatric nurses considered

  5. [Validation, adaptation and translation of the MacCAT-T into Spanish: a tool to assess the ability to make health decisions].

    PubMed

    Hernando Robles, P; Lechuga Pérez, X; Solé Llop, P; Diestre, G; Mariné Torrent, A; Rodríguez Jornet, A; Marquina Parra, D; Colomer Mirabell, O

    2012-01-01

    Capacity assessment is an essential element of the informed consent process and is the duty of the physician. The MacCAT-T instrument explores four skills needed to consent a treatment. There is no Spanish version, and the main objective of this work is to validate, adapt and translate the MacCAT-T into Spanish. The MacCAT-T was translated into Spanish and then back-translated into English. It was validated as regards its appearance and content (by 15 experts), construct (inter-rater reliability and internal consistency) and criteria (the validity of an instrument by comparing it to some external criterion, in this case the Mini Examen Cognoscitivo de Lobo). Ninety medical and surgical outpatients over 18 years were included with no deficits of expression and/or severe disorders of consciousness that did not allow them to be interviewed. They have been optimal considering different types of validity. The average application time was between 9 and 13minutes. Data are consistent with those obtained in other applications of MacCAT-T in the English language and facilitate the provision of a Spanish tool for assessing capacity. Copyright © 2011 SECA. Published by Elsevier Espana. All rights reserved.

  6. Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT) among professionals in Dutch disease-management partnerships.

    PubMed

    Cramm, Jane M; Strating, Mathilde Mh; Nieboer, Anna P

    2011-06-30

    The extent to which partnership synergy is created within quality improvement programmes in the Netherlands is unknown. In this article, we describe the psychometric testing of the Partnership Self-Assessment Tool (PSAT) among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability. The Dutch version of the PSAT was tested in twenty-two disease-management partnerships with 218 professionals. We tested the instrument by means of structural equation modelling, and examined its validity and reliability. After eliminating 14 items, the confirmatory factor analyses revealed good indices of fit with the resulting 15-item PSAT-Short version (PSAT-S). Internal consistency as represented by Cronbach's alpha ranged from acceptable (0.75) for the 'efficiency' subscale to excellent for the 'leadership' subscale (0.87). Convergent validity was provided with high correlations of the partnership dimensions and partnership synergy (ranged from 0.512 to 0.609) and high correlations with chronic illness care (ranged from 0.447 to 0.329). The psychometric properties and convergent validity of the PSAT-S were satisfactory rendering it a valid and reliable instrument for assessing partnership synergy and its dimensions of partnership functioning.

  7. Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT) among professionals in Dutch disease-management partnerships

    PubMed Central

    2011-01-01

    Background The extent to which partnership synergy is created within quality improvement programmes in the Netherlands is unknown. In this article, we describe the psychometric testing of the Partnership Self-Assessment Tool (PSAT) among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability. Methods The Dutch version of the PSAT was tested in twenty-two disease-management partnerships with 218 professionals. We tested the instrument by means of structural equation modelling, and examined its validity and reliability. Results After eliminating 14 items, the confirmatory factor analyses revealed good indices of fit with the resulting 15-item PSAT-Short version (PSAT-S). Internal consistency as represented by Cronbach's alpha ranged from acceptable (0.75) for the 'efficiency' subscale to excellent for the 'leadership' subscale (0.87). Convergent validity was provided with high correlations of the partnership dimensions and partnership synergy (ranged from 0.512 to 0.609) and high correlations with chronic illness care (ranged from 0.447 to 0.329). Conclusion The psychometric properties and convergent validity of the PSAT-S were satisfactory rendering it a valid and reliable instrument for assessing partnership synergy and its dimensions of partnership functioning. PMID:21714931

  8. Design and validation of a three-instrument toolkit for the assessment of competence in electrocardiogram rhythm recognition.

    PubMed

    Hernández-Padilla, José M; Granero-Molina, José; Márquez-Hernández, Verónica V; Suthers, Fiona; López-Entrambasaguas, Olga M; Fernández-Sola, Cayetano

    2017-06-01

    Rapid and accurate interpretation of cardiac arrhythmias by nurses has been linked with safe practice and positive patient outcomes. Although training in electrocardiogram rhythm recognition is part of most undergraduate nursing programmes, research continues to suggest that nurses and nursing students lack competence in recognising cardiac rhythms. In order to promote patient safety, nursing educators must develop valid and reliable assessment tools that allow the rigorous assessment of this competence before nursing students are allowed to practise without supervision. The aim of this study was to develop and psychometrically evaluate a toolkit to holistically assess competence in electrocardiogram rhythm recognition. Following a convenience sampling technique, 293 nursing students from a nursing faculty in a Spanish university were recruited for the study. The following three instruments were developed and psychometrically tested: an electrocardiogram knowledge assessment tool (ECG-KAT), an electrocardiogram skills assessment tool (ECG-SAT) and an electrocardiogram self-efficacy assessment tool (ECG-SES). Reliability and validity (content, criterion and construct) of these tools were meticulously examined. A high Cronbach's alpha coefficient demonstrated the excellent reliability of the instruments (ECG-KAT=0.89; ECG-SAT=0.93; ECG-SES=0.98). An excellent context validity index (scales' average content validity index>0.94) and very good criterion validity were evidenced for all the tools. Regarding construct validity, principal component analysis revealed that all items comprising the instruments contributed to measure knowledge, skills or self-efficacy in electrocardiogram rhythm recognition. Moreover, known-groups analysis showed the tools' ability to detect expected differences in competence between groups with different training experiences. The three-instrument toolkit developed showed excellent psychometric properties for measuring competence in

  9. Improvement of communication and interpersonal competence in telenursing--development of a self-assessment tool.

    PubMed

    Johnson, Christina; Wilhelmsson, Susan; Börjeson, Sussanne; Lindberg, Malou

    2015-06-01

    The aim of this study was to develop a self-assessment tool aiming to raise telenurses' awareness of their communication and interpersonal competence, and highlight areas in need of improvement. Several studies have revealed the need for development of communication competence in telenursing. Structured analyses of conversations with patients/callers, is one way to increase telenurses' awareness of their unique communication and interpersonal competence. Instrument development, Validation assessment using the method Content Validity Index. The process to determine content validity was done in two stages; the development stage and the assessment stage. The development stage started with a literature search. The assessment stage was separated into two phases, assessment by an expert group and assessment and test by telenurses. The telenurses also participated in consensus discussions. A telenursing self-assessment tool with 58 items was developed. The items were sorted into five sections according to the nursing process. This study describes the thorough development process of the telenursing self-assessment tool to be used by telenurses in order to become aware of their unique communication and interpersonal competence when analysing their own conversations with patients/callers. As a formative tool it is meant to provide self-direction, feedback and coaching, and create learning opportunities. The self-assessment tool helps the telenurse to follow the nursing process, to be patient-centred, and it is meant to provide self-direction, feedback, and coaching, as well as create learning opportunities. The tool can contribute to the development of communication and interpersonal competence in telephone advice nursing. Further development of the tool may provide an objective scoring instrument for evaluating communication training and education in the field. © 2014 John Wiley & Sons Ltd.

  10. The Music Therapy Session Assessment Scale (MT-SAS): Validation of a new tool for music therapy process evaluation.

    PubMed

    Raglio, Alfredo; Gnesi, Marco; Monti, Maria Cristina; Oasi, Osmano; Gianotti, Marta; Attardo, Lapo; Gontero, Giulia; Morotti, Lara; Boffelli, Sara; Imbriani, Chiara; Montomoli, Cristina; Imbriani, Marcello

    2017-11-01

    Music therapy (MT) interventions are aimed at creating and developing a relationship between patient and therapist. However, there is a lack of validated observational instruments to consistently evaluate the MT process. The purpose of this study was the validation of Music Therapy Session Assessment Scale (MT-SAS), designed to assess the relationship between therapist and patient during active MT sessions. Videotapes of a single 30-min session per patient were considered. A pilot study on the videotapes of 10 patients was carried out to help refine the items, define the scoring system and improve inter-rater reliability among the five raters. Then, a validation study on 100 patients with different clinical conditions was carried out. The Italian MT-SAS was used throughout the process, although we also provide an English translation. The final scale consisted of 7 binary items accounting for eye contact, countenance, and nonverbal and sound-music communication. In the pilot study, raters were found to share an acceptable level of agreement in their assessments. Explorative factorial analysis disclosed a single homogeneous factor including 6 items (thus supporting an ordinal total score), with only the item about eye contact being unrelated to the others. Moreover, the existence of 2 different archetypal profiles of attuned and disattuned behaviours was highlighted through multiple correspondence analysis. As suggested by the consistent results of 2 different analyses, MT-SAS is a reliable tool that globally evaluates sonorous-musical and nonverbal behaviours related to emotional attunement and empathetic relationship between patient and therapist during active MT sessions. Copyright © 2017 John Wiley & Sons, Ltd.

  11. Verification and Validation Strategy for LWRS Tools

    DOE Office of Scientific and Technical Information (OSTI.GOV)

    Carl M. Stoots; Richard R. Schultz; Hans D. Gougar

    2012-09-01

    One intension of the Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program is to create advanced computational tools for safety assessment that enable more accurate representation of a nuclear power plant safety margin. These tools are to be used to study the unique issues posed by lifetime extension and relicensing of the existing operating fleet of nuclear power plants well beyond their first license extension period. The extent to which new computational models / codes such as RELAP-7 can be used for reactor licensing / relicensing activities depends mainly upon the thoroughness with which they have been verifiedmore » and validated (V&V). This document outlines the LWRS program strategy by which RELAP-7 code V&V planning is to be accomplished. From the perspective of developing and applying thermal-hydraulic and reactivity-specific models to reactor systems, the US Nuclear Regulatory Commission (NRC) Regulatory Guide 1.203 gives key guidance to numeric model developers and those tasked with the validation of numeric models. By creating Regulatory Guide 1.203 the NRC defined a framework for development, assessment, and approval of transient and accident analysis methods. As a result, this methodology is very relevant and is recommended as the path forward for RELAP-7 V&V. However, the unique issues posed by lifetime extension will require considerations in addition to those addressed in Regulatory Guide 1.203. Some of these include prioritization of which plants / designs should be studied first, coupling modern supporting experiments to the stringent needs of new high fidelity models / codes, and scaling of aging effects.« less

  12. Diagnostic flexible pharyngo-laryngoscopy: development of a procedure specific assessment tool using a Delphi methodology.

    PubMed

    Melchiors, Jacob; Henriksen, Mikael Johannes Vuokko; Dikkers, Frederik G; Gavilán, Javier; Noordzij, J Pieter; Fried, Marvin P; Novakovic, Daniel; Fagan, Johannes; Charabi, Birgitte W; Konge, Lars; von Buchwald, Christian

    2018-05-01

    Proper training and assessment of skill in flexible pharyngo-laryngoscopy are central in the education of otorhinolaryngologists. To facilitate an evidence-based approach to curriculum development in this field, a structured analysis of what constitutes flexible pharyngo-laryngoscopy is necessary. Our aim was to develop an assessment tool based on this analysis. We conducted an international Delphi study involving experts from twelve countries in five continents. Utilizing reiterative assessment, the panel defined the procedure and reached consensus (defined as 80% agreement) on the phrasing of an assessment tool. FIFTY PANELISTS COMPLETED THE DELPHI PROCESS. THE MEDIAN AGE OF THE PANELISTS WAS 44 YEARS (RANGE 33-64 YEARS). MEDIAN EXPERIENCE IN OTORHINOLARYNGOLOGY WAS 15 YEARS (RANGE 6-35 YEARS). TWENTY-FIVE WERE SPECIALIZED IN LARYNGOLOGY, 16 WERE HEAD AND NECK SURGEONS, AND NINE WERE GENERAL OTORHINOLARYNGOLOGISTS. AN ASSESSMENT TOOL WAS CREATED CONSISTING OF TWELVE DISTINCT ITEMS.: Conclusion The gathering of validity evidence for assessment of core procedural skills within Otorhinolaryngology is central to the development of a competence-based education. The use of an international Delphi panel allows for the creation of an assessment tool which is widely applicable and valid. This work allows for an informed approach to technical skills training for flexible pharyngo-laryngoscopy and as further validity evidence is gathered allows for a valid assessment of clinical performance within this important skillset.

  13. Validity of a self-report survey tool measuring the nutrition and physical activity environment of primary schools.

    PubMed

    Nathan, Nicole; Wolfenden, Luke; Morgan, Philip J; Bell, Andrew C; Barker, Daniel; Wiggers, John

    2013-06-13

    Valid tools measuring characteristics of the school environment associated with the physical activity and dietary behaviours of children are needed to accurately evaluate the impact of initiatives to improve school environments. The aim of this study was to assess the validity of Principal self-report of primary school healthy eating and physical activity environments. Primary school Principals (n = 42) in New South Wales, Australia were invited to complete a telephone survey of the school environment; the School Environment Assessment Tool - SEAT. Equivalent observational data were collected by pre-service teachers located within the school. The SEAT, involved 65 items that assessed food availability via canteens, vending machines and fundraisers and the presence of physical activity facilities, equipment and organised physical activities. Kappa statistics were used to assess agreement between the two measures. Almost 70% of the survey demonstrated moderate to almost perfect agreement. Substantial agreement was found for 10 of 13 items assessing foods sold for fundraising, 3 of 6 items assessing physical activity facilities of the school, and both items assessing organised physical activities that occurred at recess and lunch and school sport. Limited agreement was found for items assessing foods sold through canteens and access to small screen recreation. The SEAT provides researchers and policy makers with a valid tool for assessing aspects of the school food and physical activity environment.

  14. Validity and inter-observer reliability of subjective hand-arm vibration assessments.

    PubMed

    Coenen, Pieter; Formanoy, Margriet; Douwes, Marjolein; Bosch, Tim; de Kraker, Heleen

    2014-07-01

    Exposure to mechanical vibrations at work (e.g., due to handling powered tools) is a potential occupational risk as it may cause upper extremity complaints. However, reliable and valid assessment methods for vibration exposure at work are lacking. Measuring hand-arm vibration objectively is often difficult and expensive, while often used information provided by manufacturers lacks detail. Therefore, a subjective hand-arm vibration assessment method was tested on validity and inter-observer reliability. In an experimental protocol, sixteen tasks handling powered tools were executed by two workers. Hand-arm vibration was assessed subjectively by 16 observers according to the proposed subjective assessment method. As a gold standard reference, hand-arm vibration was measured objectively using a vibration measurement device. Weighted κ's were calculated to assess validity, intra-class-correlation coefficients (ICCs) were calculated to assess inter-observer reliability. Inter-observer reliability of the subjective assessments depicting the agreement among observers can be expressed by an ICC of 0.708 (0.511-0.873). The validity of the subjective assessments as compared to the gold-standard reference can be expressed by a weighted κ of 0.535 (0.285-0.785). Besides, the percentage of exact agreement of the subjective assessment compared to the objective measurement was relatively low (i.e., 52% of all tasks). This study shows that subjectively assessed hand-arm vibrations are fairly reliable among observers and moderately valid. This assessment method is a first attempt to use subjective risk assessments of hand-arm vibration. Although, this assessment method can benefit from some future improvement, it can be of use in future studies and in field-based ergonomic assessments. Copyright © 2014 Elsevier Ltd and The Ergonomics Society. All rights reserved.

  15. Validation of the Mobile Information Software Evaluation Tool (MISET) With Nursing Students.

    PubMed

    Secco, M Loretta; Furlong, Karen E; Doyle, Glynda; Bailey, Judy

    2016-07-01

    This study evaluated the Mobile Information Software Evaluation Tool (MISET) with a sample of Canadian undergraduate nursing students (N = 240). Psychometric analyses determined how well the MISET assessed the extent that nursing students find mobile device-based information resources useful and supportive of learning in the clinical and classroom settings. The MISET has a valid three-factor structure with high explained variance (74.7%). Internal consistency reliabilities were high for the MISET total (.90) and three subscales: Usefulness/Helpfulness, Information Literacy Support, and Use of Evidence-Based Sources (.87 to .94). Construct validity evidence included significantly higher mean total MISET, Helpfulness/Usefulness, and Information Literacy Support scores for senior students and those with higher computer competence. The MISET is a promising tool to evaluate mobile information technologies and information literacy support; however, longitudinal assessment of changes in scores over time would determine scale sensitivity and responsiveness. [J Nurs Educ. 2016;55(7):385-390.]. Copyright 2016, SLACK Incorporated.

  16. Development and validation of the FRAGIRE tool for assessment an older person's risk for frailty.

    PubMed

    Vernerey, Dewi; Anota, Amelie; Vandel, Pierre; Paget-Bailly, Sophie; Dion, Michele; Bailly, Vanessa; Bonin, Marie; Pozet, Astrid; Foubert, Audrey; Benetkiewicz, Magdalena; Manckoundia, Patrick; Bonnetain, Franck

    2016-11-17

    Frailty is highly prevalent in elderly people. While significant progress has been made to understand its pathogenesis process, few validated questionnaire exist to assess the multidimensional concept of frailty and to detect people frail or at risk to become frail. The objectives of this study were to construct and validate a new frailty-screening instrument named Frailty Groupe Iso-Ressource Evaluation (FRAGIRE) that accurately predicts the risk for frailty in older adults. A prospective multicenter recruitment of the elderly patients was undertaken in France. The subjects were classified into financially-helped group (FH, with financial assistance) and non-financially helped group (NFH, without any financial assistance), considering FH subjects are more frail than the NFH group and thus representing an acceptable surrogate population for frailty. Psychometric properties of the FRAGIRE grid were assessed including discrimination between the FH and NFH groups. Items reduction was made according to statistical analyses and experts' point of view. The association between items response and tests with "help requested status" was assessed in univariate and multivariate unconditional logistic regression analyses and a prognostic score to become frail was finally proposed for each subject. Between May 2013 and July 2013, 385 subjects were included: 338 (88%) in the FH group and 47 (12%) in the NFH group. The initial FRAGIRE grid included 65 items. After conducting the item selection, the final grid of the FRAGIRE was reduced to 19 items. The final grid showed fair discrimination ability to predict frailty (area under the curve (AUC) = 0.85) and good calibration (Hosmer-Lemeshow P-value = 0.580), reflecting a good agreement between the prediction by the final model and actual observation. The Cronbach's alpha for the developed tool scored as high as 0.69 (95% Confidence Interval: 0.64 to 0.74). The final prognostic score was excellent, with an AUC of 0

  17. Exploring the Validity of a Second Language Intercultural Pragmatics Assessment Tool

    ERIC Educational Resources Information Center

    Timpe-Laughlin, Veronika; Choi, Ikkyu

    2017-01-01

    Pragmatics has been a key component of language competence frameworks. While the majority of second/foreign language (L2) pragmatics tests have targeted productive skills, the assessment of receptive pragmatic skills remains a developing field. This study explores validation evidence for a test of receptive L2 pragmatic ability called the American…

  18. Positioning Tool Validation Report

    DTIC Science & Technology

    1999-11-01

    U.S. Coast Guard Research and Development Center 1082 Shennecossett Road, Groton, CT 06340-6096 Report No. CG-D-06-00 Positioning Tool...standard, specification, or regulation. Marc B. Mandler, Ph.D. Technical Director United States Coast Guard Research & Development Center 1082 ...Validation Report 7. Author(s) Jay Spalding i. Performing Organization Name and Address U.S. Coast Guard Research and Development Center 1082

  19. Statistical methodology: II. Reliability and validity assessment in study design, Part B.

    PubMed

    Karras, D J

    1997-02-01

    Validity measures the correspondence between a test and other purported measures of the same or similar qualities. When a reference standard exists, a criterion-based validity coefficient can be calculated. If no such standard is available, the concepts of content and construct validity may be used, but quantitative analysis may not be possible. The Pearson and Spearman tests of correlation are often used to assess the correspondence between tests, but do not account for measurement biases and may yield misleading results. Techniques that measure interest differences may be more meaningful in validity assessment, and the kappa statistic is useful for analyzing categorical variables. Questionnaires often can be designed to allow quantitative assessment of reliability and validity, although this may be difficult. Inclusion of homogeneous questions is necessary to assess reliability. Analysis is enhanced by using Likert scales or similar techniques that yield ordinal data. Validity assessment of questionnaires requires careful definition of the scope of the test and comparison with previously validated tools.

  20. Validity in work-based assessment: expanding our horizons.

    PubMed

    Govaerts, Marjan; van der Vleuten, Cees P M

    2013-12-01

    Although work-based assessments (WBA) may come closest to assessing habitual performance, their use for summative purposes is not undisputed. Most criticism of WBA stems from approaches to validity consistent with the quantitative psychometric framework. However, there is increasing research evidence that indicates that the assumptions underlying the predictive, deterministic framework of psychometrics may no longer hold. In this discussion paper we argue that meaningfulness and appropriateness of current validity evidence can be called into question and that we need alternative strategies to assessment and validity inquiry that build on current theories of learning and performance in complex and dynamic workplace settings. Drawing from research in various professional fields we outline key issues within the mechanisms of learning, competence and performance in the context of complex social environments and illustrate their relevance to WBA. In reviewing recent socio-cultural learning theory and research on performance and performance interpretations in work settings, we demonstrate that learning, competence (as inferred from performance) as well as performance interpretations are to be seen as inherently contextualised, and can only be under-stood 'in situ'. Assessment in the context of work settings may, therefore, be more usefully viewed as a socially situated interpretive act. We propose constructivist-interpretivist approaches towards WBA in order to capture and understand contextualised learning and performance in work settings. Theoretical assumptions underlying interpretivist assessment approaches call for a validity theory that provides the theoretical framework and conceptual tools to guide the validation process in the qualitative assessment inquiry. Basic principles of rigour specific to qualitative research have been established, and they can and should be used to determine validity in interpretivist assessment approaches. If used properly, these

  1. Monitoring dietary intake and physical activity electronically: feasibility, usability, and ecological validity of a mobile-based Ecological Momentary Assessment tool.

    PubMed

    Spook, Jorinde Eline; Paulussen, Theo; Kok, Gerjo; Van Empelen, Pepijn

    2013-09-24

    Despite the growing body of research on complex lifestyle behaviors (eg, Dietary Intake [DI] and Physical Activity [PA]), monitoring of these behaviors has been hampered by a lack of suitable methods. A possible solution to this deficiency is mobile-based Ecological Momentary Assessment (mEMA), which enables researchers to collect data on participants' states in real-time by means of a smartphone application. However, feasibility, usability, and ecological validity need to be anticipated and managed in order to enhance the validity of mEMA. To examine the feasibility, usability, and ecological validity of a mEMA application (app) with regard to DI and PA among Dutch vocational education students. The students (n=30) participated in the mEMA study for seven consecutive days. They downloaded the mEMA app on their smartphone. Feasibility and usability of the mEMA app were evaluated by completing an online evaluation after seven days of participation. Ecological validity was measured by assessing the degree to which the content of the mEMA app approximated the real-world setting that was being examined, through several multiple-choice questions. Compliance rates, as registered by the mEMA app, declined 46% over a seven-day period, while self-reported compliance, as measured with an online evaluation questionnaire afterwards, indicated a smaller decrease in compliance (29%). The students evaluated the mEMA app as feasible and usable. Ecological validity analyses showed that all DI and almost all PA multiple-choice options were covered with the compound response categories. The mEMA app offers the opportunity to assess complex health behaviors (eg, DI and PA) in real-time settings, in which specifically routinized behaviors are involved. However, the mEMA app faced several challenges that needed to be overcome in order to improve its validity. Overall, the present study showed that the mEMA app is a usable and ecologically valid tool to measure DI and PA behaviors among

  2. Validity of the Consensual Assessment Technique--Evidence with Three Groups of Judges and an Elementary School Student Sample

    ERIC Educational Resources Information Center

    Long, Haiying

    2012-01-01

    As one of the most widely used creativity assessment tools, the Consensual Assessment Technique (CAT) has been praised as a valid tool to assess creativity. In Amabile's (1982) seminal work, the inter-rater reliability was defined as construct validity of the CAT. During the past three decades, researchers followed this definition and…

  3. Movement skill assessment of typically developing preschool children: a review of seven movement skill assessment tools.

    PubMed

    Cools, Wouter; Martelaer, Kristine De; Samaey, Christiane; Andries, Caroline

    2009-06-01

    The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child's physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools' usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children. Key pointsThis review discusses seven movement skill assessment tool's test content, reliability, validity and normative samples.The seven assessment tools all showed to be of great value. Strengths and weaknesses indicate that test choice will depend on specific purpose of test use.Further data collection should also include larger data samples of able bodied preschool children.Admitting PE specialists in assessment of fundamental movement skill performance among preschool children is recommended.The assessment tool's normative data samples would benefit from frequent movement skill performance follow-up of today's children. MOT 4-6: Motoriktest fur vier- bis sechsjährige Kinder, M-ABC: Movement Assessment Battery for Children, PDMS: Peabody Development Scales, KTK: Körper-Koordinationtest für Kinder

  4. Validation of the World Health Organization tool for situational analysis to assess emergency and essential surgical care at district hospitals in Ghana.

    PubMed

    Osen, Hayley; Chang, David; Choo, Shelly; Perry, Henry; Hesse, Afua; Abantanga, Francis; McCord, Colin; Chrouser, Kristin; Abdullah, Fizan

    2011-03-01

    The World Health Organization (WHO) Tool for Situational Analysis to Assess Emergency and Essential Surgical Care (hereafter called the WHO Tool) has been used in more than 25 countries and is the largest effort to assess surgical care in the world. However, it has not yet been independently validated. Test-retest reliability is one way to validate the degree to which tests instruments are free from random error. The aim of the present field study was to determine the test-retest reliability of the WHO Tool. The WHO Tool was mailed to 10 district hospitals in Ghana. Written instructions were provided along with a letter from the Ghana Health Services requesting the hospital administrator to complete the survey tool. After ensuring delivery and completion of the forms, the study team readministered the WHO Tool at the time of an on-site visit less than 1 month later. The results of the two tests were compared to calculate kappa statistics for each of the 152 questions in the WHO Tool. The kappa statistic is a statistical measure of the degree of agreement above what would be expected based on chance alone. Ten hospitals were surveyed twice over a short interval (i.e., less than 1 month). Weighted and unweighted kappa statistics were calculated for 152 questions. The median unweighted kappa for the entire survey was 0.43 (interquartile range 0-0.84). The infrastructure section (24 questions) had a median kappa of 0.81; the human resources section (13 questions) had a median kappa of 0.77; the surgical procedures section (67 questions) had a median kappa of 0.00; and the emergency surgical equipment section (48 questions) had a median kappa of 0.81. Hospital capacity survey questions related to infrastructure characteristics had high reliability. However, questions related to process of care had poor reliability and may benefit from supplemental data gathered by direct observation. Limitations to the study include the small sample size: 10 district hospitals in a

  5. The IDEA Assessment Tool: Assessing the Reporting, Diagnostic Reasoning, and Decision-Making Skills Demonstrated in Medical Students' Hospital Admission Notes.

    PubMed

    Baker, Elizabeth A; Ledford, Cynthia H; Fogg, Louis; Way, David P; Park, Yoon Soo

    2015-01-01

    Construct: Clinical skills are used in the care of patients, including reporting, diagnostic reasoning, and decision-making skills. Written comprehensive new patient admission notes (H&Ps) are a ubiquitous part of student education but are underutilized in the assessment of clinical skills. The interpretive summary, differential diagnosis, explanation of reasoning, and alternatives (IDEA) assessment tool was developed to assess students' clinical skills using written comprehensive new patient admission notes. The validity evidence for assessment of clinical skills using clinical documentation following authentic patient encounters has not been well documented. Diagnostic justification tools and postencounter notes are described in the literature (1,2) but are based on standardized patient encounters. To our knowledge, the IDEA assessment tool is the first published tool that uses medical students' H&Ps to rate students' clinical skills. The IDEA assessment tool is a 15-item instrument that asks evaluators to rate students' reporting, diagnostic reasoning, and decision-making skills based on medical students' new patient admission notes. This study presents validity evidence in support of the IDEA assessment tool using Messick's unified framework, including content (theoretical framework), response process (interrater reliability), internal structure (factor analysis and internal-consistency reliability), and relationship to other variables. Validity evidence is based on results from four studies conducted between 2010 and 2013. First, the factor analysis (2010, n = 216) yielded a three-factor solution, measuring patient story, IDEA, and completeness, with reliabilities of .79, .88, and .79, respectively. Second, an initial interrater reliability study (2010) involving two raters demonstrated fair to moderate consensus (κ = .21-.56, ρ =.42-.79). Third, a second interrater reliability study (2011) with 22 trained raters also demonstrated fair to moderate agreement

  6. Assessing behavioural changes in ALS: cross-validation of ALS-specific measures.

    PubMed

    Pinto-Grau, Marta; Costello, Emmet; O'Connor, Sarah; Elamin, Marwa; Burke, Tom; Heverin, Mark; Pender, Niall; Hardiman, Orla

    2017-07-01

    The Beaumont Behavioural Inventory (BBI) is a behavioural proxy report for the assessment of behavioural changes in ALS. This tool has been validated against the FrSBe, a non-ALS-specific behavioural assessment, and further comparison of the BBI against a disease-specific tool was considered. This study cross-validates the BBI against the ALS-FTD-Q. Sixty ALS patients, 8% also meeting criteria for FTD, were recruited. All patients were evaluated using the BBI and the ALS-FTD-Q, completed by a carer. Correlational analysis was performed to assess construct validity. Precision, sensitivity, specificity, and overall accuracy of the BBI when compared to the ALS-FTD-Q, were obtained. The mean score of the whole sample on the BBI was 11.45 ± 13.06. ALS-FTD patients scored significantly higher than non-demented ALS patients (31.6 ± 14.64, 9.62 ± 11.38; p < 0.0001). A significant large positive correlation between the BBI and the ALS-FTD-Q was observed (r = 0.807, p < 0.0001), and no significant correlations between the BBI and other clinical/demographic characteristics indicate good convergent and discriminant validity, respectively. 72% of overall concordance was observed. Precision, sensitivity, and specificity for the classification of severely impaired patients were adequate. However, lower concordance in the classification of mild behavioural changes was observed, with higher sensitivity using the BBI, most likely secondary to BBI items which endorsed behavioural aspects not measured by the ALS-FTD-Q. Good construct validity has been further confirmed when the BBI is compared to an ALS-specific tool. Furthermore, the BBI is a more comprehensive behavioural assessment for ALS, as it measures the whole behavioural spectrum in this condition.

  7. External validation of a simple clinical tool used to predict falls in people with Parkinson disease

    PubMed Central

    Duncan, Ryan P.; Cavanaugh, James T.; Earhart, Gammon M.; Ellis, Terry D.; Ford, Matthew P.; Foreman, K. Bo; Leddy, Abigail L.; Paul, Serene S.; Canning, Colleen G.; Thackeray, Anne; Dibble, Leland E.

    2015-01-01

    Background Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD. METHODS We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment. RESULTS The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76 –0.89), comparable to the developmental study. CONCLUSION The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual’s risk of an impending fall. PMID:26003412

  8. External validation of a simple clinical tool used to predict falls in people with Parkinson disease.

    PubMed

    Duncan, Ryan P; Cavanaugh, James T; Earhart, Gammon M; Ellis, Terry D; Ford, Matthew P; Foreman, K Bo; Leddy, Abigail L; Paul, Serene S; Canning, Colleen G; Thackeray, Anne; Dibble, Leland E

    2015-08-01

    Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD. We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment. The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76-0.89), comparable to the developmental study. The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual's risk of an impending fall. Copyright © 2015 Elsevier Ltd. All rights reserved.

  9. Validation of a method for assessing resident physicians' quality improvement proposals.

    PubMed

    Leenstra, James L; Beckman, Thomas J; Reed, Darcy A; Mundell, William C; Thomas, Kris G; Krajicek, Bryan J; Cha, Stephen S; Kolars, Joseph C; McDonald, Furman S

    2007-09-01

    Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. Seven raters used the instrument to assess 45 resident QI proposals. Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

  10. Falls screening and assessment tools used in acute mental health settings: a review of policies in England and Wales

    PubMed Central

    Narayanan, V.; Dickinson, A.; Victor, C.; Griffiths, C.; Humphrey, D.

    2016-01-01

    Objectives There is an urgent need to improve the care of older people at risk of falls or who experience falls in mental health settings. The aims of this study were to evaluate the individual falls risk assessment tools adopted by National Health Service (NHS) mental health trusts in England and healthcare boards in Wales, to evaluate the comprehensiveness of these tools and to review their predictive validity. Methods All NHS mental health trusts in England (n = 56) and healthcare boards in Wales (n = 6) were invited to supply their falls policies and other relevant documentation (e.g. local falls audits). In order to check the comprehensiveness of tools listed in policy documents, the risk variables of the tools adopted by the mental health trusts’ policies were compared with the 2004 National Institute for Health and Care Excellence (NICE) falls prevention guidelines. A comprehensive analytical literature review was undertaken to evaluate the predictive validity of the tools used in these settings. Results Falls policies were obtained from 46 mental health trusts. Thirty-five policies met the study inclusion criteria and were included in the analysis. The main falls assessment tools used were the St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY), Falls Risk Assessment Scale for the Elderly, Morse Falls Scale (MFS) and Falls Risk Assessment Tool (FRAT). On detailed examination, a number of different versions of the FRAT were evident; validated tools had inconsistent predictive validity and none of them had been validated in mental health settings. Conclusions Falls risk assessment is the most commonly used component of risk prevention strategies, but most policies included unvalidated tools and even well validated tool such as the STRATIFY and the MFS that are reported to have inconsistent predictive accuracy. This raises questions about operational usefulness, as none of these tools have been tested in acute mental health

  11. Above, Beyond, and Over the Side rails: Evaluating the New Memorial Emergency Department Fall-Risk-Assessment Tool.

    PubMed

    Scott, Robin A; Oman, Kathleen S; Flarity, Kathleen; Comer, Jennifer L

    2018-03-06

    Patient falls are a significant issue in hospitalized patients and financially costly to hospitals. The Joint Commission requires that patients be assessed for fall risk and interventions in place to mitigate the risk of falls. It is imperative to have a patient population/setting specific fall risk assessment tool to identify patients at risk for falling. The purpose of this study was to evaluate the reliability and validity of the 2013 Memorial ED Fall Risk Assessment tool (MEDFRAT) specifically designed for the ED population. A two-phase prospective design was used for this study. Phase one determined the interrater reliability of the MEDFRAT. Phase two assessed the validity of the MEDFRAT in an emergency department (ED) within a 600-bed academic/teaching institution; Level II Trauma Center with >100,000 annual patient visits. The Memorial ED Fall Risk Assessment Tool was validated in this ED setting. The tool demonstrated positive interrater reliability (k=0.701) and when implemented with a falls prevention strategy and staff education demonstrated a 48% decrease in ED fall rate (0.57 falls/1000 patient visits) post implementation during the study period. The MEDFRAT, an evidenced based ED-specific fall risk tool was implemented on the basis of the risk factors consistently identified in the literature: prior fall history, impaired mobility, altered mental status, altered elimination, and the use of sedative medication. The Memorial ED Fall Risk Assessment Tool demonstrated to be a valid tool for this hospital system. Copyright © 2018 Emergency Nurses Association. Published by Elsevier Inc. All rights reserved.

  12. Methods for assessing reliability and validity for a measurement tool: a case study and critique using the WHO haemoglobin colour scale.

    PubMed

    White, Sarah A; van den Broek, Nynke R

    2004-05-30

    Before introducing a new measurement tool it is necessary to evaluate its performance. Several statistical methods have been developed, or used, to evaluate the reliability and validity of a new assessment method in such circumstances. In this paper we review some commonly used methods. Data from a study that was conducted to evaluate the usefulness of a specific measurement tool (the WHO Colour Scale) is then used to illustrate the application of these methods. The WHO Colour Scale was developed under the auspices of the WHO to provide a simple portable and reliable method of detecting anaemia. This Colour Scale is a discrete interval scale, whereas the actual haemoglobin values it is used to estimate are on a continuous interval scale and can be measured accurately using electrical laboratory equipment. The methods we consider are: linear regression, correlation coefficients, paired t-tests plotting differences against mean values and deriving limits of agreement; kappa and weighted kappa statistics, sensitivity and specificity, an intraclass correlation coefficient and the repeatability coefficient. We note that although the definition and properties of each of these methods is well established inappropriate methods continue to be used in medical literature for assessing reliability and validity, as evidenced in the context of the evaluation of the WHO Colour Scale. Copyright 2004 John Wiley & Sons, Ltd.

  13. Reliability and Validity of the Alberta Context Tool (ACT) with Professional Nurses: Findings from a Multi-Study Analysis

    PubMed Central

    Squires, Janet E.; Hayduk, Leslie; Hutchinson, Alison M.; Mallick, Ranjeeta; Norton, Peter G.; Cummings, Greta G.; Estabrooks, Carole A.

    2015-01-01

    Although organizational context is central to evidence-based practice, underdeveloped measurement hindersitsassessment. The Alberta Context Tool, comprised of 59 items that tap10 modifiable contextual concepts, was developed to address this gap. The purpose of this study to examine the reliability and validity of scores obtained when the Alberta Context Tool is completed by professional nurses across different healthcare settings. Five separate studies (N = 2361 nurses across different care settings) comprised the study sample. Reliability and validity were assessed. Cronbach’s alpha exceeded 0.70 for9/10 Alberta Context Tool concepts. Item-total correlations exceeded acceptable standards for 56/59items. Confirmatory Factor Analysescoordinated acceptably with the Alberta Context Tool’s proposed latent structure. The mean values for each Alberta Context Tool concept increased from low to high levels of research utilization(as hypothesized) further supporting its validity. This study provides robust evidence forreliability and validity of scores obtained with the Alberta Context Tool when administered to professional nurses. PMID:26098857

  14. Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings.

    PubMed

    Bergström, Anna; Skeen, Sarah; Duc, Duong M; Blandon, Elmer Zelaya; Estabrooks, Carole; Gustavsson, Petter; Hoa, Dinh Thi Phuong; Källestål, Carina; Målqvist, Mats; Nga, Nguyen Thu; Persson, Lars-Åke; Pervin, Jesmin; Peterson, Stefan; Rahman, Anisur; Selling, Katarina; Squires, Janet E; Tomlinson, Mark; Waiswa, Peter; Wallin, Lars

    2015-08-15

    The gap between what is known and what is practiced results in health service users not benefitting from advances in healthcare, and in unnecessary costs. A supportive context is considered a key element for successful implementation of evidence-based practices (EBP). There were no tools available for the systematic mapping of aspects of organizational context influencing the implementation of EBPs in low- and middle-income countries (LMICs). Thus, this project aimed to develop and psychometrically validate a tool for this purpose. The development of the Context Assessment for Community Health (COACH) tool was premised on the context dimension in the Promoting Action on Research Implementation in Health Services framework, and is a derivative product of the Alberta Context Tool. Its development was undertaken in Bangladesh, Vietnam, Uganda, South Africa and Nicaragua in six phases: (1) defining dimensions and draft tool development, (2) content validity amongst in-country expert panels, (3) content validity amongst international experts, (4) response process validity, (5) translation and (6) evaluation of psychometric properties amongst 690 health workers in the five countries. The tool was validated for use amongst physicians, nurse/midwives and community health workers. The six phases of development resulted in a good fit between the theoretical dimensions of the COACH tool and its psychometric properties. The tool has 49 items measuring eight aspects of context: Resources, Community engagement, Commitment to work, Informal payment, Leadership, Work culture, Monitoring services for action and Sources of knowledge. Aspects of organizational context that were identified as influencing the implementation of EBPs in high-income settings were also found to be relevant in LMICs. However, there were additional aspects of context of relevance in LMICs specifically Resources, Community engagement, Commitment to work and Informal payment. Use of the COACH tool will allow

  15. Validation of a novel duplex ultrasound objective structured assessment of technical skills (DUOSATS) for arterial stenosis detection.

    PubMed

    Jaffer, U; Singh, P; Pandey, V A; Aslam, M; Standfield, N J

    2014-01-01

    Duplex ultrasound facilitates bedside diagnosis and hence timely patient care. Its uptake has been hampered by training and accreditation issues. We have developed an assessment tool for Duplex arterial stenosis measurement for both simulator and patient based training. A novel assessment tool: duplex ultrasound assessment of technical skills was developed. A modified duplex ultrasound assessment of technical skills was used for simulator training. Novice, intermediate experience and expert users of duplex ultrasound were invited to participate. Participants viewed an instructional video and were allowed ample time to familiarize with the equipment. Participants' attempts were recorded and independently assessed by four experts using the modified duplex ultrasound assessment of technical skills. 'Global' assessment was also done on a four point Likert scale. Content, construct and concurrent validity as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. The simulator had good satisfaction rating from participants: median 4; range 3-5. Receiver operator characteristic analysis has established a cut point of 22/ 34 and 25/ 40 were most appropriate for simulator and patient based assessment respectively. We have validated a novel assessment tool for duplex arterial stenosis detection. Further work is underway to establish transference validity of simulator training to improved skill in scanning patients. We have developed and validated duplex ultrasound assessment of technical skills for simulator training.

  16. Drug-Associated Risk Tool: development and validation of a self-assessment questionnaire to screen for hospitalised patients at risk for drug-related problems.

    PubMed

    Kaufmann, Carole P; Stämpfli, Dominik; Mory, Nadine; Hersberger, Kurt E; Lampert, Markus L

    2018-03-09

    Identifying patients with a high risk for drug-related problems (DRPs) might optimise the allocation of targeted pharmaceutical care during the hospital stay and on discharge. To develop a self-assessment screening tool to identify patients at risk for DRPs and validate the tool regarding feasibility, acceptability and the reliability of the patients' answers. Prospective validation study. Two mid-sized hospitals (300-400 beds). 195 patients, exclusion criteria: under 18 years old, patients with a health status not allowing a meaningful communication (eg, delirium, acute psychosis, advanced dementia, aphasia, clouded consciousness state), palliative or terminally ill patients. Twenty-seven risk factors for the development of DRPs, identified in a previous study, provided the basis of the self-assessment questionnaire, the Drug-Associated Risk Tool (DART). Consenting patients filled in DART, and we compared their answers with objective patient data from medical records and laboratory data. One hundred and sixty-four patients filled in DART V.1.0 in an average time of 7 min. After a first validation, we identified statements with a low sensitivity and revised the wording of the questions related to heart insufficiency, renal impairment or liver impairment. The revised DART (V.2.0) was validated in 31 patients presenting heart insufficiency, renal impairment or liver impairment as comorbidity and reached an average specificity of 88% (range 27-100) and an average sensitivity of 67% (range 21-100). DART showed a satisfying feasibility and reliability. The specificity of the statements was mostly high. The sensitivity varied and was higher in statements concerning diseases that require regular disease control and attention to self-care and drug management. Asking patients about their conditions, medications and related problems can facilitate getting a first, broad picture of the risk for DRPs and possible pharmaceutical needs. © Article author(s) (or their employer

  17. Improving the Validity and Reliability of Large Scale Writing Assessment.

    ERIC Educational Resources Information Center

    Fenton, Ray; Straugh, Tom; Stofflet, Fred; Garrison, Steve

    This paper examines the efforts of the Anchorage School District, Alaska, to improve the validity of its writing assessment as a useful tool for the training of teachers and the characterization of the quality of student writing. The paper examines how a number of changes in the process and scoring of the Anchorage Writing Assessment affected the…

  18. The Mental Disability Military Assessment Tool: A Reliable Tool for Determining Disability in Veterans with Post-traumatic Stress Disorder.

    PubMed

    Fokkens, Andrea S; Groothoff, Johan W; van der Klink, Jac J L; Popping, Roel; Stewart, Roy E; van de Ven, Lex; Brouwer, Sandra; Tuinstra, Jolanda

    2015-09-01

    An assessment tool was developed to assess disability in veterans who suffer from post-traumatic stress disorder (PTSD) due to a military mission. The objective of this study was to determine the reliability, intra-rater and inter-rater variation of the Mental Disability Military (MDM) assessment tool. Twenty-four assessment interviews of veterans with an insurance physician were videotaped. Each videotaped interview was assessed by a group of five independent raters on limitations of the veterans using the MDM assessment tool. After 2 months the raters repeated this procedure. Next the intra-rater and inter-rater variation was assessed with an adjusted version of AG09 computing weighted percentage agreement. The results of this study showed that both the intra-rater variation and inter-rater variation on the ten subcategories of the MDM assessment tool were small, with an agreement of 84-100% within raters and 93-100% between raters. The MDM assessment tool proves to be a reliable instrument to measure PTSD limitations in functioning in Dutch military veterans who apply for disability compensation. Further research is needed to assess the validity of this instrument.

  19. Reliability and validity of a novel tool to comprehensively assess food and beverage marketing in recreational sport settings.

    PubMed

    Prowse, Rachel J L; Naylor, Patti-Jean; Olstad, Dana Lee; Carson, Valerie; Mâsse, Louise C; Storey, Kate; Kirk, Sara F L; Raine, Kim D

    2018-05-31

    Current methods for evaluating food marketing to children often study a single marketing channel or approach. As the World Health Organization urges the removal of unhealthy food marketing in children's settings, methods that comprehensively explore the exposure and power of food marketing within a setting from multiple marketing channels and approaches are needed. The purpose of this study was to test the inter-rater reliability and the validity of a novel settings-based food marketing audit tool. The Food and beverage Marketing Assessment Tool for Settings (FoodMATS) was developed and its psychometric properties evaluated in five public recreation and sport facilities (sites) and subsequently used in 51 sites across Canada for a cross-sectional analysis of food marketing. Raters recorded the count of food marketing occasions, presence of child-targeted and sports-related marketing techniques, and the physical size of marketing occasions. Marketing occasions were classified by healthfulness. Inter-rater reliability was tested using Cohen's kappa (κ) and intra-class correlations (ICC). FoodMATS scores for each site were calculated using an algorithm that represented the theoretical impact of the marketing environment on food preferences, purchases, and consumption. Higher FoodMATS scores represented sites with higher exposure to, and more powerful (unhealthy, child-targeted, sports-related, large) food marketing. Validity of the scoring algorithm was tested through (1) Pearson's correlations between FoodMATS scores and facility sponsorship dollars, and (2) sequential multiple regression for predicting "Least Healthy" food sales from FoodMATS scores. Inter-rater reliability was very good to excellent (κ = 0.88-1.00, p < 0.001; ICC = 0.97, p < 0.001). There was a strong positive correlation between FoodMATS scores and food sponsorship dollars, after controlling for facility size (r = 0.86, p < 0.001). The FoodMATS score explained 14% of the

  20. Validity of the Special Needs Education Assessment Tool (SNEAT), a Newly Developed Scale for Children with Disabilities.

    PubMed

    Kohara, Aiko; Han, ChangWan; Kwon, HaeJin; Kohzuki, Masahiro

    2015-11-01

    The improvement of the quality of life (QOL) of children with disabilities has been considered important. Therefore, the Special Needs Education Assessment Tool (SNEAT) was developed based on the concept of QOL to objectively evaluate the educational outcome of children with disabilities. SNEAT consists of 11 items in three domains: physical functioning, mental health, and social functioning. This study aimed to verify the reliability and construct validity of SNEAT using 93 children collected from the classes on independent activities of daily living for children with disabilities in Okinawa Prefecture between October and November 2014. Survey data were collected in a longitudinal prospective cohort study. The reliability of SNEAT was verified via the internal consistency method and the test-pretest method; both the coefficient of Cronbach's α and the intra-class correlation coefficient were over 0.7. The validity of SNEAT was also verified via one-way repeated-measures ANOVA and the latent growth curve model. The scores of all the items and domains and the total scores obtained from one-way repeated-measures ANOVA were the same as the predicted scores. SNEAT is valid based on its goodness-of-fit values obtained using the latent growth curve model, where the values of comparative fit index (0.983) and root mean square error of approximation (0.062) were within the goodness-of-fit range. These results indicate that SNEAT has high reliability and construct validity and may contribute to improve QOL of children with disabilities in the classes on independent activities of daily living for children with disabilities.

  1. Validity of the growth model of the 'computerized visual perception assessment tool for Chinese characters structures'.

    PubMed

    Wu, Huey-Min; Li, Cheng-Hsaun; Kuo, Bor-Chen; Yang, Yu-Mao; Lin, Chin-Kai; Wan, Wei-Hsiang

    2017-08-01

    Morphological awareness is the foundation for the important developmental skills involved with vocabulary, as well as understanding the meaning of words, orthographic knowledge, reading, and writing. Visual perception of space and radicals in two-dimensional positions of Chinese characters' morphology is very important in identifying Chinese characters. The important predictive variables of special and visual perception in Chinese characters identification were investigated in the growth model in this research. The assessment tool is the "Computerized Visual Perception Assessment Tool for Chinese Characters Structures" developed by this study. There are two constructs, basic stroke and character structure. In the basic stroke, there are three subtests of one, two, and more than three strokes. In the character structure, there are three subtests of single-component character, horizontal-compound character, and vertical-compound character. This study used purposive sampling. In the first year, 551 children 4-6 years old participated in the study and were monitored for one year. In the second year, 388 children remained in the study and the successful follow-up rate was 70.4%. This study used a two-wave cross-lagged panel design to validate the growth model of the basic stroke and the character structure. There was significant correlation of the basic stroke and the character structure at different time points. The abilities in the basic stroke and in the character structure steadily developed over time for preschool children. Children's knowledge of the basic stroke effectively predicted their knowledge of the basic stroke and the character structure. Copyright © 2017 Elsevier Ltd. All rights reserved.

  2. Risk assessment tools to identify women with increased risk of osteoporotic fracture: complexity or simplicity? A systematic review.

    PubMed

    Rubin, Katrine Hass; Friis-Holmberg, Teresa; Hermann, Anne Pernille; Abrahamsen, Bo; Brixen, Kim

    2013-08-01

    A huge number of risk assessment tools have been developed. Far from all have been validated in external studies, more of them have absence of methodological and transparent evidence, and few are integrated in national guidelines. Therefore, we performed a systematic review to provide an overview of existing valid and reliable risk assessment tools for prediction of osteoporotic fractures. Additionally, we aimed to determine if the performance of each tool was sufficient for practical use, and last, to examine whether the complexity of the tools influenced their discriminative power. We searched PubMed, Embase, and Cochrane databases for papers and evaluated these with respect to methodological quality using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS) checklist. A total of 48 tools were identified; 20 had been externally validated, however, only six tools had been tested more than once in a population-based setting with acceptable methodological quality. None of the tools performed consistently better than the others and simple tools (i.e., the Osteoporosis Self-assessment Tool [OST], Osteoporosis Risk Assessment Instrument [ORAI], and Garvan Fracture Risk Calculator [Garvan]) often did as well or better than more complex tools (i.e., Simple Calculated Risk Estimation Score [SCORE], WHO Fracture Risk Assessment Tool [FRAX], and Qfracture). No studies determined the effectiveness of tools in selecting patients for therapy and thus improving fracture outcomes. High-quality studies in randomized design with population-based cohorts with different case mixes are needed. Copyright © 2013 American Society for Bone and Mineral Research.

  3. The East London glaucoma prediction score: web-based validation of glaucoma risk screening tool

    PubMed Central

    Stephen, Cook; Benjamin, Longo-Mbenza

    2013-01-01

    AIM It is difficult for Optometrists and General Practitioners to know which patients are at risk. The East London glaucoma prediction score (ELGPS) is a web based risk calculator that has been developed to determine Glaucoma risk at the time of screening. Multiple risk factors that are available in a low tech environment are assessed to provide a risk assessment. This is extremely useful in settings where access to specialist care is difficult. Use of the calculator is educational. It is a free web based service. Data capture is user specific. METHOD The scoring system is a web based questionnaire that captures and subsequently calculates the relative risk for the presence of Glaucoma at the time of screening. Three categories of patient are described: Unlikely to have Glaucoma; Glaucoma Suspect and Glaucoma. A case review methodology of patients with known diagnosis is employed to validate the calculator risk assessment. RESULTS Data from the patient records of 400 patients with an established diagnosis has been captured and used to validate the screening tool. The website reports that the calculated diagnosis correlates with the actual diagnosis 82% of the time. Biostatistics analysis showed: Sensitivity = 88%; Positive predictive value = 97%; Specificity = 75%. CONCLUSION Analysis of the first 400 patients validates the web based screening tool as being a good method of screening for the at risk population. The validation is ongoing. The web based format will allow a more widespread recruitment for different geographic, population and personnel variables. PMID:23550097

  4. Judging the Neonatal Abstinence Syndrome Assessment Tools to Guide Future Tool Development: The use of Clinimetrics as Opposed to Psychometrics.

    PubMed

    Westgate, Philip M; Gomez-Pomar, Enrique

    2017-01-01

    In the face of the current Neonatal Abstinence Syndrome (NAS) epidemic, there is considerable variability in the assessment and management of infants with NAS. In this manuscript, we particularly focus on NAS assessment, with special attention given to the popular Finnegan Neonatal Abstinence Score (FNAS). A major instigator of the problem of variable practices is that multiple modified versions of the FNAS exist and continue to be proposed, including shortened versions. Furthermore, the validity of such assessment tools has been questioned, and as a result, the need for better tools has been suggested. The ultimate purpose of this manuscript, therefore, is to increase researchers' and clinicians' understanding on how to judge the usefulness of NAS assessment tools in order to guide future tool development and to reduce variable practices. In short, we suggest that judgment of NAS assessment tools should be made on a clinimetrics viewpoint as opposed to psychometrically. We provide examples, address multiple issues that must be considered, and discuss future tool development. Furthermore, we urge researchers and clinicians to come together, utilizing their knowledge and experience, to assess the utility and practicality of existing assessment tools and to determine if one or more new or modified tools are needed with the goal of increased agreement on the assessment of NAS in practice.

  5. Development and validation of a tool to assess knowledge and attitudes towards generic medicines among students in Greece: The ATtitude TOwards GENerics (ATTOGEN) questionnaire

    PubMed Central

    Katsari, Vasiliki; Niakas, Dimitris

    2017-01-01

    Introduction The use of generic medicines is a cost-effective policy, often dictated by fiscal restraints. To our knowledge, no fully validated tool exploring the students’ knowledge and attitudes towards generic medicines exists. The aim of our study was to develop and validate a questionnaire exploring the knowledge and attitudes of M.Sc. in Health Care Management students and recent alumni’s towards generic drugs in Greece. Materials and methods The development of the questionnaire was a result of literature review and pilot-testing of its preliminary versions to researchers and students. The final version of the questionnaire contains 18 items measuring the respondents’ knowledge and attitude towards generic medicines on a 5-point Likert scale. Given the ordinal nature of the data, ordinal alpha and polychoric correlations were computed. The sample was randomly split into two halves. Exploratory factor analysis, performed in the first sample, was used for the creation of multi-item scales. Confirmatory factor analysis and Generalized Linear Latent and Mixed Model analysis (GLLAMM) with the use of the rating scale model were used in the second sample to assess goodness of fit. An assessment of internal consistency reliability, test-retest reliability, and construct validity was also performed. Results Among 1402 persons contacted, 986 persons completed our questionnaire (response rate = 70.3%). Overall Cronbach’s alpha was 0.871. The conjoint use of exploratory and confirmatory factor analysis resulted in a six-scale model, which seemed to fit the data well. Five of the six scales, namely trust, drug quality, state audit, fiscal impact and drug substitution were found to be valid and reliable, while the knowledge scale suffered only from low inter-scale correlations and a ceiling effect. However, the subsequent confirmatory factor and GLLAMM analyses indicated a good fit of the model to the data. Conclusions The ATTOGEN instrument proved to be a reliable

  6. Development and validation of a tool to assess knowledge and attitudes towards generic medicines among students in Greece: The ATtitude TOwards GENerics (ATTOGEN) questionnaire.

    PubMed

    Domeyer, Philip J; Aletras, Vassilis; Anagnostopoulos, Fotios; Katsari, Vasiliki; Niakas, Dimitris

    2017-01-01

    The use of generic medicines is a cost-effective policy, often dictated by fiscal restraints. To our knowledge, no fully validated tool exploring the students' knowledge and attitudes towards generic medicines exists. The aim of our study was to develop and validate a questionnaire exploring the knowledge and attitudes of M.Sc. in Health Care Management students and recent alumni's towards generic drugs in Greece. The development of the questionnaire was a result of literature review and pilot-testing of its preliminary versions to researchers and students. The final version of the questionnaire contains 18 items measuring the respondents' knowledge and attitude towards generic medicines on a 5-point Likert scale. Given the ordinal nature of the data, ordinal alpha and polychoric correlations were computed. The sample was randomly split into two halves. Exploratory factor analysis, performed in the first sample, was used for the creation of multi-item scales. Confirmatory factor analysis and Generalized Linear Latent and Mixed Model analysis (GLLAMM) with the use of the rating scale model were used in the second sample to assess goodness of fit. An assessment of internal consistency reliability, test-retest reliability, and construct validity was also performed. Among 1402 persons contacted, 986 persons completed our questionnaire (response rate = 70.3%). Overall Cronbach's alpha was 0.871. The conjoint use of exploratory and confirmatory factor analysis resulted in a six-scale model, which seemed to fit the data well. Five of the six scales, namely trust, drug quality, state audit, fiscal impact and drug substitution were found to be valid and reliable, while the knowledge scale suffered only from low inter-scale correlations and a ceiling effect. However, the subsequent confirmatory factor and GLLAMM analyses indicated a good fit of the model to the data. The ATTOGEN instrument proved to be a reliable and valid tool, suitable for assessing students' knowledge and

  7. Validating a tool to measure auxiliary nurse midwife and nurse motivation in rural Nepal.

    PubMed

    Morrison, Joanna; Batura, Neha; Thapa, Rita; Basnyat, Regina; Skordis-Worrall, Jolene

    2015-05-12

    A global shortage of health workers in rural areas increases the salience of motivating and supporting existing health workers. Understandings of motivation may vary in different settings, and it is important to use measurement methods that are contextually appropriate. We identified a measurement tool, previously used in Kenya, and explored its validity and reliability to measure the motivation of auxiliary nurse midwives (ANM) and staff nurses (SN) in rural Nepal. Qualitative and quantitative methods were used to assess the content validity, the construct validity, the internal consistency and the reliability of the tool. We translated the tool into Nepali and it was administered to 137 ANMs and SNs in three districts. We collected qualitative data from 78 nursing personnel and district- and central-level stakeholders using interviews and focus group discussions. We calculated motivation scores for ANMs and SNs using the quantitative data and conducted statistical tests for validity and reliability. Motivation scores were compared with qualitative data. Descriptive exploratory analysis compared mean motivation scores by ANM and SN sociodemographic characteristics. The concept of self-efficacy was added to the tool before data collection. Motivation was revealed through conscientiousness. Teamwork and the exertion of extra effort were not adequately captured by the tool, but important in illustrating motivation. The statement on punctuality was problematic in quantitative analysis, and attendance was more expressive of motivation. The calculated motivation scores usually reflected ANM and SN interview data, with some variation in other stakeholder responses. The tool scored within acceptable limits in validity and reliability testing and was able to distinguish motivation of nursing personnel with different sociodemographic characteristics. We found that with minor modifications, the tool provided valid and internally consistent measures of motivation among ANMs

  8. Ethical decision-making climate in the ICU: theoretical framework and validation of a self-assessment tool.

    PubMed

    Van den Bulcke, Bo; Piers, Ruth; Jensen, Hanne Irene; Malmgren, Johan; Metaxa, Victoria; Reyners, Anna K; Darmon, Michael; Rusinova, Katerina; Talmor, Daniel; Meert, Anne-Pascale; Cancelliere, Laura; Zubek, Làszló; Maia, Paolo; Michalsen, Andrej; Decruyenaere, Johan; Kompanje, Erwin J O; Azoulay, Elie; Meganck, Reitske; Van de Sompel, Ariëlla; Vansteelandt, Stijn; Vlerick, Peter; Vanheule, Stijn; Benoit, Dominique D

    2018-02-23

    Literature depicts differences in ethical decision-making (EDM) between countries and intensive care units (ICU). To better conceptualise EDM climate in the ICU and to validate a tool to assess EDM climates. Using a modified Delphi method, we built a theoretical framework and a self-assessment instrument consisting of 35 statements. This Ethical Decision-Making Climate Questionnaire (EDMCQ) was developed to capture three EDM domains in healthcare: interdisciplinary collaboration and communication; leadership by physicians; and ethical environment. This instrument was subsequently validated among clinicians working in 68 adult ICUs in 13 European countries and the USA. Exploratory and confirmatory factor analysis was used to determine the structure of the EDM climate as perceived by clinicians. Measurement invariance was tested to make sure that variables used in the analysis were comparable constructs across different groups. Of 3610 nurses and 1137 physicians providing ICU bedside care, 2275 (63.1%) and 717 (62.9%) participated respectively. Statistical analyses revealed that a shortened 32-item version of the EDMCQ scale provides a factorial valid measurement of seven facets of the extent to which clinicians perceive an EDM climate: self-reflective and empowering leadership by physicians; practice and culture of open interdisciplinary reflection; culture of not avoiding end-of-life decisions; culture of mutual respect within the interdisciplinary team; active involvement of nurses in end-of-life care and decision-making; active decision-making by physicians; and practice and culture of ethical awareness. Measurement invariance of the EDMCQ across occupational groups was shown, reflecting that nurses and physicians interpret the EDMCQ items in a similar manner. The 32-item version of the EDMCQ might enrich the EDM climate measurement, clinicians' behaviour and the performance of healthcare organisations. This instrument offers opportunities to develop tailored ICU

  9. The Universal Design for Play Tool: Establishing Validity and Reliability

    ERIC Educational Resources Information Center

    Ruffino, Amy Goetz; Mistrett, Susan G.; Tomita, Machiko; Hajare, Poonam

    2006-01-01

    The Universal Design for Play (UDP) Tool is an instrument designed to evaluate the presence of universal design (UD) features in toys. This study evaluated its psychometric properties, including content validity, construct validity, and test-retest reliability. The UDP tool was designed to assist in selecting toys most appropriate for children…

  10. An Extended Validity Argument for Assessing Feedback Culture.

    PubMed

    Rougas, Steven; Clyne, Brian; Cianciolo, Anna T; Chan, Teresa M; Sherbino, Jonathan; Yarris, Lalena M

    2015-01-01

    NEGEA 2015 CONFERENCE ABSTRACT (EDITED): Measuring an Organization's Culture of Feedback: Can It Be Done? Steven Rougas and Brian Clyne. CONSTRUCT: This study sought to develop a construct for measuring formative feedback culture in an academic emergency medicine department. Four archetypes (Market, Adhocracy, Clan, Hierarchy) reflecting an organization's values with respect to focus (internal vs. external) and process (flexibility vs. stability and control) were used to characterize one department's receptiveness to formative feedback. The prevalence of residents' identification with certain archetypes served as an indicator of the department's organizational feedback culture. New regulations have forced academic institutions to implement wide-ranging changes to accommodate competency-based milestones and their assessment. These changes challenge residencies that use formative feedback from faculty as a major source of data for determining training advancement. Though various approaches have been taken to improve formative feedback to residents, there currently exists no tool to objectively measure the organizational culture that surrounds this process. Assessing organizational culture, commonly used in the business sector to represent organizational health, may help residency directors gauge their program's success in fostering formative feedback. The Organizational Culture Assessment Instrument (OCAI) is widely used, extensively validated, applicable to survey research, and theoretically based and may be modifiable to assess formative feedback culture in the emergency department. Using a modified Delphi technique and several iterations of focus groups amongst educators at one institution, four of the original six OCAI domains (which each contain 4 possible responses) were modified to create a 16-item Formative Feedback Culture Tool (FFCT) that was administered to 26 residents (response rate = 55%) at a single academic emergency medicine department. The mean

  11. Assessing the Validity and Reliability of the Peristomal Skin Lesion Assessment Instrument Adapted for Use in Turkey.

    PubMed

    Ay, Ali; Bulut, Hulya

    2015-08-01

    Many ostomy patients experience peristomal skin lesions. A descriptive study was conducted to assess the validity, usability, and reliability of the Peristomal Skin Lesions Assessment instrument (SACS instrument) adapted to Turkish from English. The SACS Instrument consists of 2 main assessments: lesion type (utilizing definitions and photographs) and lesion area by location around the ostomy. The study was performed in 2 stages: 1) the SACS language was changed and its content validity established; and 2) the instrument\\'92s content validity and inter-observer agreement (consistency) were determined among pairs of nurses who used the tool to assess peristomal skin lesions. Patients (included if they were >18 years old and receiving treatment/observation at 1 of the 4 participating stomatherapy units) and 8 stomatherapy nurses also completed appropriate sociodemographic questionnaires. Of the 393 patients screened during the 7-month study, 100 (average age 56.74 \\'b1 14.03 years, 55 men) participated; most (79) had a planned operation. A little more than half (59) of the patients had colorectal cancer and 28 had their stoma site marked preoperatively by a stomatherapy nurse. The most common peristomal skin lesion risk factors were having an ileostomy and unplanned surgery. The content validity index of the entire Turkish SACS instrument was 1, and the inter-observer agreement Kappa statistic was very good (K = 0.90, 95% CI 0.80- 0.99). Individual SACS item K values ranged from K = 0.84 (95% CI 0.63\\'961) to K = 1 (95% CI 1). Most (62.5%) nurses found the terms and pictures used in the SACS classification adequate and suitable, and 50% believed the Turkish version of the SACS instrument was a valid and suitable assessment tool for use by Turkish stomatherapy nurses. Validity and reliability studies involving larger and more diverse patient and nurse samples are warranted.

  12. Building validation tools for knowledge-based systems

    NASA Technical Reports Server (NTRS)

    Stachowitz, R. A.; Chang, C. L.; Stock, T. S.; Combs, J. B.

    1987-01-01

    The Expert Systems Validation Associate (EVA), a validation system under development at the Lockheed Artificial Intelligence Center for more than a year, provides a wide range of validation tools to check the correctness, consistency and completeness of a knowledge-based system. A declarative meta-language (higher-order language), is used to create a generic version of EVA to validate applications written in arbitrary expert system shells. The architecture and functionality of EVA are presented. The functionality includes Structure Check, Logic Check, Extended Structure Check (using semantic information), Extended Logic Check, Semantic Check, Omission Check, Rule Refinement, Control Check, Test Case Generation, Error Localization, and Behavior Verification.

  13. Validation of the first peoples cultural capability measurement tool with undergraduate health students: A descriptive cohort study.

    PubMed

    West, Roianne; Mills, Kyly; Rowland, Dale; Creedy, Debra K

    2018-05-01

    Health professional graduates require the capacity to work safely, both clinically and culturally, when delivering care to Indigenous peoples worldwide. In the Australian context, the Aboriginal and Torres Strait Islander Health Curriculum Framework (The Framework) provides guidance for health professional programs to integrate, teach and assess Aboriginal and Torres Strait Islander peoples' (First Peoples) health content. There is, however, a lack of validated tools that measure the development of students' cultural capabilities. To validate the Cultural Capability Measurement Tool with a cohort of health professional students. A descriptive cohort design was used. All students (N = 753) enrolled in a discrete First Peoples Health course at an Australian university were invited to complete the Cultural Capability Measurement Tool. The tool was tested for reliability, content and construct validity using confirmatory factor analysis; and concurrent validity using and the Cultural Understanding Self-Assessment Tool. A sample of 418 (73% response rate) was recruited. Most participants were enrolled in the Bachelor of Nursing program (n = 369, 82%). The Cultural Capability Measurement Tool had a Cronbach's alpha coefficient of 0.86. A five-factor solution was confirmed which reflected the cultural capability domains and accounted for 51% of the variance. Scores correlated with students' cultural understanding (r = 0.28, p < 0.001). Successful implementation of The Framework requires instruments to measure changes in students' cultural capabilities. Measuring nursing students' cultural capabilities can inform their development, identify areas of strengths and deficits for educators, and will ultimately contribute to the development of a culturally safe nursing workforce. Copyright © 2018 Elsevier Ltd. All rights reserved.

  14. Is a sphygmomanometer a valid and reliable tool to measure the isometric strength of hip muscles? A systematic review.

    PubMed

    Toohey, Liam Anthony; De Noronha, Marcos; Taylor, Carolyn; Thomas, James

    2015-02-01

    Muscle strength measurement is a key component of physiotherapists' assessment and is frequently used as an outcome measure. A sphygmomanometer is an instrument commonly used to measure blood pressure that can be potentially used as a tool to assess isometric muscle strength. To systematically review the evidence on the reliability and validity of a sphygmomanometer for measuring isometric strength of hip muscles. A literature search was conducted across four databases. Studies were eligible if they presented data on reliability and/or validity, used a sphygmomanometer to measure isometric muscle strength of the hip region, and were peer reviewed. The individual studies were evaluated for quality using a standardized critical appraisal tool. A total of 644 articles were screened for eligibility, with five articles chosen for inclusion. The use of a sphygmomanometer to objectively assess isometric muscle strength of the hip muscles appears to be reliable with intraclass correlation coefficient values ranging from 0.66 to 0.94 in elderly and young populations. No studies were identified that have assessed the validity of a sphygmomanometer. The sphygmomanometer appears to be reliable for assessment of isometric muscle strength around the hip joint, but further research is warranted to establish its validity.

  15. Validity of a quantitative clinical measurement tool of trunk posture in idiopathic scoliosis.

    PubMed

    Fortin, Carole; Feldman, Debbie E; Cheriet, Farida; Labelle, Hubert

    2010-09-01

    Concurrent validity between postural indices obtained from digital photographs (two-dimensional [2D]), surface topography imaging (three-dimensional [3D]), and radiographs. To assess the validity of a quantitative clinical postural assessment tool of the trunk based on photographs (2D) as compared to a surface topography system (3D) as well as indices calculated from radiographs. To monitor progression of scoliosis or change in posture over time in young persons with idiopathic scoliosis (IS), noninvasive and nonionizing methods are recommended. In a clinical setting, posture can be quite easily assessed by calculating key postural indices from photographs. Quantitative postural indices of 70 subjects aged 10 to 20 years old with IS (Cobb angle, 15 degrees -60 degrees) were measured from photographs and from 3D trunk surface images taken in the standing position. Shoulder, scapula, trunk list, pelvis, scoliosis, and waist angles indices were calculated with specially designed software. Frontal and sagittal Cobb angles and trunk list were also calculated on radiographs. The Pearson correlation coefficients (r) was used to estimate concurrent validity of the 2D clinical postural tool of the trunk with indices extracted from the 3D system and with those obtained from radiographs. The correlation between 2D and 3D indices was good to excellent for shoulder, pelvis, trunk list, and thoracic scoliosis (0.81>r<0.97; P<0.01) but fair to moderate for thoracic kyphosis, lumbar lordosis, and thoracolumbar or lumbar scoliosis (0.30>r<0.56; P<0.05). The correlation between 2D and radiograph spinal indices was fair to good (-0.33 to -0.80 with Cobb angles and 0.76 for trunk list; P<0.05). This tool will facilitate clinical practice by monitoring trunk posture among persons with IS. Further, it may contribute to a reduction in the use of radiographs to monitor scoliosis progression.

  16. Challenges of NDE simulation tool validation, optimization, and utilization for composites

    NASA Astrophysics Data System (ADS)

    Leckey, Cara A. C.; Seebo, Jeffrey P.; Juarez, Peter

    2016-02-01

    Rapid, realistic nondestructive evaluation (NDE) simulation tools can aid in inspection optimization and prediction of inspectability for advanced aerospace materials and designs. NDE simulation tools may someday aid in the design and certification of aerospace components; potentially shortening the time from material development to implementation by industry and government. Furthermore, ultrasound modeling and simulation are expected to play a significant future role in validating the capabilities and limitations of guided wave based structural health monitoring (SHM) systems. The current state-of-the-art in ultrasonic NDE/SHM simulation is still far from the goal of rapidly simulating damage detection techniques for large scale, complex geometry composite components/vehicles containing realistic damage types. Ongoing work at NASA Langley Research Center is focused on advanced ultrasonic simulation tool development. This paper discusses challenges of simulation tool validation, optimization, and utilization for composites. Ongoing simulation tool development work is described along with examples of simulation validation and optimization challenges that are more broadly applicable to all NDE simulation tools. The paper will also discuss examples of simulation tool utilization at NASA to develop new damage characterization methods for composites, and associated challenges in experimentally validating those methods.

  17. School environment assessment tools to address behavioural risk factors of non-communicable diseases: A scoping review.

    PubMed

    Saluja, Kiran; Rawal, Tina; Bassi, Shalini; Bhaumik, Soumyadeep; Singh, Ankur; Park, Min Hae; Kinra, Sanjay; Arora, Monika

    2018-06-01

    We aimed to identify, describe and analyse school environment assessment (SEA) tools that address behavioural risk factors (unhealthy diet, physical inactivity, tobacco and alcohol consumption) for non-communicable diseases (NCD). We searched in MEDLINE and Web of Science, hand-searched reference lists and contacted experts. Basic characteristics, measures assessed and measurement properties (validity, reliability, usability) of identified tools were extracted. We narratively synthesized the data and used content analysis to develop a list of measures used in the SEA tools. Twenty-four SEA tools were identified, mostly from developed countries. Out of these, 15 were questionnaire based, 8 were checklists or observation based tools and one tool used a combined checklist/observation based and telephonic questionnaire approach. Only 1 SEA tool had components related to all the four NCD risk factors, 2 SEA tools has assessed three NCD risk factors (diet/nutrition, physical activity, tobacco), 10 SEA tools has assessed two NCD risk factors (diet/nutrition and physical activity) and 11 SEA tools has assessed only one of the NCD risk factor. Several measures were used in the tools to assess the four NCD risk factors, but tobacco and alcohol was sparingly included. Measurement properties were reported for 14 tools. The review provides a comprehensive list of measures used in SEA tools which could be a valuable resource to guide future development of such tools. A valid and reliable SEA tool which could simultaneously evaluate all NCD risk factors, that has been tested in different settings with varying resource availability is needed.

  18. Asking For Help is Helpful: Validation of a Brief Lifestyle and Mood Assessment Tool in Primary Health Care

    PubMed Central

    Goodyear-Smith, Felicity; Arroll, Bruce; Coupe, Nicole

    2009-01-01

    PURPOSE The short, validated, self-administered, Case-finding and Help Assessment Tool (CHAT) for lifestyle and mental health assessment of adult patients in primary health care addresses inactivity, tobacco use, alcohol and other drug misuse, problem gambling, depression, anxiety and stress, abuse, and anger problems. For each issue patients are asked whether they would like help, either during the consultation or at a later date. This study aims to assess the value of the help question. METHODS Validation of the CHAT was conducted according to the STAndards for Reporting of Diagnostic accuracy studies statement for diagnostic tests. The setting was Auckland primary care practices with populations ranging from socioeconomically advantaged to deprived. Participants were 755 consecutive primary care patients who completed the CHAT plus the help question and reference standards. Sensitivity, specificity, and likelihood ratios with and without the addition of help the question were calculated. RESULTS Sensitivity ranged from 80% to 98% for the more-common conditions (depression, nicotine dependency, anxiety, problematic drinking). For each condition, specificity increased with the addition of the help question: depression increased from 73% to 98%; anxiety 77% to 99%; drinking 85% to 99%; verbal anger 92% to 99%; verbal abuse 97% to 99%; problematic drinking and gambling 98% to 99%. CONCLUSIONS The help question increased specificity without compromising sensitivity and reduced false positives, thereby increasing the positive predictive value. It allowed patients with comorbidities to prioritize issues they wished to address, indicate their readiness to change, promote self-determination, and give the clinician an indication of which topics to pursue. PMID:19433841

  19. Development of a self-assessment teamwork tool for use by medical and nursing students.

    PubMed

    Gordon, Christopher J; Jorm, Christine; Shulruf, Boaz; Weller, Jennifer; Currie, Jane; Lim, Renee; Osomanski, Adam

    2016-08-24

    Teamwork training is an essential component of health professional student education. A valid and reliable teamwork self-assessment tool could assist students to identify desirable teamwork behaviours with the potential to promote learning about effective teamwork. The aim of this study was to develop and evaluate a self-assessment teamwork tool for health professional students for use in the context of emergency response to a mass casualty. The authors modified a previously published teamwork instrument designed for experienced critical care teams for use with medical and nursing students involved in mass casualty simulations. The 17-item questionnaire was administered to students immediately following the simulations. These scores were used to explore the psychometric properties of the tool, using Exploratory and Confirmatory Factor Analysis. 202 (128 medical and 74 nursing) students completed the self-assessment teamwork tool for students. Exploratory factor analysis revealed 2 factors (5 items - Teamwork coordination and communication; 4 items - Information sharing and support) and these were justified with confirmatory factor analysis. Internal consistency was 0.823 for Teamwork coordination and communication, and 0.812 for Information sharing and support. These data provide evidence to support the validity and reliability of the self-assessment teamwork tool for students This self-assessment tool could be of value to health professional students following team training activities to help them identify the attributes of effective teamwork.

  20. Assessment Tools for Evaluation of Oral Feeding in Infants Younger Than 6 Months.

    PubMed

    Pados, Britt F; Park, Jinhee; Estrem, Hayley; Awotwi, Araba

    2016-04-01

    Feeding difficulty is common in infants younger than 6 months. Identification of infants in need of specialized treatment is critical to ensure appropriate nutrition and feeding skill development. Valid and reliable assessment tools help clinicians objectively evaluate feeding. To identify and evaluate assessment tools available for clinical assessment of bottle- and breastfeeding in infants younger than 6 months. CINAHL, HaPI, PubMed, and Web of Science were searched for "infant feeding" and "assessment tool." The literature (n = 237) was reviewed for relevant assessment tools. A secondary search was conducted in CINAHL and PubMed for additional literature on identified tools. Eighteen assessment tools met inclusion criteria. Of these, 7 were excluded because of limited available literature or because they were intended for use with a specific diagnosis or in research only. There are 11 assessment tools available for clinical practice. Only 2 of these were intended for bottle-feeding. All 11 indicated that they were appropriate for use with breastfeeding. None of the available tools have adequate psychometric development and testing. All of the tools should be used with caution. The Early Feeding Skills Assessment and Bristol Breastfeeding Assessment Tool had the most supportive psychometric development and testing. Feeding assessment tools need to be developed and tested to guide optimal clinical care of infants from birth through 6 months. A tool that assesses both bottle- and breastfeeding would allow for consistent assessment across feeding methods.

  1. Validation of the MISSCARE-BRASIL survey - A tool to assess missed nursing care.

    PubMed

    Siqueira, Lillian Dias Castilho; Caliri, Maria Helena Larcher; Haas, Vanderlei José; Kalisch, Beatrice; Dantas, Rosana Aparecida Spadoti

    2017-12-21

    to analyze the metric validity and reliability properties of the MISSCARE-BRASIL survey. methodological research conducted by assessing construct validity and reliability via confirmatory factor analysis, known-groups validation, convergent construct validation, analysis of internal consistency and test-retest reliability. The sample consisted of 330 nursing professionals, of whom 86 participated in the retest phase. of the 330 participants, 39.7% were aides, 33% technicians, 20.9% nurses, and 6.4% nurses with administrative roles. Confirmatory factorial analysis demonstrated that the Brazilian Portuguese version of the instrument is adequately adjusted to the dimensional structure the scale authors originally proposed. The correlation between "satisfaction with position/role" and "satisfaction with teamwork" and the survey's missed care variables was moderate (Spearman's coefficient =0.35; p<0.001). The results of the Student's t-test indicated known-group validity. Professionals from closed units reported lower levels of missed care in comparison with the other units. The reliability showed a strong correlation, with the exception of "institutional management/leadership style" (intraclass correlation coefficient (ICC)=0.15; p=0.04). The internal consistency was adequate (Cronbach's alpha was greater than 0.70). the MISSCARE-BRASIL was valid and reliable in the group studied. The application of the MISSCARE-BRASIL can contribute to identifying solutions for missed nursing care.

  2. A Comparative Study of Adolescent Risk Assessment Instruments: Predictive and Incremental Validity

    ERIC Educational Resources Information Center

    Welsh, Jennifer L.; Schmidt, Fred; McKinnon, Lauren; Chattha, H. K.; Meyers, Joanna R.

    2008-01-01

    Promising new adolescent risk assessment tools are being incorporated into clinical practice but currently possess limited evidence of predictive validity regarding their individual and/or combined use in risk assessments. The current study compares three structured adolescent risk instruments, Youth Level of Service/Case Management Inventory…

  3. Assessing attachment in school-aged children: Do the School-Age Assessment of Attachment and Family Drawings work together as complementary tools?

    PubMed

    Carr-Hopkins, Rebecca; De Burca, Calem; Aldridge, Felicity A

    2017-07-01

    Our goal was to identify an assessment package that could improve treatment planning for troubled children and their families. To assess the validity of our tools, we tested the relations among the School-Age Assessment of Attachment, the Family Drawing and children's risk status. We used the Dynamic-Maturational Model of Attachment and Adaptation to interpret the assessments in the hope of identifying a gradient of risk, and explore whether a new coding method improved the validity of Family Drawings and their utility as a tool to complement the School-Age Assessment of Attachment. The participants were 89 children, aged between 5 and 12 years; 32 children were involved with mental health services or child protection. Each child completed a School-Age Assessment of Attachment and a Family Drawing. Both assessments differentiated between clinical and normative referrals with moderate effect sizes when dichotomizing risk versus non-risk attachment. When the analysis incorporated a gradient of six attachment classifications, the effect sizes decreased, but specificity of risk increased. The School-Age Assessment of Attachment had greater validity for discriminating risk, and type of risk, than the Family Drawings. With a School-Age Assessment of Attachment and family history, the Family Drawing can provide information about distress that some children do not provide verbally. Integration of the two assessment tools alongside information about parental and family functioning appears to be the key to formulating children's problems.

  4. Testing the validity, reliability and utility of the Self-Administration of Medication (SAM) tool in patients undergoing rehabilitation.

    PubMed

    Anderson, Jessica; Manias, Elizabeth; Kusljic, Snezana; Finch, Sue

    2014-01-01

    Determination of patients' ability to self-administer medications in the hospital has largely been determined using the subjective judgment of health professionals. To examine the validity, reliability and utility of the Self-Administration of Medication (SAM) tool as an objective means to determine patients' ability to self-administer in a rehabilitation unit of a public teaching hospital in Melbourne, Australia. To assess validity of the SAM tool, associations were examined between the total SAM tool score and of the patients' competence to self-administer from the perceptions of the tool administrator, patients and nurses. Validity also was determined from a principal component analysis. Pearson correlations were calculated for how SAM scores related to scores obtained from the Functional Independence Measure (FIM) and Barthel Score Index (BSI). To assess the SAM tool's reliability, a Cronbach's alpha coefficient was calculated. Utility of the SAM tool was evidenced by documenting its administration time. One hundred patients participated in this study. The SAM tool had a Cronbach's alpha coefficient of 0.75 and took a mean time of 5.36 min to complete. The capability to self-medicate section of the SAM tool had strong correlations with the FIM (r = 0.485) and BSI (r = 0.472) data, respectively, and the total SAM tool had moderate and strong correlations with the nurses' (r = 0.315) and tool administrator's (r = 0.632) perceptions of patients' ability to self-administer, respectively. Bland-Altman and ROC curve analyses showed poor agreement between the total SAM tool score and the nurses' perceptions. The SAM tool demonstrated acceptable overall internal consistency. It only requires a short time to be completed and is more objective than seeking out health professionals' perceptions. Additional research is needed to further validate this approach to determining patients' ability to self-medicate. Crown Copyright © 2014. Published by Elsevier Inc. All

  5. The National Clinical Assessment Tool for Medical Students in the Emergency Department (NCAT-EM)

    PubMed Central

    Jung, Julianna; Franzen, Douglas; Lawson, Luan; Manthey, David; Tews, Matthew; Dubosh, Nicole; Fisher, Jonathan; Haughey, Marianne; House, Joseph B.; Trainor, Arleigh; Wald, David A.; Hiller, Katherine

    2018-01-01

    Introduction Clinical assessment of medical students in emergency medicine (EM) clerkships is a highly variable process that presents unique challenges and opportunities. Currently, clerkship directors use institution-specific tools with unproven validity and reliability that may or may not address competencies valued most highly in the EM setting. Standardization of assessment practices and development of a common, valid, specialty-specific tool would benefit EM educators and students. Methods A two-day national consensus conference was held in March 2016 in the Clerkship Directors in Emergency Medicine (CDEM) track at the Council of Residency Directors in Emergency Medicine (CORD) Academic Assembly in Nashville, TN. The goal of this conference was to standardize assessment practices and to create a national clinical assessment tool for use in EM clerkships across the country. Conference leaders synthesized the literature, articulated major themes and questions pertinent to clinical assessment of students in EM, clarified the issues, and outlined the consensus-building process prior to consensus-building activities. Results The first day of the conference was dedicated to developing consensus on these key themes in clinical assessment. The second day of the conference was dedicated to discussing and voting on proposed domains to be included in the national clinical assessment tool. A modified Delphi process was initiated after the conference to reconcile questions and items that did not reach an a priori level of consensus. Conclusion The final tool, the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM) is presented here. PMID:29383058

  6. External validation of Global Evaluative Assessment of Robotic Skills (GEARS).

    PubMed

    Aghazadeh, Monty A; Jayaratna, Isuru S; Hung, Andrew J; Pan, Michael M; Desai, Mihir M; Gill, Inderbir S; Goh, Alvin C

    2015-11-01

    We demonstrate the construct validity, reliability, and utility of Global Evaluative Assessment of Robotic Skills (GEARS), a clinical assessment tool designed to measure robotic technical skills, in an independent cohort using an in vivo animal training model. Using a cross-sectional observational study design, 47 voluntary participants were categorized as experts (>30 robotic cases completed as primary surgeon) or trainees. The trainee group was further divided into intermediates (≥5 but ≤30 cases) or novices (<5 cases). All participants completed a standardized in vivo robotic task in a porcine model. Task performance was evaluated by two expert robotic surgeons and self-assessed by the participants using the GEARS assessment tool. Kruskal-Wallis test was used to compare the GEARS performance scores to determine construct validity; Spearman's rank correlation measured interobserver reliability; and Cronbach's alpha was used to assess internal consistency. Performance evaluations were completed on nine experts and 38 trainees (14 intermediate, 24 novice). Experts demonstrated superior performance compared to intermediates and novices overall and in all individual domains (p < 0.0001). In comparing intermediates and novices, the overall performance difference trended toward significance (p = 0.0505), while the individual domains of efficiency and autonomy were significantly different between groups (p = 0.0280 and 0.0425, respectively). Interobserver reliability between expert ratings was confirmed with a strong correlation observed (r = 0.857, 95 % CI [0.691, 0.941]). Experts and participant scoring showed less agreement (r = 0.435, 95 % CI [0.121, 0.689] and r = 0.422, 95 % CI [0.081, 0.0672]). Internal consistency was excellent for experts and participants (α = 0.96, 0.98, 0.93). In an independent cohort, GEARS was able to differentiate between different robotic skill levels, demonstrating excellent construct validity. As a standardized

  7. Critical thinking skills in midwifery practice: Development of a self-assessment tool for students.

    PubMed

    Carter, Amanda G; Creedy, Debra K; Sidebotham, Mary

    2017-07-01

    Develop and test a tool designed for use by pre-registration midwifery students to self-appraise their critical thinking in practice. A descriptive cohort design was used. All students (n=164) enrolled in a three-year Bachelor of Midwifery program in Queensland, Australia. The staged model for tool development involved item generation, mapping draft items to critical thinking concepts and expert review to test content validity, pilot testing of the tool to a convenience sample of students, and psychometric testing. Students (n=126, 76.8% response rate) provided demographic details, completed the new tool, and five questions from the Motivated Strategies for Learning Questionnaire (MSLQ) via an online platform or paper version. A high content validity index score of 0.97 was achieved through expert review. Construct validity via factor analysis revealed four factors: seeks information, reflects on practice, facilitates shared decision making, and evaluates practice. The mean total score for the tool was 124.98 (SD=12.58). Total and subscale scores correlated significantly. The scale achieved good internal reliability with a Cronbach's alpha coefficient of 0.92. Concurrent validity with the MSLQ subscale was 0.35 (p<0.001). This study established the reliability and validity of the CACTiM - student version for use by pre-registration midwifery students to self-assess critical thinking in practice. Critical thinking skills are vital for safe and effective midwifery practice. Students' assessment of their critical thinking development throughout their pre-registration programme makes these skills explicit, and could guide teaching innovation to address identified deficits. The availability of a reliable and valid tool assists research into the development of critical thinking in education and practice. Crown Copyright © 2017. Published by Elsevier Ltd. All rights reserved.

  8. Development of a golf-specific load monitoring tool: Content validity and feasibility.

    PubMed

    Williams, Scott B; Gastin, Paul B; Saw, Anna E; Robertson, Sam

    2018-05-01

    Athletes often record details of their training and competitions, supported by information such as environmental conditions, travel, as well as how they felt. However, it is not known how prevalent these practices are in golfers, or how valuable this process is perceived. The purpose of this study was to develop a golf-specific load monitoring tool (GLMT), and establish the content validity and feasibility of this tool amongst high-level golfers. In the first phase of development, 21 experts were surveyed to determine the suitability of items for inclusion in the GLMT. Of the 36 items, 21 received >78% agreement, a requirement to establish content validity and for inclusion in the GLMT. Total duration was the preferred metric for golf-specific activities, whilst rating of perceived exertion (RPE) was preferred for measuring physical training. In the second phase, feasibility of the tool was assessed by surveying 13 high-level male golfers following 28-days of daily GLMT use. All items included in the GLMT were deemed feasible to record, with all players participating in the feasibility study providing high to very high ratings. Golfers responded that they would consider using a load monitoring tool of this nature long term, provided it can be completed in less than five minutes per day.

  9. The development of a multimedia online language assessment tool for young children with autism.

    PubMed

    Lin, Chu-Sui; Chang, Shu-Hui; Liou, Wen-Ying; Tsai, Yu-Show

    2013-10-01

    This study aimed to provide early childhood special education professionals with a standardized and comprehensive language assessment tool for the early identification of language learning characteristics (e.g., hyperlexia) of young children with autism. In this study, we used computer technology to develop a multi-media online language assessment tool that presents auditory or visual stimuli. This online comprehensive language assessment consists of six subtests: decoding, homographs, auditory vocabulary comprehension, visual vocabulary comprehension, auditory sentence comprehension, and visual sentence comprehension. Three hundred typically developing children and 35 children with autism from Tao-Yuan County in Taiwan aged 4-6 participated in this study. The Cronbach α values of the six subtests ranged from .64 to .97. The variance explained by the six subtests ranged from 14% to 56%, the current validity of each subtest with the Peabody Picture Vocabulary Test-Revised ranged from .21 to .45, and the predictive validity of each subtest with WISC-III ranged from .47 to .75. This assessment tool was also found to be able to accurately differentiate children with autism up to 92%. These results indicate that this assessment tool has both adequate reliability and validity. Additionally, 35 children with autism have completed the entire assessment in this study without exhibiting any extremely troubling behaviors. However, future research is needed to increase the sample size of both typically developing children and young children with autism and to overcome the technical challenges associated with internet issues. Copyright © 2013 Elsevier Ltd. All rights reserved.

  10. Assessment Tool Development for Extracurricular Smet Programs for Girls

    NASA Astrophysics Data System (ADS)

    House, Jody; Johnson, Molly; Borthwick, Geoffrey

    Many different programs have been designed to increase girls' interest in and exposure to science, mathematics, engineering, and technology (SMET). Two of these programs are discussed and contrasted in the dimensions of length, level of science content, pedagogical approach, degree of self- vs. parent-selected participants, and amount of communitybuilding content. Two different evaluation tools were used. For one program, a modified version of the University of Pittsburgh's undergraduate engineering attitude assessment survey was used. Program participants' responses were compared to those from a fifth grade, mixed-sex science class. The only gender difference found was in the area of parental encouragement. The girls in the special class were more encouraged to participate in SMET areas. For the second program, a new age-appropriate tool developed specifically for these types of programs was used, and the tool itself was evaluated. The results indicate that the new tool has construct validity. On the basis of these preliminary results, a long-term plan for the continued development of the assessment tool is outlined.

  11. Development, Sensibility, and Validity of a Systemic Autoimmune Rheumatic Disease Case Ascertainment Tool.

    PubMed

    Armstrong, Susan M; Wither, Joan E; Borowoy, Alan M; Landolt-Marticorena, Carolina; Davis, Aileen M; Johnson, Sindhu R

    2017-01-01

    Case ascertainment through self-report is a convenient but often inaccurate method to collect information. The purposes of this study were to develop, assess the sensibility, and validate a tool to identify cases of systemic autoimmune rheumatic diseases (SARD) in the outpatient setting. The SARD tool was administered to subjects sampled from specialty clinics. Determinants of sensibility - comprehensibility, feasibility, validity, and acceptability - were evaluated using a numeric rating scale from 1-7. Comprehensibility was evaluated using the Flesch Reading Ease and the Flesch-Kincaid Grade Level. Self-reported diagnoses were validated against medical records using Cohen's κ statistic. There were 141 participants [systemic lupus erythematosus (SLE), systemic sclerosis (SSc), rheumatoid arthritis, Sjögren syndrome (SS), inflammatory myositis (polymyositis/dermatomyositis; PM/DM), and controls] who completed the questionnaire. The Flesch Reading Ease score was 77.1 and the Flesch-Kincaid Grade Level was 4.4. Respondents endorsed (mean ± SD) comprehensibility (6.12 ± 0.92), feasibility (5.94 ± 0.81), validity (5.35 ± 1.10), and acceptability (3.10 ± 2.03). The SARD tool had a sensitivity of 0.91 (95% CI 0.88-0.94) and a specificity of 0.99 (95% CI 0.96-1.00). The agreement between the SARD tool and medical record was κ = 0.82 (95% CI 0.77-0.88). Subgroup analysis by SARD found κ coefficients for SLE to be κ = 0.88 (95% CI 0.79-0.97), SSc κ = 1.0 (95% CI 1.0-1.0), PM/DM κ = 0.72 (95% CI 0.49-0.95), and SS κ = 0.85 (95% CI 0.71-0.99). The screening questions had sensitivity ranging from 0.96 to 1.0 and specificity ranging from 0.88 to 1.0. This SARD case ascertainment tool has demonstrable sensibility and validity. The use of both screening and confirmatory questions confers added accuracy.

  12. Measuring patient-provider communication skills in Rwanda: Selection, adaptation and assessment of psychometric properties of the Communication Assessment Tool.

    PubMed

    Cubaka, Vincent Kalumire; Schriver, Michael; Vedsted, Peter; Makoul, Gregory; Kallestrup, Per

    2018-04-23

    To identify, adapt and validate a measure for providers' communication and interpersonal skills in Rwanda. After selection, translation and piloting of the measure, structural validity, test-retest reliability, and differential item functioning were assessed. Identification and adaptation: The 14-item Communication Assessment Tool (CAT) was selected and adapted. Content validation found all items highly relevant in the local context except two, which were retained upon understanding the reasoning applied by patients. Eleven providers and 291 patients were involved in the field-testing. Confirmatory factor analysis showed a good fit for the original one factor model. Test-retest reliability assessment revealed a mean quadratic weighted Kappa = 0.81 (range: 0.69-0.89, N = 57). The average proportion of excellent scores was 15.7% (SD: 24.7, range: 9.9-21.8%, N = 180). Differential item functioning was not observed except for item 1, which focuses on greetings, for age groups (p = 0.02, N = 180). The Kinyarwanda version of CAT (K-CAT) is a reliable and valid patient-reported measure of providers' communication and interpersonal skills. K-CAT was validated on nurses and its use on other types of providers may require further validation. K-CAT is expected to be a valuable feedback tool for providers in practice and in training. Copyright © 2018 Elsevier B.V. All rights reserved.

  13. Development of the AGREE II, part 2: assessment of validity of items and tools to support application

    PubMed Central

    Brouwers, Melissa C.; Kho, Michelle E.; Browman, George P.; Burgers, Jako S.; Cluzeau, Françoise; Feder, Gene; Fervers, Béatrice; Graham, Ian D.; Hanna, Steven E.; Makarski, Julie

    2010-01-01

    Background We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user’s manual in the β version of the AGREE II. Methods We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user’s manual and completed a survey assessing the manual. Results In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the β-AGREE II could be improved. Interpretation The validity of the items was established and the user’s manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user’s manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation. PMID:20513779

  14. Validating workplace performance assessments in health sciences students: a case study from speech pathology.

    PubMed

    McAllister, Sue; Lincoln, Michelle; Ferguson, Allison; McAllister, Lindy

    2013-01-01

    Valid assessment of health science students' ability to perform in the real world of workplace practice is critical for promoting quality learning and ultimately certifying students as fit to enter the world of professional practice. Current practice in performance assessment in the health sciences field has been hampered by multiple issues regarding assessment content and process. Evidence for the validity of scores derived from assessment tools are usually evaluated against traditional validity categories with reliability evidence privileged over validity, resulting in the paradoxical effect of compromising the assessment validity and learning processes the assessments seek to promote. Furthermore, the dominant statistical approaches used to validate scores from these assessments fall under the umbrella of classical test theory approaches. This paper reports on the successful national development and validation of measures derived from an assessment of Australian speech pathology students' performance in the workplace. Validation of these measures considered each of Messick's interrelated validity evidence categories and included using evidence generated through Rasch analyses to support score interpretation and related action. This research demonstrated that it is possible to develop an assessment of real, complex, work based performance of speech pathology students, that generates valid measures without compromising the learning processes the assessment seeks to promote. The process described provides a model for other health professional education programs to trial.

  15. The COA360: a tool for assessing the cultural competency of healthcare organizations.

    PubMed

    LaVeist, Thomas A; Relosa, Rachel; Sawaya, Nadia

    2008-01-01

    The U.S. Census Bureau projects that by 2050, non-Hispanic whites will be in the numerical minority. This rapid diversification requires healthcare organizations to pay closer attention to cross-cultural issues if they are to meet the healthcare needs of the nation and continue to maintain a high standard of care. Although scorecards and benchmarking are widely used to gauge healthcare organizations' performance in various areas, these tools have been underused in relation to cultural preparedness or initiatives. The likely reason for this is the lack of a validated tool specifically designed to examine cultural competency. Existing validated cultural competency instruments evaluate individuals, not organizations. In this article, we discuss a study to validate the Cultural Competency Organizational Assessment--360 or the COA360, an instrument designed to appraise a healthcare organization's cultural competence. The Office of Minority Health and the Joint Commission have each developed standards for measuring the cultural competency of organizations. The COA360 is designed to assess adherence to both of these sets of standards. For this validation study, we enlisted a panel of national experts. The panel rated each dimension of the COA360, and the combination of items for each of the scale's 14 dimensions was rated above 4.13 (on 5-point scale). Our conclusion points to the validity of the COA360. As such, it is a valuable tool not only for assessing a healthcare organization's cultural readiness but also for benchmarking its progress in addressing cultural and diversity issues.

  16. Tools for assessing fall risk in the elderly: a systematic review and meta-analysis.

    PubMed

    Park, Seong-Hi

    2018-01-01

    The prevention of falls among the elderly is arguably one of the most important public health issues in today's aging society. The aim of this study was to assess which tools best predict the risk of falls in the elderly. Electronic searches were performed using Medline, EMBASE, the Cochrane Library, CINAHL, etc., using the following keywords: "fall risk assessment", "elderly fall screening", and "elderly mobility scale". The QUADAS-2 was applied to assess the internal validity of the diagnostic studies. Selected studies were meta-analyzed with MetaDisc 1.4. A total of 33 studies were eligible out of the 2,321 studies retrieved from selected databases. Twenty-six assessment tools for fall risk were used in the selected articles, and they tended to vary based on the setting. The fall risk assessment tools currently used for the elderly did not show sufficiently high predictive validity for differentiating high and low fall risks. The Berg Balance scale and Mobility Interaction Fall chart showed stable and high specificity, while the Downton Fall Risk Index, Hendrich II Fall Risk Model, St. Thomas's Risk Assessment Tool in Falling elderly inpatients, Timed Up and Go test, and Tinetti Balance scale showed the opposite results. We concluded that rather than a single measure, two assessment tools used together would better evaluate the characteristics of falls by the elderly that can occur due to a multitude of factors and maximize the advantages of each for predicting the occurrence of falls.

  17. The effectiveness of environment assessment tools to guide refurbishment of Australian residential aged care facilities: A systematic review.

    PubMed

    Neylon, Samantha; Bulsara, Caroline; Hill, Anne-Marie

    2017-06-01

    To determine applicability of environment assessment tools in guiding minor refurbishments of Australian residential aged care facilities. Studies conducted in residential aged care settings using assessment tools which address the physical environment were eligible for inclusion in a systematic review. Given these studies are limited, tools which have not yet been utilised in research settings were also included. Tools were analysed using a critical appraisal screen. Forty-three publications met the inclusion criteria. Ten environment assessment tools were identified, of which four addressed all seven minor refurbishment domains of lighting, colour and contrast, sound, flooring, furniture, signage and way finding. Only one had undergone reliability and validity testing. There are four tools which may be suitable to use for minor refurbishment of Australian residential aged care facilities. Data on their reliability, validity and quality are limited. © 2017 AJA Inc.

  18. Validation of an explanatory tool for data-fused displays for high-technology future aircraft

    NASA Astrophysics Data System (ADS)

    Fletcher, Georgina C. L.; Shanks, Craig R.; Selcon, Stephen J.

    1996-05-01

    As the number of sensor and data sources in the military cockpit increases, pilots will suffer high levels of workload which could result in reduced performance and the loss of situational awareness. A DRA research program has been investigating the use of data-fused displays in decision support and has developed and laboratory-tested an explanatory tool for displaying information in air combat scenarios. The tool has been designed to provide pictorial explanations of data that maintain situational awareness by involving the pilot in the hostile aircraft threat assessment task. This paper reports a study carried out to validate the success of the explanatory tool in a realistic flight simulation facility. Aircrew were asked to perform a threat assessment task, either with or without the explanatory tool providing information in the form of missile launch success zone envelopes, while concurrently flying a waypoint course within set flight parameters. The results showed that there was a significant improvement (p less than 0.01) in threat assessment accuracy of 30% when using the explanatory tool. This threat assessment performance advantage was achieved without a trade-off with flying task performance. Situational awareness measures showed no general differences between the explanatory and control conditions, but significant learning effects suggested that the explanatory tool makes the task initially more intuitive and hence less demanding on the pilots' attentional resources. The paper concludes that DRA's data-fused explanatory tool is successful at improving threat assessment accuracy in a realistic simulated flying environment, and briefly discusses the requirements for further research in the area.

  19. Motivational Interviewing Skills in Health Care Encounters (MISHCE): Development and psychometric testing of an assessment tool.

    PubMed

    Petrova, Tatjana; Kavookjian, Jan; Madson, Michael B; Dagley, John; Shannon, David; McDonough, Sharon K

    2015-01-01

    Motivational interviewing (MI) has demonstrated a significant impact as an intervention strategy for addiction management, change in lifestyle behaviors, and adherence to prescribed medication and other treatments. Key elements to studying MI include training in MI of professionals who will use it, assessment of skills acquisition in trainees, and the use of a validated skills assessment tool. The purpose of this research project was to develop a psychometrically valid and reliable tool that has been designed to assess MI skills competence in health care provider trainees. The goal was to develop an assessment tool that would evaluate the acquisition and use of specific MI skills and principles, as well as the quality of the patient-provider therapeutic alliance in brief health care encounters. To address this purpose, specific steps were followed, beginning with a literature review. This review contributed to the development of relevant conceptual and operational definitions, selecting a scaling technique and response format, and methods for analyzing validity and reliability. Internal consistency reliability was established on 88 video recorded interactions. The inter-rater and test-retest reliability were established using randomly selected 18 from the 88 interactions. The assessment tool Motivational Interviewing Skills for Health Care Encounters (MISHCE) and a manual for use of the tool were developed. Validity and reliability of MISHCE were examined. Face and content validity were supported with well-defined conceptual and operational definitions and feedback from an expert panel. Reliability was established through internal consistency, inter-rater reliability, and test-retest reliability. The overall internal consistency reliability (Cronbach's alpha) for all fifteen items was 0.75. MISHCE demonstrated good inter-rater reliability and good to excellent test-retest reliability. MISHCE assesses the health provider's level of knowledge and skills in brief

  20. CROSS-CULTURAL ADAPTATION AND VALIDATION OF THE KOREAN VERSION OF THE CUMBERLAND ANKLE INSTABILITY TOOL.

    PubMed

    Ko, Jupil; Rosen, Adam B; Brown, Cathleen N

    2015-12-01

    The Cumberland Ankle Instability Tool (CAIT) is a valid and reliable patient reported outcome used to assess the presence and severity of chronic ankle instability (CAI). The CAIT has been cross-culturally adapted into other languages for use in non-English speaking populations. However, there are no valid questionnaires to assess CAI in individuals who speak Korean. The purpose of this study was to translate, cross-culturally adapt, and validate the CAIT, for use in a Korean-speaking population with CAI. Cross-cultural reliability study. The CAIT was cross-culturally adapted into Korean according to accepted guidelines and renamed the Cumberland Ankle Instability Tool-Korean (CAIT-K). Twenty-three participants (12 males, 11 females) who were bilingual in English and Korean were recruited and completed the original and adapted versions to assess agreement between versions. An additional 168 national level Korean athletes (106 male, 62 females; age = 20.3 ± 1.1 yrs), who participated in ≥ 90 minutes of physical activity per week, completed the final version of the CAIT-K twice within 14 days. Their completed questionnaires were assessed for internal consistency, test-retest reliability, criterion validity, and construct validity. For bilingual participants, intra-class correlation coefficients (ICC2,1) between the CAIT and the CAIT-K for test-retest reliability were 0.95 (SEM=1.83) and 0.96 (SEM=1.50) in right and left limbs, respectively. The Cronbach's alpha coefficients were 0.92 and 0.90 for the CAIT-K in right and left limbs, respectively. For native Korean speakers, the CAIT-K had high internal consistency (Cronbach's α=0.89) and intra-class correlation coefficient (ICC2,1 = 0.94, SEM=1.72), correlation with the physical component score (rho=0.70, p = 0.001) of the Short-Form Health Survey (SF-36), and the Kaiser-Meyer-Olkin score was 0.87. The original CAIT was translated, cross-culturally adapted, and validated from English to Korean

  1. Predictive Validity of Measures of the Pathfinder Scaling Algorithm on Programming Performance: Alternative Assessment Strategy for Programming Education

    ERIC Educational Resources Information Center

    Lau, Wilfred W. F.; Yuen, Allan H. K.

    2009-01-01

    Recent years have seen a shift in focus from assessment of learning to assessment for learning and the emergence of alternative assessment methods. However, the reliability and validity of these methods as assessment tools are still questionable. In this article, we investigated the predictive validity of measures of the Pathfinder Scaling…

  2. Validity of portfolio assessment: which qualities determine ratings?

    PubMed

    Driessen, Erik W; Overeem, Karlijn; van Tartwijk, Jan; van der Vleuten, Cees P M; Muijtjens, Arno M M

    2006-09-01

    The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio programme aimed at stimulating Year 1 medical students to reflect on their professional and personal development. In later curricular years, this portfolio is also used to judge clinical competence. We developed an instrument, the Portfolio Analysis Scoring Inventory, to examine the impact of form and content aspects on portfolio assessment. The Inventory consists of 15 items derived from interviews with experienced mentors, the literature, and the criteria for reflective competence used in the regular portfolio assessment procedure. Forty portfolios, selected from 231 portfolios for which ratings from the regular assessment procedure were available, were rated by 2 researchers, independently, using the Inventory. Regression analysis was used to estimate the correlation between the ratings from the regular assessment and those resulting from the Inventory items. Inter-rater agreement ranged from 0.46 to 0.87. The strongest predictor of the variance in the regular ratings was 'quality of reflection' (R 0.80; R2 66%). No further items accounted for a significant proportion of variance. Irrelevant items, such as writing style and lay-out, had negligible effects. The absence of an impact of irrelevant criteria appears to support the validity of the portfolio assessment procedure. Further studies should examine the portfolio's validity for the assessment of clinical competence.

  3. Risk of malnutrition (over and under-nutrition): validation of the JaNuS screening tool.

    PubMed

    Donini, Lorenzo M; Ricciardi, Laura Maria; Neri, Barbara; Lenzi, Andrea; Marchesini, Giulio

    2014-12-01

    Malnutrition (over and under-nutrition) is highly prevalent in patients admitted to hospital and it is a well-known risk factor for increased morbidity and mortality. Nutritional problems are often misdiagnosed, and especially the coexistence of over and undernutrition is not usually recognized. We aimed to develop and validate a screening tool for the easy detection and reporting of both undernutrition and overnutrition, specifically identifying the clinical conditions where the two types of malnutrition coexist. The study consisted of three phases: 1) selection of an appropriate study population (estimation sample) and of the hospital admission parameters to identify overnutrition and undernutrition; 2) combination of selected variables to create a screening tool to assess the nutritional risk in case of undernutrition, overnutrition, or the copresence of both the conditions, to be used by non-specialist health care professionals; 3) validation of the screening tool in a different patient sample (validation sample). Two groups of variables (12 for undernutrition, 7 for overnutrition) were identified in separate logistic models for their correlation with the outcome variables. Both models showed high efficacy, sensitivity and specificity (overnutrition, 97.7%, 99.6%, 66.6%, respectively; undernutrition, 84.4%, 83.6%, 84.8%). The logistic models were used to construct a two-faced test (named JaNuS - Just A Nutritional Screening) fitting into a two-dimension Cartesian coordinate graphic system. In the validation sample the JaNuS test confirmed its predictive value. Internal consistency and test-retest analysis provide evidence for the reliability of the test. The study provides a screening tool for the assessment of the nutritional risk, based on parameters easy-to-use by health care personnel lacking nutritional competence and characterized by excellent predictive validity. The test might be confidently applied in the clinical setting to determine the importance of

  4. Content validity of the Geriatric Health Assessment Instrument

    PubMed Central

    Pedreira, Rhaine Borges Santos; Rocha, Saulo Vasconcelos; dos Santos, Clarice Alves; Vasconcelos, Lélia Renata Carneiro; Reis, Martha Cerqueira

    2016-01-01

    ABSTRACT Objective Assess the content validity of the Elderly Health Assessment Tool with low education. Methods The data collection instrument/questionnaire was prepared and submitted to an expert panel comprising four healthcare professionals experienced in research on epidemiology of aging. The experts were allowed to suggest item inclusion/exclusion and were asked to rate the ability of individual items in questionnaire blocks to encompass target dimensions as “not valid”, “somewhat valid” or “valid”, using an interval scale. Percent agreement and the Content Validity Index were used as measurements of inter-rater agreement; the minimum acceptable inter-rater agreement was set at 80%. Results The mean instrument percent agreement rate was 86%, ranging from 63 to 99%, and from 50 to 100% between and within blocks respectively. The Mean Content Validity Index score was 93.47%, ranging from 50 to 100% between individual items. Conclusion The instrument showed acceptable psychometric properties for application in geriatric populations with low levels of education. It enabled identifying diseases and assisted in choice of strategies related to health of the elderly. PMID:27462889

  5. The Language Exposure Assessment Tool: Quantifying Language Exposure in Infants and Children

    ERIC Educational Resources Information Center

    DeAnda, Stephanie; Bosch, Laura; Poulin-Dubois, Diane; Zesiger, Pascal; Friend, Margaret

    2016-01-01

    Purpose: The aim of this study was to develop the Language Exposure Assessment Tool (LEAT) and to examine its cross-linguistic validity, reliability, and utility. The LEAT is a computerized interview-style assessment that requests parents to estimate language exposure. The LEAT yields an automatic calculation of relative language exposure and…

  6. Objective assessment of technique in laparoscopic colorectal surgery: what are the existing tools?

    PubMed

    Foster, J D; Francis, N K

    2015-01-01

    Assessment can improve the effectiveness of surgical training and enable valid judgments of competence. Laparoscopic colon resection surgery is now taught within surgical residency programs, and assessment tools are increasingly used to stimulate formative feedback and enhance learning. Formal assessment of technical performance in laparoscopic colon resection has been successfully applied at the specialist level in the English "LAPCO" National Training Program. Objective assessment tools need to be developed for training and assessment in laparoscopic rectal cancer resection surgery. Simulation may have a future role in assessment and accreditation in laparoscopic colorectal surgery; however, existing virtual reality models are not ready to be used for assessment of this advanced surgery.

  7. Measurement properties of tools used to assess suicidality in autistic and general population adults: A systematic review.

    PubMed

    Cassidy, S A; Bradley, L; Bowen, E; Wigham, S; Rodgers, J

    2018-05-05

    Adults diagnosed with autism are at significantly increased risk of suicidal thoughts, suicidal behaviours and dying by suicide. However, it is unclear whether any validated tools are currently available to effectively assess suicidality in autistic adults in research and clinical practice. This is crucial for understanding and preventing premature death by suicide in this vulnerable group. This two stage systematic review therefore aimed to identify tools used to assess suicidality in autistic and general population adults, evaluate these tools for their appropriateness and measurement properties, and make recommendations for appropriate selection of suicidality assessment tools in research and clinical practice. Three databases were searched (PsycInfo, Medline and Web of Knowledge). Four frequently used suicidality assessment tools were identified, and subsequently rated for quality of the evidence in support of their measurement properties using the COSMIN checklist. Despite studies having explored suicidality in autistic adults, none had utilised a validated tool. Overall, there was lack of evidence in support of suicidality risk assessments successfully predicting future suicide attempts. We recommend adaptations to current suicidality assessment tools and priorities for future research, in order to better conceptualise suicidality and its measurement in autism. Copyright © 2018 The Authors. Published by Elsevier Ltd.. All rights reserved.

  8. Psychometric validity of the Cochlear Implant Function Index (CIFI): a quality of life assessment tool for adult cochlear implant users.

    PubMed

    Coelho, Daniel H; Hammerschlag, Paul E; Bat-Chava, Yael; Kohan, Darius

    2009-06-01

    The Cochlear Implant Function Index (CIFI) is created to assess adult cochlear implant (CI) auditory effectiveness in real world situations. Our objective is to evaluate the CIFI as a reliable psychometric tool to assess 1) reliance on visual assistance, 2) telephone use, 3) communication at work, 4) 'hearing' in noise, 5) in groups, and 6) in large room settings. Based upon Guttman scaling properties, the CIFI elicits implanted respondent's functional level with auditory independence from Level 1 (still requiring signing) to level 4 (without any help beyond CI). A blinded, retrospective questionnaire is anonymously answered by cochlear implant recipients. CI centers of tertiary care medical centers, CI support group, and an interactive web page of a hearing and speech center in a large metropolitan region. 245 respondents from a varied adult CI population implanted for one month to 19 years prior to answering the questionnaire. An assessment tool of CI function. A coefficient of reproducibility (CR) for the Guttman scale format equal or greater than 0.90, indicating good scalability. CR in the CIFI was above 0.90. Effective scalability and mean scores from 2.5 to 3.5 for the six areas examined (1.00-4.00) were achieved. The psychometric properties of this user friendly survey demonstrate consistently good scalability. Based on these findings, the CIFI provides a validated tool that can be used for systematic comparisons between groups of patients or for follow-up outcomes in patients who use cochlear implants. Further study is indicated to correlate CIFI scores with sound and speech perception scores. Copyright 2009 John Wiley & Sons, Ltd.

  9. SurvExpress: an online biomarker validation tool and database for cancer gene expression data using survival analysis.

    PubMed

    Aguirre-Gamboa, Raul; Gomez-Rueda, Hugo; Martínez-Ledesma, Emmanuel; Martínez-Torteya, Antonio; Chacolla-Huaringa, Rafael; Rodriguez-Barrientos, Alberto; Tamez-Peña, José G; Treviño, Victor

    2013-01-01

    Validation of multi-gene biomarkers for clinical outcomes is one of the most important issues for cancer prognosis. An important source of information for virtual validation is the high number of available cancer datasets. Nevertheless, assessing the prognostic performance of a gene expression signature along datasets is a difficult task for Biologists and Physicians and also time-consuming for Statisticians and Bioinformaticians. Therefore, to facilitate performance comparisons and validations of survival biomarkers for cancer outcomes, we developed SurvExpress, a cancer-wide gene expression database with clinical outcomes and a web-based tool that provides survival analysis and risk assessment of cancer datasets. The main input of SurvExpress is only the biomarker gene list. We generated a cancer database collecting more than 20,000 samples and 130 datasets with censored clinical information covering tumors over 20 tissues. We implemented a web interface to perform biomarker validation and comparisons in this database, where a multivariate survival analysis can be accomplished in about one minute. We show the utility and simplicity of SurvExpress in two biomarker applications for breast and lung cancer. Compared to other tools, SurvExpress is the largest, most versatile, and quickest free tool available. SurvExpress web can be accessed in http://bioinformatica.mty.itesm.mx/SurvExpress (a tutorial is included). The website was implemented in JSP, JavaScript, MySQL, and R.

  10. SurvExpress: An Online Biomarker Validation Tool and Database for Cancer Gene Expression Data Using Survival Analysis

    PubMed Central

    Aguirre-Gamboa, Raul; Gomez-Rueda, Hugo; Martínez-Ledesma, Emmanuel; Martínez-Torteya, Antonio; Chacolla-Huaringa, Rafael; Rodriguez-Barrientos, Alberto; Tamez-Peña, José G.; Treviño, Victor

    2013-01-01

    Validation of multi-gene biomarkers for clinical outcomes is one of the most important issues for cancer prognosis. An important source of information for virtual validation is the high number of available cancer datasets. Nevertheless, assessing the prognostic performance of a gene expression signature along datasets is a difficult task for Biologists and Physicians and also time-consuming for Statisticians and Bioinformaticians. Therefore, to facilitate performance comparisons and validations of survival biomarkers for cancer outcomes, we developed SurvExpress, a cancer-wide gene expression database with clinical outcomes and a web-based tool that provides survival analysis and risk assessment of cancer datasets. The main input of SurvExpress is only the biomarker gene list. We generated a cancer database collecting more than 20,000 samples and 130 datasets with censored clinical information covering tumors over 20 tissues. We implemented a web interface to perform biomarker validation and comparisons in this database, where a multivariate survival analysis can be accomplished in about one minute. We show the utility and simplicity of SurvExpress in two biomarker applications for breast and lung cancer. Compared to other tools, SurvExpress is the largest, most versatile, and quickest free tool available. SurvExpress web can be accessed in http://bioinformatica.mty.itesm.mx/SurvExpress (a tutorial is included). The website was implemented in JSP, JavaScript, MySQL, and R. PMID:24066126

  11. Body Dysmorphic Disorder in aesthetic rhinoplasty: Validating a new screening tool.

    PubMed

    Lekakis, Garyfalia; Picavet, Valerie A; Gabriëls, Loes; Grietens, Jente; Hellings, Peter W

    2016-08-01

    To validate a new screening tool for body dysmorphic disorder (BDD) in patients seeking aesthetic rhinoplasty. We performed a prospective instrument validation study in an academic rhinology clinic. The Body Dysmorphic Disorder Questionnaire-Aesthetic Surgery (BDDQ-AS) is a seven-item short questionnaire validated in 116 patients undergoing aesthetic rhinoplasty. Screening was positive if the patient acknowledged on the BDDQ-AS that he/she was concerned about their appearance (question 1 = yes) AND preoccupied with these concerns (question 2 = yes) AND that these concerns caused at least moderate distress or impairment in different domains of daily life (question 3 or 4 or 5 or 6 ≥ 3 or question 7 = yes). Construct validity was assessed by comparing the BDDQ-AS to the Sheehan Disability Scale and the Derriford Appearance Scale-59. To determine concurrent validity, the BDDQ-AS was compared to the Yale-Brown Obsessive Compulsive Scale Modified for BDD. Finally, the predictive value of the BDDQ-AS on satisfaction 12 months after rhinoplasty was evaluated using a visual analogue scale and the Rhinoplasty Outcome Evaluation. Reliability of the BDDQ-AS was adequate, with Cronbach alpha = .83 for rhinoplasty patients and .84 for controls. Sensitivity was 89.6% and specificity 81.4%. BDDQ-AS-positive patients (n = 55) were more impaired in daily life and experienced more appearance-related distress and dysfunction compared to BDDQ-AS-negative patients. Moreover, they had more severe BDD symptoms. Finally, BDDQ-AS-positive patients were less satisfied after surgery compared to BDDQ-AS-negative patients. We hereby validated a new screening tool for BDD in an aesthetic rhinoplasty population. 3b. Laryngoscope, 126:1739-1745, 2016. © 2016 The American Laryngological, Rhinological and Otological Society, Inc.

  12. Development of the Assessment of Burden of COPD tool: an integrated tool to measure the burden of COPD.

    PubMed

    Slok, Annerika H M; in 't Veen, Johannes C C M; Chavannes, Niels H; van der Molen, Thys; Rutten-van Mölken, Maureen P M H; Kerstjens, Huib A M; Salomé, Philippe L; Holverda, Sebastiaan; Dekhuijzen, P N Richard; Schuiten, Denise; Asijee, Guus M; van Schayck, Onno C P

    2014-07-10

    In deciding on the treatment plan for patients with chronic obstructive pulmonary disease (COPD), the burden of COPD as experienced by patients should be the core focus. It is therefore important for daily practice to develop a tool that can both assess the burden of COPD and facilitate communication with patients in clinical practice. This paper describes the development of an integrated tool to assess the burden of COPD in daily practice. A definition of the burden of COPD was formulated by a Dutch expert team. Interviews showed that patients and health-care providers agreed on this definition. We found no existing instruments that fully measured burden of disease according to this definition. However, the Clinical COPD Questionnaire meets most requirements, and was therefore used and adapted. The adapted questionnaire is called the Assessment of Burden of COPD (ABC) scale. In addition, the ABC tool was developed, of which the ABC scale is the core part. The ABC tool is a computer program with an algorithm that visualises outcomes and provides treatment advice. The next step in the development of the tool is to test the validity and effectiveness of both the ABC scale and tool in daily practice.

  13. Assessing the Efficacy of the Measure of Understanding of Macroevolution as a Valid Tool for Undergraduate Non-Science Majors

    ERIC Educational Resources Information Center

    Romine, William Lee; Walter, Emily Marie

    2014-01-01

    Efficacy of the Measure of Understanding of Macroevolution (MUM) as a measurement tool has been a point of contention among scholars needing a valid measure for knowledge of macroevolution. We explored the structure and construct validity of the MUM using Rasch methodologies in the context of a general education biology course designed with an…

  14. Quality Assessment of Comparative Diagnostic Accuracy Studies: Our Experience Using a Modified Version of the QUADAS-2 Tool

    ERIC Educational Resources Information Center

    Wade, Ros; Corbett, Mark; Eastwood, Alison

    2013-01-01

    Assessing the quality of included studies is a vital step in undertaking a systematic review. The recently revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool (QUADAS-2), which is the only validated quality assessment tool for diagnostic accuracy studies, does not include specific criteria for assessing comparative studies. As…

  15. Assessing patient capacity to consent to treatment: an integrative review of instruments and tools.

    PubMed

    Lamont, Scott; Jeon, Yun-Hee; Chiarella, Mary

    2013-09-01

    To provide a narrative synthesis of research findings on instruments or tools designed to aid assessment of patient capacity to consent to treatment. Capacity assessment is of significant priority within health care as a finding of incapacity is a vehicle for the removal of many of an individual's fundamental rights. Despite there being many instruments and tools available to aid health professionals in the assessment of patient capacity, there are no standardised guidelines from professional bodies that inform the assessment of mental capacity. Integrative review. Primary studies of instruments or tools concerning assessment of patient capacity to consent to treatment, published in English in peer-reviewed journals between January 2005-December 2010, were included in the review. Review papers of capacity assessment instruments were included for years including and prior to 2006. Nineteen instruments were found which assess patient capacity to consent. Key themes were identified in terms of capacity domains assessed, psychometric properties, instrument implementation, patient populations studied and instrument versus clinician judgement. Despite a plethora of capacity assessment instruments and tools available, only a small number of instruments were found to have demonstrated both reliability and validity. Further research is required to improve the validity of existing capacity assessment instruments. Increased attention to patient rights and autonomy arguably places a considerable burden on healthcare professionals to facilitate capacity assessments across a continuum of health care. Despite a plethora of capacity assessment instruments and tools being available to healthcare professionals, a comprehensive assessment requires time and is often difficult in the acute care setting. A strictly formulaic approach to the assessment of capacity is unlikely to capture specific individual nuances; therefore, capacity assessment instruments should support, but not

  16. Development of a tool to support holistic generic assessment of clinical procedure skills.

    PubMed

    McKinley, Robert K; Strand, Janice; Gray, Tracey; Schuwirth, Lambert; Alun-Jones, Tom; Miller, Helen

    2008-06-01

    The challenges of maintaining comprehensive banks of valid checklists make context-specific checklists for assessment of clinical procedural skills problematic. This paper reports the development of a tool which supports generic holistic assessment of clinical procedural skills. We carried out a literature review, focus groups and non-participant observation of assessments with interview of participants, participant evaluation of a pilot objective structured clinical examination (OSCE), a national modified Delphi study with prior definitions of consensus and an OSCE. Participants were volunteers from a large acute teaching trust, a teaching primary care trust and a national sample of National Health Service staff. Results In total, 86 students, trainees and staff took part in the focus groups, observation of assessments and pilot OSCE, 252 in the Delphi study and 46 candidates and 50 assessors in the final OSCE. We developed a prototype tool with 5 broad categories amongst which were distributed 38 component competencies. There was > 70% agreement (our prior definition of consensus) at the first round of the Delphi study for inclusion of all categories and themes and no consensus for inclusion of additional categories or themes. Generalisability was 0.76. An OSCE based on the instrument has a predicted reliability of 0.79 with 12 stations and 1 assessor per station or 10 stations and 2 assessors per station. This clinical procedural skills assessment tool enables reliable assessment and has content and face validity for the assessment of clinical procedural skills. We have designated it the Leicester Clinical Procedure Assessment Tool (LCAT).

  17. Systematic Omics Analysis Review (SOAR) Tool to Support Risk Assessment

    PubMed Central

    McConnell, Emma R.; Bell, Shannon M.; Cote, Ila; Wang, Rong-Lin; Perkins, Edward J.; Garcia-Reyero, Natàlia; Gong, Ping; Burgoon, Lyle D.

    2014-01-01

    Environmental health risk assessors are challenged to understand and incorporate new data streams as the field of toxicology continues to adopt new molecular and systems biology technologies. Systematic screening reviews can help risk assessors and assessment teams determine which studies to consider for inclusion in a human health assessment. A tool for systematic reviews should be standardized and transparent in order to consistently determine which studies meet minimum quality criteria prior to performing in-depth analyses of the data. The Systematic Omics Analysis Review (SOAR) tool is focused on assisting risk assessment support teams in performing systematic reviews of transcriptomic studies. SOAR is a spreadsheet tool of 35 objective questions developed by domain experts, focused on transcriptomic microarray studies, and including four main topics: test system, test substance, experimental design, and microarray data. The tool will be used as a guide to identify studies that meet basic published quality criteria, such as those defined by the Minimum Information About a Microarray Experiment standard and the Toxicological Data Reliability Assessment Tool. Seven scientists were recruited to test the tool by using it to independently rate 15 published manuscripts that study chemical exposures with microarrays. Using their feedback, questions were weighted based on importance of the information and a suitability cutoff was set for each of the four topic sections. The final validation resulted in 100% agreement between the users on four separate manuscripts, showing that the SOAR tool may be used to facilitate the standardized and transparent screening of microarray literature for environmental human health risk assessment. PMID:25531884

  18. Use of dietary assessment tools in randomized trials evaluating diet-based interventions in pregnancy: a systematic review of literature.

    PubMed

    Al Wattar, Bassel H; Mylrea-Lowndes, Bronacha; Morgan, Catrin; Moore, Amanda P; Thangaratinam, Shakila

    2016-12-01

    Accurate assessment of dietary intake in interventional trials is the key to evaluate changes in dietary behaviour and compliance. We evaluated the use of dietary assessment tools in randomized trials on diet-based interventions in pregnancy by a systematic review. We updated our previous search (until January 2012) on trials of diet and lifestyle interventions in pregnancy using Medline and EMBASE up to December 2015. Two independent reviewers undertook study selection and data extraction. We assessed the characteristics of dietary assessment tools, the timing and frequency of use and any validation undertaken.Two-thirds (39/58, 67%) of the included studies used some form of tools to assess dietary intake. Multiple days' food diaries were the most commonly used (23/39, 59%). Three studies (3/39, 8%) validated the used tools in a pregnant population. Three studies (3/39, 8%) prespecified the criteria for adherence to the intervention. The use of dietary assessment tools was not associated with study quality, year of publication, journal impact factor, type of journal and the study sample size. Although self-reporting dietary assessment tools are widely used in interventional dietary trials in pregnancy, the quality and applicability of existing tools are low.

  19. An Evaluation Framework and Instrument for Evaluating e-Assessment Tools

    ERIC Educational Resources Information Center

    Singh, Upasana Gitanjali; de Villiers, Mary Ruth

    2017-01-01

    e-Assessment, in the form of tools and systems that deliver and administer multiple choice questions (MCQs), is used increasingly, raising the need for evaluation and validation of such systems. This research uses literature and a series of six empirical action research studies to develop an evaluation framework of categories and criteria called…

  20. Validation of a global scale to assess the quality of interprofessional teamwork in mental health settings.

    PubMed

    Tomizawa, Ryoko; Yamano, Mayumi; Osako, Mitue; Hirabayashi, Naotugu; Oshima, Nobuo; Sigeta, Masahiro; Reeves, Scott

    2017-12-01

    Few scales currently exist to assess the quality of interprofessional teamwork through team members' perceptions of working together in mental health settings. The purpose of this study was to revise and validate an interprofessional scale to assess the quality of teamwork in inpatient psychiatric units and to use it multi-nationally. A literature review was undertaken to identify evaluative teamwork tools and develop an additional 12 items to ensure a broad global focus. Focus group discussions considered adaptation to different care systems using subjective judgements from 11 participants in a pre-test of items. Data quality, construct validity, reproducibility, and internal consistency were investigated in the survey using an international comparative design. Exploratory factor analysis yielded five factors with 21 items: 'patient/community centred care', 'collaborative communication', 'interprofessional conflict', 'role clarification', and 'environment'. High overall internal consistency, reproducibility, adequate face validity, and reasonable construct validity were shown in the USA and Japan. The revised Collaborative Practice Assessment Tool (CPAT) is a valid measure to assess the quality of interprofessional teamwork in psychiatry and identifies the best strategies to improve team performance. Furthermore, the revised scale will generate more rigorous evidence for collaborative practice in psychiatry internationally.

  1. Pain assessment in elderly with dementia: Brazilian validation of the PACSLAC scale

    PubMed Central

    Thé, Karol Bezerra; Gazoni, Fernanda Martins; Cherpak, Guilherme Liausu; Lorenzet, Isabel Clasen; dos Santos, Luciana Alves; Nardes, Edlene Maria; dos Santos, Fânia Cristina

    2016-01-01

    ABSTRACT Objective To validate the Pain Assessment Checklist for Seniors with Limited Ability to Communicate – Portuguese in demented elderly and to analyze its measurement properties. Methods We evaluated 50 elderly with dementia, residing in a nursing home and with limited communication ability, when exposed to potentially painful situations. The tool was applied at two different moments. First, two interviewers applied it simultaneously, and the intensity of pain was asked based on the caregiver’s opinion. After 14 days, with no analgesic intervention, one of the interviewers applied it again. Results The sample comprised more females, aged over 80 years, with dementia due to Alzheimer, presenting musculoskeletal pain of moderate to severe intensity. The psychometric properties of the tool demonstrated appropriate internal consistency (Cronbach’s alpha coefficient of 0.827). The scale had excellent reproducibility, according to the intraclass correlation coefficient, and the tool has been duly validated. Conclusion The Pain Assessment Checklist for Seniors with Limited Ability to Communicate – Portuguese had adequate measuring properties for use with elderly presenting limited communication. PMID:27462888

  2. Reliability, Validity, and Factor Structure of the Current Assessment Practice Evaluation-Revised (CAPER) in a National Sample.

    PubMed

    Lyon, Aaron R; Pullmann, Michael D; Dorsey, Shannon; Martin, Prerna; Grigore, Alexandra A; Becker, Emily M; Jensen-Doss, Amanda

    2018-05-11

    Measurement-based care (MBC) is an increasingly popular, evidence-based practice, but there are no tools with established psychometrics to evaluate clinician use of MBC practices in mental health service delivery. The current study evaluated the reliability, validity, and factor structure of scores generated from a brief, standardized tool to measure MBC practices, the Current Assessment Practice Evaluation-Revised (CAPER). Survey data from a national sample of 479 mental health clinicians were used to conduct exploratory and confirmatory factor analyses, as well as reliability and validity analyses (e.g., relationships between CAPER subscales and clinician MBC attitudes). Analyses revealed competing two- and three-factor models. Regardless of the model used, scores from CAPER subscales demonstrated good reliability and convergent and divergent validity with MBC attitudes in the expected directions. The CAPER appears to be a psychometrically sound tool for assessing clinician MBC practices. Future directions for development and application of the tool are discussed.

  3. Validation of the Arabic version of the score for allergic rhinitis tool.

    PubMed

    Alharethy, Sami; Wedami, Mawaheb Al; Syouri, Falah; Alqabbani, Almaha A; Baqays, Abdulsalam; Mesallam, Tamer; Aldrees, Turki

    2017-01-01

    Allergic rhinitis (AR) is a common inflammation of the nasal mucosa in response to allergen exposure. We translated and validated the Score for Allergic Rhinitis (SFAR) into an Arabic version so that the disease can be studied in an Arabic population. SFAR is a non-invasive self-administered tool that evaluates eight items related to AR. This study aimed to translate and culturally adapt the SFAR questionnaire into Arabic, and assess the validity, consistency, and reliability of the translated version in an Arabic-speaking population of patients with suspected AR. Cross-sectional. Tertiary care hospital in Riyadh. The Arabic version of the SFAR was administered to patients with suspected AR and control participants. Comparison of the AR and control groups to determine the test-retest reliability and internal consistency of the instrument. The AR (n=173) and control (n=75) groups had significantly different Arabic SFAR scores (P < .0001). The instrument provided satisfactory internal consistency (Cronbach's alpha value of 0.7). The test-retest reliability was excellent for the total Arabic SFAR score (r =0.836, P < .0001). These findings demonstrate that the Arabic version of the SFAR is a valid tool that can be used to screen Arabic speakers with suspected AR. The absence of objective allergy testing.

  4. Assessment Tools for Evaluation of Oral Feeding in Infants Less than Six Months Old

    PubMed Central

    Pados, Britt F.; Park, Jinhee; Estrem, Hayley; Awotwi, Araba

    2015-01-01

    Background Feeding difficulty is common in infants less than six months old. Identification of infants in need of specialized treatment is critical to ensure appropriate nutrition and feeding skill development. Valid and reliable assessment tools help clinicians objectively evaluate feeding. Purpose To identify and evaluate assessment tools available for clinical assessment of bottle- and breast-feeding in infants less than six months old. Methods/Search Strategy CINAHL, HaPI, PubMed, and Web of Science were searched for “infant feeding” and “assessment tool.” The literature (n=237) was reviewed for relevant assessment tools. A secondary search was conducted in CINAHL and PubMed for additional literature on identified tools. Findings/Results Eighteen assessment tools met inclusion criteria. Of these, seven were excluded because of limited available literature or because they were intended for use with a specific diagnosis or in research only. There are 11 assessment tools available for clinical practice. Only two of these were intended for bottle-feeding. All 11 indicated they were appropriate for use with breast-feeding. None of the available tools have adequate psychometric development and testing. Implications for Practice All of the tools should be used with caution. The Early Feeding Skills Assessment and Bristol Breastfeeding Assessment Tool had the most supportive psychometric development and testing. Implications for Research Feeding assessment tools need to be developed and tested to guide optimal clinical care of infants from birth through six months. A tool that assesses both bottle- and breast-feeding would allow for consistent assessment across feeding methods. PMID:26945280

  5. Movement Skill Assessment of Typically Developing Preschool Children: A Review of Seven Movement Skill Assessment Tools

    PubMed Central

    Cools, Wouter; Martelaer, Kristine De; Samaey, Christiane; Andries, Caroline

    2009-01-01

    The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child’s physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools’ usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children. Key pointsThis review discusses seven movement skill assessment tool’s test content, reliability, validity and normative samples.The seven assessment tools all showed to be of great value. Strengths and weaknesses indicate that test choice will depend on specific purpose of test use.Further data collection should also include larger data samples of able bodied preschool children.Admitting PE specialists in assessment of fundamental movement skill performance among preschool children is recommended.The assessment tool’s normative data samples would benefit from frequent movement skill performance follow-up of today’s children. Abbreviations MOT 4-6: Motoriktest fur vier- bis sechsjährige Kinder, M-ABC: Movement Assessment Battery for Children, PDMS: Peabody Development Scales, KTK: K

  6. A web-based rapid assessment tool for production publishing solutions

    NASA Astrophysics Data System (ADS)

    Sun, Tong

    2010-02-01

    Solution assessment is a critical first-step in understanding and measuring the business process efficiency enabled by an integrated solution package. However, assessing the effectiveness of any solution is usually a very expensive and timeconsuming task which involves lots of domain knowledge, collecting and understanding the specific customer operational context, defining validation scenarios and estimating the expected performance and operational cost. This paper presents an intelligent web-based tool that can rapidly assess any given solution package for production publishing workflows via a simulation engine and create a report for various estimated performance metrics (e.g. throughput, turnaround time, resource utilization) and operational cost. By integrating the digital publishing workflow ontology and an activity based costing model with a Petri-net based workflow simulation engine, this web-based tool allows users to quickly evaluate any potential digital publishing solutions side-by-side within their desired operational contexts, and provides a low-cost and rapid assessment for organizations before committing any purchase. This tool also benefits the solution providers to shorten the sales cycles, establishing a trustworthy customer relationship and supplement the professional assessment services with a proven quantitative simulation and estimation technology.

  7. Validity and feasibility of the EMG direct observation tool (EMG-DOT).

    PubMed

    Leep Hunderfund, Andrea N; Rubin, Devon I; Laughlin, Ruple S; Sorenson, Eric J; Watson, James C; Jones, Lyell K; Juul, Dorthea; Park, Yoon Soo

    2016-04-26

    To develop a new workplace-based EMG direct observation tool (EMG-DOT) and gather validity evidence supporting its use for assessing electrodiagnostic skills among postgraduate medical trainees. The EMG-DOT was developed by experts using an iterative process. Validity evidence from content, response process, internal structure, relations to other variables, and consequences of testing was collected during the 2013-2014 academic year. Of 3,412 studies performed by trainees during the study period, 299 (9%) were assessed using the EMG-DOT. Of these, 203 (68%) involved a physician rater and 96 (32%) involved a technician rater. The 14-item EMG-DOT had excellent internal-consistency reliability (Cronbach α 0.94). Correlations between individual items and criterion-referenced global ratings of performance ranged from 0.36 to 0.72 (all p < 0.001). Mean total scores increased from 70% to 80% over 4 months of the EMG rotation (p < 0.001) despite a corresponding significant increase in case complexity (0.21-0.74 on a 3-point rating scale; p < 0.001). Trainees reported that the observational assessment exercise improved their knowledge or skills in 82% of encounters (188/230) and that feedback generated by the EMG-DOT improved the quality of care provided to patients in 58% (133/230). Trainees were "satisfied" or "very satisfied" with the observational assessment exercise in 96% of encounters (234/243). This study provides validity evidence supporting the use of EMG-DOT scores to assess electrodiagnostic skills of residents and fellows. The EMG-DOT can be used to inform milestone-based assessments of trainee performance in neurology, child neurology, physical medicine and rehabilitation, neuromuscular, and clinical neurophysiology training programs. © 2016 American Academy of Neurology.

  8. Evaluating trauma team performance in a Level I trauma center: Validation of the trauma team communication assessment (TTCA-24).

    PubMed

    DeMoor, Stephanie; Abdel-Rehim, Shady; Olmsted, Richard; Myers, John G; Parker-Raley, Jessica

    2017-07-01

    Nontechnical skills (NTS), such as team communication, are well-recognized determinants of trauma team performance and good patient care. Measuring these competencies during trauma resuscitations is essential, yet few valid and reliable tools are available. We aimed to demonstrate that the Trauma Team Communication Assessment (TTCA-24) is a valid and reliable instrument that measures communication effectiveness during activations. Two tools with adequate psychometric strength (Trauma Nontechnical Skills Scale [T-NOTECHS], Team Emergency Assessment Measure [TEAM]) were identified during a systematic review of medical literature and compared with TTCA-24. Three coders used each tool to evaluate 35 stable and 35 unstable patient activations (defined according to Advanced Trauma Life Support criteria). Interrater reliability was calculated between coders using the intraclass correlation coefficient. Spearman rank correlation coefficient was used to establish concurrent validity between TTCA-24 and the other two validated tools. Coders achieved an intraclass correlation coefficient of 0.87 for stable patient activations and 0.78 for unstable activations scoring excellent on the interrater agreement guidelines. The median score for each assessment showed good team communication for all 70 videos (TEAM, 39.8 of 54; T-NOTECHS, 17.4 of 25; and TTCA-24, 87.4 of 96). A significant correlation between TTTC-24 and T-NOTECHS was revealed (p = 0.029), but no significant correlation between TTCA-24 and TEAM (p = 0.77). Team communication was rated slightly better across all assessments for stable versus unstable patient activations, but not statistically significant. TTCA-24 correlated with T-NOTECHS, an instrument measuring nontechnical skills for trauma teams, but not TEAM, a tool that assesses communication in generic emergency settings. TTCA-24 is a reliable and valid assessment that can be a useful adjunct when evaluating interpersonal and team communication during trauma

  9. Validity and reliability of balance assessment software using the Nintendo Wii balance board: usability and validation.

    PubMed

    Park, Dae-Sung; Lee, GyuChang

    2014-06-10

    A balance test provides important information such as the standard to judge an individual's functional recovery or make the prediction of falls. The development of a tool for a balance test that is inexpensive and widely available is needed, especially in clinical settings. The Wii Balance Board (WBB) is designed to test balance, but there is little software used in balance tests, and there are few studies on reliability and validity. Thus, we developed a balance assessment software using the Nintendo Wii Balance Board, investigated its reliability and validity, and compared it with a laboratory-grade force platform. Twenty healthy adults participated in our study. The participants participated in the test for inter-rater reliability, intra-rater reliability, and concurrent validity. The tests were performed with balance assessment software using the Nintendo Wii balance board and a laboratory-grade force platform. Data such as Center of Pressure (COP) path length and COP velocity were acquired from the assessment systems. The inter-rater reliability, the intra-rater reliability, and concurrent validity were analyzed by an intraclass correlation coefficient (ICC) value and a standard error of measurement (SEM). The inter-rater reliability (ICC: 0.89-0.79, SEM in path length: 7.14-1.90, SEM in velocity: 0.74-0.07), intra-rater reliability (ICC: 0.92-0.70, SEM in path length: 7.59-2.04, SEM in velocity: 0.80-0.07), and concurrent validity (ICC: 0.87-0.73, SEM in path length: 5.94-0.32, SEM in velocity: 0.62-0.08) were high in terms of COP path length and COP velocity. The balance assessment software incorporating the Nintendo Wii balance board was used in our study and was found to be a reliable assessment device. In clinical settings, the device can be remarkably inexpensive, portable, and convenient for the balance assessment.

  10. The development and validation of the clinicians' awareness towards cognitive errors (CATChES) in clinical decision making questionnaire tool.

    PubMed

    Chew, Keng Sheng; Kueh, Yee Cheng; Abdul Aziz, Adlihafizi

    2017-03-21

    Despite their importance on diagnostic accuracy, there is a paucity of literature on questionnaire tools to assess clinicians' awareness toward cognitive errors. A validation study was conducted to develop a questionnaire tool to evaluate the Clinician's Awareness Towards Cognitive Errors (CATChES) in clinical decision making. This questionnaire is divided into two parts. Part A is to evaluate the clinicians' awareness towards cognitive errors in clinical decision making while Part B is to evaluate their perception towards specific cognitive errors. Content validation for both parts was first determined followed by construct validation for Part A. Construct validation for Part B was not determined as the responses were set in a dichotomous format. For content validation, all items in both Part A and Part B were rated as "excellent" in terms of their relevance in clinical settings. For construct validation using exploratory factor analysis (EFA) for Part A, a two-factor model with total variance extraction of 60% was determined. Two items were deleted. Then, the EFA was repeated showing that all factor loadings are above the cut-off value of >0.5. The Cronbach's alpha for both factors are above 0.6. The CATChES questionnaire tool is a valid questionnaire tool aimed to evaluate the awareness among clinicians toward cognitive errors in clinical decision making.

  11. Validating SPICES as a Screening Tool for Frailty Risks among Hospitalized Older Adults

    PubMed Central

    Aronow, Harriet Udin; Borenstein, Jeff; Haus, Flora; Braunstein, Glenn D.; Bolton, Linda Burnes

    2014-01-01

    Older patients are vulnerable to adverse hospital events related to frailty. SPICES, a common screening protocol to identify risk factors in older patients, alerts nurses to initiate care plans to reduce the probability of patient harm. However, there is little published validating the association between SPICES and measures of frailty and adverse outcomes. This paper used data from a prospective cohort study on frailty among 174 older adult inpatients to validate SPICES. Almost all patients met one or more SPICES criteria. The sum of SPICES was significantly correlated with age and other well-validated assessments for vulnerability, comorbid conditions, and depression. Individuals meeting two or more SPICES criteria had a risk of adverse hospital events three times greater than individuals with either no or one criterion. Results suggest that as a screening tool used within 24 hours of admission, SPICES is both valid and predictive of adverse events. PMID:24876954

  12. Development of the Assessment of Burden of COPD tool: an integrated tool to measure the burden of COPD

    PubMed Central

    Slok, Annerika H M; in ’t Veen, Johannes C C M; Chavannes, Niels H; van der Molen, Thys; Rutten-van Mölken, Maureen P M H; Kerstjens, Huib A M; Salomé, Philippe L; Holverda, Sebastiaan; Dekhuijzen, PN Richard; Schuiten, Denise; Asijee, Guus M; van Schayck, Onno C P

    2014-01-01

    In deciding on the treatment plan for patients with chronic obstructive pulmonary disease (COPD), the burden of COPD as experienced by patients should be the core focus. It is therefore important for daily practice to develop a tool that can both assess the burden of COPD and facilitate communication with patients in clinical practice. This paper describes the development of an integrated tool to assess the burden of COPD in daily practice. A definition of the burden of COPD was formulated by a Dutch expert team. Interviews showed that patients and health-care providers agreed on this definition. We found no existing instruments that fully measured burden of disease according to this definition. However, the Clinical COPD Questionnaire meets most requirements, and was therefore used and adapted. The adapted questionnaire is called the Assessment of Burden of COPD (ABC) scale. In addition, the ABC tool was developed, of which the ABC scale is the core part. The ABC tool is a computer program with an algorithm that visualises outcomes and provides treatment advice. The next step in the development of the tool is to test the validity and effectiveness of both the ABC scale and tool in daily practice. PMID:25010353

  13. Movie for the Assessment of Social Cognition (MASC): Spanish validation.

    PubMed

    Lahera, G; Boada, L; Pousa, E; Mirapeix, I; Morón-Nozaleda, G; Marinas, L; Gisbert, L; Pamiàs, M; Parellada, M

    2014-08-01

    We present the Spanish validation of the "Movie for the Assessment of Social Cognition" instrument (MASC-SP). We recruited 22 adolescents and young adults with Asperger syndrome and 26 participants with typical development. The MASC-SP and three other social cognition instruments (Ekman Pictures of Facial Affect test, Reading the Mind in the Eyes Test, and Happé's Strange Stories) were administered to both groups. Individuals with Asperger syndrome had significantly lower scores in all measures of social cognition. The MASC-SP showed strong correlations with all three measures and relative independence of general cognitive functions. Internal consistency was optimal (0.86) and the test-retest was good. The MASC-SP is an ecologically valid and useful tool for assessing social cognition in the Spanish population.

  14. The Assessment, Development, Assurance Pharmacist's Tool (ADAPT) for ensuring quality implementation of health promotion programs.

    PubMed

    Truong, Hoai-An; Taylor, Catherine R; DiPietro, Natalie A

    2012-02-10

    To develop and validate the Assessment, Development, Assurance Pharmacist's Tool (ADAPT), an instrument for pharmacists and student pharmacists to use in developing and implementing health promotion programs. The 36-item ADAPT instrument was developed using the framework of public health's 3 core functions (assessment, policy development, and assurance) and 10 essential services. The tool's content and usage was assessed and conducted through peer-review and initial validity testing processes. Over 20 faculty members, preceptors, and student pharmacists at 5 institutions involved in planning and implementing health promotion initiatives reviewed the instrument and conducted validity testing. The instrument took approximately 15 minutes to complete and the findings resulted in changes and improvements to elements of the programs evaluated. The ADAPT instrument fills a need to more effectively plan, develop, implement, and evaluate pharmacist-directed public health programs that are evidence-based, high-quality, and compliant with laws and regulations and facilitates documentation of pharmacists' contributions to public health.

  15. Health Heritage© a web-based tool for the collection and assessment of family health history: initial user experience and analytic validity.

    PubMed

    Cohn, W F; Ropka, M E; Pelletier, S L; Barrett, J R; Kinzie, M B; Harrison, M B; Liu, Z; Miesfeldt, S; Tucker, A L; Worrall, B B; Gibson, J; Mullins, I M; Elward, K S; Franko, J; Guterbock, T M; Knaus, W A

    2010-01-01

    A detailed family health history is currently the most potentially useful tool for diagnosis and risk assessment in clinical genetics. We developed and evaluated the usability and analytic validity of a patient-driven web-based family health history collection and analysis tool. Health Heritage(©) guides users through the collection of their family health history by relative, generates a pedigree, completes risk assessment, stratification, and recommendations for 89 conditions. We compared the performance of Health Heritage to that of Usual Care using a nonrandomized cohort trial of 109 volunteers. We contrasted the completeness and sensitivity of family health history collection and risk assessments derived from Health Heritage and Usual Care to those obtained by genetic counselors and genetic assessment teams. Nearly half (42%) of the Health Heritage participants reported discovery of health risks; 63% found the information easy to understand and 56% indicated it would change their health behavior. Health Heritage consistently outperformed Usual Care in the completeness and accuracy of family health history collection, identifying 60% of the elevated risk conditions specified by the genetic team versus 24% identified by Usual Care. Health Heritage also had greater sensitivity than Usual Care when comparing the identification of risks. These results suggest a strong role for automated family health history collection and risk assessment and underscore the potential of these data to serve as the foundation for comprehensive, cost-effective personalized genomic medicine. Copyright © 2010 S. Karger AG, Basel.

  16. Validation of a Low-Thrust Mission Design Tool Using Operational Navigation Software

    NASA Technical Reports Server (NTRS)

    Englander, Jacob A.; Knittel, Jeremy M.; Williams, Ken; Stanbridge, Dale; Ellison, Donald H.

    2017-01-01

    Design of flight trajectories for missions employing solar electric propulsion requires a suitably high-fidelity design tool. In this work, the Evolutionary Mission Trajectory Generator (EMTG) is presented as a medium-high fidelity design tool that is suitable for mission proposals. EMTG is validated against the high-heritage deep-space navigation tool MIRAGE, demonstrating both the accuracy of EMTG's model and an operational mission design and navigation procedure using both tools. The validation is performed using a benchmark mission to the Jupiter Trojans.

  17. Tool development to assess the work related neck and upper limb musculoskeletal disorders among female garment workers in Sri-Lanka.

    PubMed

    Amarasinghe, Nirmalie Champika; De AlwisSenevirathne, Rohini

    2016-10-17

    Musculoskeletal disorders (MSDs) have been identified as a predisposing factor for lesser productivity, but no validated tool has been developed to assess them in the Sri- Lankan context. To develop a validated tool to assess the neck and upper limb MSDs. It comprises three components: item selections, item reduction using principal component analysis, and validation. A tentative self-administrated questionnaire was developed, translated, and pre-tested. Four important domains - neck, shoulder, elbow and wrist - were identified through principal component analysis. Prevalence of any MSDs was 38.1% and prevalence of neck, shoulder, elbow and wrist MSDs are 12.85%, 13.71%, 12%, 13.71% respectively. Content and criterion validity of the tool was assessed. Separate ROC curves were produced and sensitivity and specificity of neck (83.1%, 71.7%), shoulder (97.6%, 91.9%), elbow (98.2%, 87.2%), and wrist (97.6%, 94.9%) was determined. Cronbach's Alpha and correlation coefficient was above 0.7. The tool has high sensitivity, specificity, internal consistency, and test re-test reliability.

  18. Validity Issues in Clinical Assessment.

    ERIC Educational Resources Information Center

    Foster, Sharon L.; Cone, John D.

    1995-01-01

    Validation issues that arise with measures of constructs and behavior are addressed with reference to general reasons for using assessment procedures in clinical psychology. A distinction is made between the representational phase of validity assessment and the elaborative validity phase in which the meaning and utility of scores are examined.…

  19. Creation of a Tool for Assessing Knowledge in Evidence-Based Decision-Making in Practicing Health Care Providers.

    PubMed

    Spurr, Kathy; Dechman, Gail; Lackie, Kelly; Gilbert, Robert

    2016-01-01

    Evidence-based decision-making (EBDM) is the process health care providers (HCPs) use to identify and appraise potential evidence. It supports the integration of best research evidence with clinical expertise and patient values into the decision-making process for patient care. Competence in this process is essential to delivery of optimal care. There is no objective tool that assesses EBDM across HCP groups. This research aimed to develop a content valid tool to assess knowledge of the principles of evidence-based medicine and the EBDM process, for use with all HCPs. A Delphi process was used in the creation of the tool. Pilot testing established its content validity with the added benefit of evaluating HCPs' knowledge of EBDM. Descriptive statistics and multivariate mixed models were used to evaluate individual survey responses in total, as well as within each EBDM component. The tool consisted of 26 multiple-choice questions. A total of 12,884 HCPs in Nova Scotia were invited to participate in the web-based validation study, yielding 818 (6.3%) participants, 471 of whom completed all questions. The mean overall score was 68%. Knowledge in one component, integration of evidence with clinical expertise and patient preferences, was identified as needing development across all HCPs surveyed. A content valid tool for assessing HCP EBDM knowledge was created and can be used to support the development of continuing education programs to enhance EBDM competency.

  20. [Validation of the German version of Eating Assessment Tool for head and neck cancer patients].

    PubMed

    Zaretsky, Eugen; Steinbach-Hundt, Silke; Pluschinski, Petra; Grethel, Isabel

    2018-04-10

    The assessment of subjective swallowing complaints constitutes an important element in a multidimensional, modern management of head and neck cancer patients suffering from dysphagia. For this purpose, an internationally recognized and validated 10-item questionnaire EAT-10 is used that was developed and validated by Belafski et al. in 2008. The purpose of the present study is the translation of EAT-10 into the German language and its validation for head and neck cancer patients. After the translation of EAT-10 into German according to the guidelines for the translation of foreign measuring instruments, a validation of gEAT-10 was carried out on the basis of the sample of 210 head and neck cancer patients. The reliability was determined by means of the internal consistency (Cronbach's Alpha) and item-total correlations (Spearman). The construct validity was verified by the uni- and multivariate analyses of the distribution of gEAT-10 total scores depending on gender, age, BMI, tumor stage and localization as well as type of the oncological therapy. The internal consistency amounted to α = .94, the item-total correlations varied between ρ = .59 and ρ = .85. No significant associations between gEAT-10 total scores and gender as well as age were identified in univariate calculations. Such associations were found for BMI, tumor stage and localization as well as type of the oncological therapy. However, only the tumor stage yielded a significant result in a regression. The gEAT-10 was shown to be a reliable and construct valid questionnaire for the assessment of subjective swallowing complaints in patients with head and neck cancer. © Georg Thieme Verlag KG Stuttgart · New York.

  1. Verification and Validation of NASA-Supported Enhancements to Decision Support Tools of PECAD

    NASA Technical Reports Server (NTRS)

    Ross, Kenton W.; McKellip, Rodney; Moore, Roxzana F.; Fendley, Debbie

    2005-01-01

    This section of the evaluation report summarizes the verification and validation (V&V) of recently implemented, NASA-supported enhancements to the decision support tools of the Production Estimates and Crop Assessment Division (PECAD). The implemented enhancements include operationally tailored Moderate Resolution Imaging Spectroradiometer (MODIS) products and products of the Global Reservoir and Lake Monitor (GRLM). The MODIS products are currently made available through two separate decision support tools: the MODIS Image Gallery and the U.S. Department of Agriculture (USDA) Foreign Agricultural Service (FAS) MODIS Normalized Difference Vegetation Index (NDVI) Database. Both the Global Reservoir and Lake Monitor and MODIS Image Gallery provide near-real-time products through PECAD's CropExplorer. This discussion addresses two areas: 1. Assessments of the standard NASA products on which these enhancements are based. 2. Characterizations of the performance of the new operational products.

  2. Development and Validation of a Fine-Motor Assessment Tool for Use with Young Children in a Chinese Population

    ERIC Educational Resources Information Center

    Siu, Andrew M. H.; Lai, Cynthia Y. Y.; Chiu, Amy S. M.; Yip, Calvin C. K.

    2011-01-01

    Objectives: Most of the fine-motor assessment tools used in Hong Kong have been designed in Western countries, so there is a need to develop a standardized assessment which is relevant to the culture and daily living tasks of the local (that is, Chinese) population. This study aimed to (1) develop a fine-motor assessment tool (the Hong Kong…

  3. Validity of the Nintendo Wii® balance board for the assessment of standing balance in Parkinson's disease.

    PubMed

    Holmes, Jeffrey D; Jenkins, Mary E; Johnson, Andrew M; Hunt, Michael A; Clark, Ross A

    2013-04-01

    Impaired postural stability places individuals with Parkinson's at an increased risk for falls. Given the high incidence of fall-related injuries within this population, ongoing assessment of postural stability is important. To evaluate the validity of the Nintendo Wii(®) balance board as a measurement tool for the assessment of postural stability in individuals with Parkinson's. Twenty individuals with Parkinson's participated. Subjects completed testing on two balance tasks with eyes open and closed on a Wii(®) balance board and biomechanical force platform. Bland-Altman plots and a two-way, random-effects, single measure intraclass correlation coefficient model were used to assess concurrent validity of centre-of-pressure data. Concurrent validity was demonstrated to be excellent across balance tasks (intraclass correlation coefficients = 0.96, 0.98, 0.92, 0.94). This study suggests that the Wii(®) balance board is a valid tool for the quantification of postural stability among individuals with Parkinson's.

  4. Using Qualitative Methods to Improve Questionnaires for Spanish Speakers: Assessing Face Validity of a Food Behavior Checklist

    PubMed Central

    BANNA, JINAN C.; VERA BECERRA, LUZ E.; KAISER, LUCIA L.; TOWNSEND, MARILYN S.

    2015-01-01

    Development of outcome measures relevant to health nutrition behaviors requires a rigorous process of testing and revision. Whereas researchers often report performance of quantitative data collection to assess questionnaire validity and reliability, qualitative testing procedures are often overlooked. This report outlines a procedure for assessing face validity of a Spanish-language dietary assessment tool. Reviewing the literature produced no rigorously validated Spanish-language food behavior assessment tools for the US Department of Agriculture’s food assistance and education programs. In response to this need, this study evaluated the face validity of a Spanish-language food behavior checklist adapted from a 16-item English version of a food behavior checklist shown to be valid and reliable for limited-resource English speakers. The English version was translated using rigorous methods involving initial translation by one party and creation of five possible versions. Photos were modified based on client input and new photos were taken as necessary. A sample of low-income, Spanish-speaking women completed cognitive interviews (n=20). Spanish translation experts (n=7) fluent in both languages and familiar with both cultures made minor modifications but essentially approved client preferences. The resulting checklist generated a readability score of 93, indicating low reading difficulty. The Spanish-language checklist has adequate face validity in the target population and is ready for further validation using convergent measures. At the conclusion of testing, this instrument may be used to evaluate nutrition education interventions in California. These qualitative procedures provide a framework for designing evaluation tools for low-literate audiences participating in the US Department of Agriculture food assistance and education programs. PMID:20102831

  5. Using qualitative methods to improve questionnaires for Spanish speakers: assessing face validity of a food behavior checklist.

    PubMed

    Banna, Jinan C; Vera Becerra, Luz E; Kaiser, Lucia L; Townsend, Marilyn S

    2010-01-01

    Development of outcome measures relevant to health nutrition behaviors requires a rigorous process of testing and revision. Whereas researchers often report performance of quantitative data collection to assess questionnaire validity and reliability, qualitative testing procedures are often overlooked. This report outlines a procedure for assessing face validity of a Spanish-language dietary assessment tool. Reviewing the literature produced no rigorously validated Spanish-language food behavior assessment tools for the US Department of Agriculture's food assistance and education programs. In response to this need, this study evaluated the face validity of a Spanish-language food behavior checklist adapted from a 16-item English version of a food behavior checklist shown to be valid and reliable for limited-resource English speakers. The English version was translated using rigorous methods involving initial translation by one party and creation of five possible versions. Photos were modified based on client input and new photos were taken as necessary. A sample of low-income, Spanish-speaking women completed cognitive interviews (n=20). Spanish translation experts (n=7) fluent in both languages and familiar with both cultures made minor modifications but essentially approved client preferences. The resulting checklist generated a readability score of 93, indicating low reading difficulty. The Spanish-language checklist has adequate face validity in the target population and is ready for further validation using convergent measures. At the conclusion of testing, this instrument may be used to evaluate nutrition education interventions in California. These qualitative procedures provide a framework for designing evaluation tools for low-literate audiences participating in the US Department of Agriculture food assistance and education programs. Copyright 2010 American Dietetic Association. Published by Elsevier Inc. All rights reserved.

  6. Competency assessment tools: An exploration of the pedagogical issues facing competency assessment for nurses in the clinical environment.

    PubMed

    Franklin, Natasha; Melville, Paula

    2015-01-01

    Competency assessment is a paradigm that is common in the healthcare environment and this being particularly true within the nursing profession. Demonstration of competence is necessary to meet the requirements of healthcare organisations and is a mandated requirement of nurses by the Nursing and Midwifery Board of Australia. Within the nursing education sector, one approach to determine competence, is through the use of competency assessment tools. Despite widespread use of competency assessment tools there remains ongoing concerns relating to the efficacy of competency assessment tools as a mean to demonstrate 'competency' amongst enrolled and registered nurses in the clinical environment. The authors of this paper ascertain that competency assessment tools run a serious risk of being nothing more than a 'quick-fix' means of assessment to demonstrate 'nursing competence' required for key performance indicators and clinical governance and that will provide evidence for accreditation standards. Based on this premise, the authors, provide an alternative approach to the use of competency assessment tools that moves away from a 'tick-box' approach to a 'patient-centred' competency model. This approach increases the reliability and validity of competency assessments, allows for the recognition of the knowledge, skills and experience of individual nurses, offers a more satisfying and rewarding approach to demonstrating 'competency' for nurses and finally, demonstrates 'real-life' competency.

  7. Reliability and validity assessment of gastrointestinal dystemperaments questionnaire: a novel scale in Persian traditional medicine

    PubMed Central

    Hoseinzadeh, Hamidreza; Taghipour, Ali; Yousefi, Mahdi

    2018-01-01

    Background Development of a questionnaire based on the resources of Persian traditional medicine seems necessary. One of the problems faced by practitioners of traditional medicine is the different opinions regarding the diagnosis of general temperament or temperament of member. One of the reasons is the lack of validity tools, and it has led to difficulties in training the student of traditional medicine and the treatment of patients. The differences in the detection methods, have given rise to several treatment methods. Objective The present study aimed to develop a questionnaire and standard software for diagnosis of gastrointestinal dystemperaments. Methods The present research is a tool developing study which included 8 stages of developing the items, determining the statements based on items, assessing the face validity, assessing the content validity, assessing the reliability, rating the items, developing a software for calculation of the total score of the questionnaire named GDS v.1.1, and evaluating the concurrent validity using statistical tests including Cronbach’s alpha coefficient, Cohen’s kappa coefficient. Results Based on the results, 112 notes including 62 symptoms were extracted from resources, and 58 items were obtained from in-person interview sessions with a panel of experts. A statement was selected for each item and, after merging a number of statements, a total of 49 statements were finally obtained. By calculating the score of statement impact and determining the content validity, respectively, 6 and 10 other items were removed from the list of statements. Standardized Cronbach’s alpha for this questionnaire was obtained 0.795 and its concurrent validity was equal to 0.8. Conclusion A quantitative tool was developed for diagnosis and examination of gastrointestinal dystemperaments. The developed questionnaire is adequately reliable and valid for this purpose. In addition, the software can be used for clinical diagnosis. PMID

  8. Consequential Validity of an Assistive Technology Supplement for the School Function Assessment

    ERIC Educational Resources Information Center

    Silverman, Michelle Kaye; Smith, Roger O.

    2006-01-01

    Educators and therapists implement assistive technology to maximize educational outcomes of students with disabilities. However, few measure the outcomes of interventions because of a lack of valid measurement tools. This study investigated whether an assistive technology supplement for the School Function Assessment demonstrates an important…

  9. Validity and reliability of the Turkish version of the pressure ulcer prevention knowledge assessment instrument.

    PubMed

    Tulek, Zeliha; Polat, Cansu; Ozkan, Ilknur; Theofanidis, Dimitris; Togrol, Rifat Erdem

    2016-11-01

    Sound knowledge of pressure ulcers is important to enable good prevention. There are limited instruments assessing pressure ulcer knowledge. The Pressure Ulcer Prevention Knowledge Assessment Instrument is among the scales of which psychometric properties have been studied rigorously and reflects the latest evidence. This study aimed to evaluate the validity and reliability of the Turkish version of the Pressure Ulcer Prevention Knowledge Assessment Instrument (PUPKAI-T), an instrument that assesses knowledge of pressure ulcer prevention by using multiple-choice questions. Linguistic validity was verified through front-to-back translation. Psychometric properties of the instrument were studied on a sample of 150 nurses working in a tertiary hospital in Istanbul, Turkey. The content validity index of the translated instrument was 0.94, intra-class correlation coefficients were between 0.37 and 0.80, item difficulty indices were between 0.21 and 0.88, discrimination indices were 0.20-0.78, and the Kuder Richardson for the internal consistency was 0.803. The PUPKAI-T was found to be a valid and reliable tool to evaluate nurses' knowledge on pressure ulcer prevention. The PUPKAI-T may be a useful tool for determining educational needs of nurses on pressure ulcer prevention. Copyright © 2016 Tissue Viability Society. Published by Elsevier Ltd. All rights reserved.

  10. Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training.

    PubMed

    Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

    2014-03-01

    Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists⁄surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The 'Endoscopy Task Load Index' was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers.

  11. Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions.

    PubMed

    Barton, K L; Wrieden, W L; Anderson, A S

    2011-12-01

    Food skills programmes are widely used as a means to improve confidence in food preparation, the use of basic food skills and food selections amongst low income communities. However, the impact of such interventions are rarely evaluated as a result of a lack of validated assessment tools appropriate for use within this target group. A two-page questionnaire utilising a closed-question format was designed based on key domains known to be influenced by cooking skills programmes. Content validity was assessed by a panel of public health experts and face validity by individuals, typical of those who may attend cooking skills classes. Internal and repeat reliability were assessed with groups of adults attending community-based classes. The feasibility of using the tool in community settings was also assessed. The draft questionnaire was amended as appropriate subsequent to content and face validity testing. Cronbach's alpha for confidence and knowledge sections was 0.86 and 0.84, respectively, indicating good internal consistency. Spearman correlation coefficients for repeat reliability testing between time 1 and time 2 for each item were in the range 0.46-0.91 (all significant at P < 0.001), indicating that the questionnaire elicited stable responses for repeated use. Feasibility testing highlighted the need for detailed instructions for course tutors on how to distribute and check questionnaires for completion. This tool provides a standardised method of evaluating cooking skills interventions that could be utilised in the development and evaluation of multicentre cooking skills interventions. © 2011 The Authors. Journal of Human Nutrition and Dietetics © 2011 The British Dietetic Association Ltd.

  12. The Chelsea critical care physical assessment tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study.

    PubMed

    Corner, E J; Wood, H; Englebretsen, C; Thomas, A; Grant, R L; Nikoletou, D; Soni, N

    2013-03-01

    To develop a scoring system to measure physical morbidity in critical care - the Chelsea Critical Care Physical Assessment Tool (CPAx). The development process was iterative involving content validity indices (CVI), a focus group and an observational study of 33 patients to test construct validity against the Medical Research Council score for muscle strength, peak cough flow, Australian Therapy Outcome Measures score, Glasgow Coma Scale score, Bloomsbury sedation score, Sequential Organ Failure Assessment score, Short Form 36 (SF-36) score, days of mechanical ventilation and inter-rater reliability. Trauma and general critical care patients from two London teaching hospitals. Users of the CPAx felt that it possessed content validity, giving a final CVI of 1.00 (P<0.05). Construct validation data showed moderate to strong significant correlations between the CPAx score and all secondary measures, apart from the mental component of the SF-36 which demonstrated weak correlation with the CPAx score (r=0.024, P=0.720). Reliability testing showed internal consistency of α=0.798 and inter-rater reliability of κ=0.988 (95% confidence interval 0.791 to 1.000) between five raters. This pilot work supports proof of concept of the CPAx as a measure of physical morbidity in the critical care population, and is a cogent argument for further investigation of the scoring system. Copyright © 2012 Chartered Society of Physiotherapy. Published by Elsevier Ltd. All rights reserved.

  13. Development and assessment of the Alberta Context Tool.

    PubMed

    Estabrooks, Carole A; Squires, Janet E; Cummings, Greta G; Birdsell, Judy M; Norton, Peter G

    2009-12-15

    The context of healthcare organizations such as hospitals is increasingly accepted as having the potential to influence the use of new knowledge. However, the mechanisms by which the organizational context influences evidence-based practices are not well understood. Current measures of organizational context lack a theory-informed approach, lack construct clarity and generally have modest psychometric properties. This paper presents the development and initial psychometric validation of the Alberta Context Tool (ACT), an eight dimension measure of organizational context for healthcare settings. Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS) framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially modifiable. The English version of the ACT was completed by 764 nurses (752 valid responses) working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity. Factor analysis indicated a 13-factor solution (accounting for 59.26% of the variance in 'organizational context'). The composition of the factors was similar to those originally conceptualized. Cronbach's alpha for the 13 factors ranged from .54 to .91 with 4 factors performing below the commonly accepted alpha cut off of .70. Bivariate associations between instrumental research utilization levels (which the ACT was developed to predict) and the ACT's 13 factors were statistically significant at the 5% level for 12 of the 13 factors. Each factor also showed a trend of increasing mean score ranging from the lowest level to the

  14. Development and assessment of the Alberta Context Tool

    PubMed Central

    2009-01-01

    Background The context of healthcare organizations such as hospitals is increasingly accepted as having the potential to influence the use of new knowledge. However, the mechanisms by which the organizational context influences evidence-based practices are not well understood. Current measures of organizational context lack a theory-informed approach, lack construct clarity and generally have modest psychometric properties. This paper presents the development and initial psychometric validation of the Alberta Context Tool (ACT), an eight dimension measure of organizational context for healthcare settings. Methods Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS) framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially modifiable. The English version of the ACT was completed by 764 nurses (752 valid responses) working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity. Results Factor analysis indicated a 13-factor solution (accounting for 59.26% of the variance in 'organizational context'). The composition of the factors was similar to those originally conceptualized. Cronbach's alpha for the 13 factors ranged from .54 to .91 with 4 factors performing below the commonly accepted alpha cut off of .70. Bivariate associations between instrumental research utilization levels (which the ACT was developed to predict) and the ACT's 13 factors were statistically significant at the 5% level for 12 of the 13 factors. Each factor also showed a trend of increasing mean score ranging

  15. Reliability and validity analysis of the transfer assessment instrument.

    PubMed

    McClure, Laura A; Boninger, Michael L; Ozawa, Haishin; Koontz, Alicia

    2011-03-01

    To describe the development and evaluate the reliability and validity of a newly created outcome measure, the Transfer Assessment Instrument (TAI), to assess the quality of transfers performed by full-time wheelchair users. Repeated measures. 2009 National Veterans Wheelchair Games in Spokane, WA. A convenience sample of full-time wheelchair users (N=40) who perform sitting pivot or standing pivot transfers. Not applicable. Intraclass correlation coefficients (ICCs) for reliability and Spearman correlation coefficients for concurrent validity between the TAI and a global assessment scale (0-100 visual analog scale [VAS]). No adverse events occurred during testing. Intrarater ICCs for 3 raters ranged between .35 and .89, and the interrater ICC was .642. Correlations between the TAI and a global assessment VAS ranged between .19 (P=.285) and .69 (P>.000). Item analyses of the tool found a wide range of results, from weak to good reliability. Evaluators found the TAI to be safe and able to be completed in a short time. The TAI is a safe, quick outcome measure that uses equipment typically found in a clinical setting and does not ask participants to perform new skills. Reliability and validity testing found the TAI to have acceptable interrater and a wide range of intrarater reliability. Future work indicates the need for continued refinement including removal or modification of items found to have low reliability, improved education for clinicians, and further reliability and validity analysis with a more diverse subject population. The TAI has the potential to fill a void in assessment of transfers. Copyright © 2011 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

  16. Helping coaches apply the principles of representative learning design: validation of a tennis specific practice assessment tool.

    PubMed

    Krause, Lyndon; Farrow, Damian; Reid, Machar; Buszard, Tim; Pinder, Ross

    2018-06-01

    Representative Learning Design (RLD) is a framework for assessing the degree to which experimental or practice tasks simulate key aspects of specific performance environments (i.e. competition). The key premise being that when practice replicates the performance environment, skills are more likely to transfer. In applied situations, however, there is currently no simple or quick method for coaches to assess the key concepts of RLD (e.g. during on-court tasks). The aim of this study was to develop a tool for coaches to efficiently assess practice task design in tennis. A consensus-based tool was developed using a 4-round Delphi process with 10 academic and 13 tennis-coaching experts. Expert consensus was reached for the inclusion of seven items, each consisting of two sub-questions related to (i) the task goal and (ii) the relevance of the task to competition performance. The Representative Practice Assessment Tool (RPAT) is proposed for use in assessing and enhancing practice task designs in tennis to increase the functional coupling between information and movement, and to maximise the potential for skill transfer to competition contexts.

  17. External Validation of a Tool Predicting 7-Year Risk of Developing Cardiovascular Disease, Type 2 Diabetes or Chronic Kidney Disease.

    PubMed

    Rauh, Simone P; Rutters, Femke; van der Heijden, Amber A W A; Luimes, Thomas; Alssema, Marjan; Heymans, Martijn W; Magliano, Dianna J; Shaw, Jonathan E; Beulens, Joline W; Dekker, Jacqueline M

    2018-02-01

    Chronic cardiometabolic diseases, including cardiovascular disease (CVD), type 2 diabetes (T2D) and chronic kidney disease (CKD), share many modifiable risk factors and can be prevented using combined prevention programs. Valid risk prediction tools are needed to accurately identify individuals at risk. We aimed to validate a previously developed non-invasive risk prediction tool for predicting the combined 7-year-risk for chronic cardiometabolic diseases. The previously developed tool is stratified for sex and contains the predictors age, BMI, waist circumference, use of antihypertensives, smoking, family history of myocardial infarction/stroke, and family history of diabetes. This tool was externally validated, evaluating model performance using area under the receiver operating characteristic curve (AUC)-assessing discrimination-and Hosmer-Lemeshow goodness-of-fit (HL) statistics-assessing calibration. The intercept was recalibrated to improve calibration performance. The risk prediction tool was validated in 3544 participants from the Australian Diabetes, Obesity and Lifestyle Study (AusDiab). Discrimination was acceptable, with an AUC of 0.78 (95% CI 0.75-0.81) in men and 0.78 (95% CI 0.74-0.81) in women. Calibration was poor (HL statistic: p < 0.001), but improved considerably after intercept recalibration. Examination of individual outcomes showed that in men, AUC was highest for CKD (0.85 [95% CI 0.78-0.91]) and lowest for T2D (0.69 [95% CI 0.65-0.74]). In women, AUC was highest for CVD (0.88 [95% CI 0.83-0.94)]) and lowest for T2D (0.71 [95% CI 0.66-0.75]). Validation of our previously developed tool showed robust discriminative performance across populations. Model recalibration is recommended to account for different disease rates. Our risk prediction tool can be useful in large-scale prevention programs for identifying those in need of further risk profiling because of their increased risk for chronic cardiometabolic diseases.

  18. The development and assessment of an evaluation tool for pediatric resident competence in leading simulated pediatric resuscitations.

    PubMed

    Grant, Estée C; Grant, Vincent J; Bhanji, Farhan; Duff, Jonathan P; Cheng, Adam; Lockyer, Jocelyn M

    2012-07-01

    It is critical that competency in pediatric resuscitation is achieved and assessed during residency or post graduate medical training. The purpose of this study was to create and evaluate a tool to measure all elements of pediatric resuscitation team leadership competence. An initial set of items, derived from a literature review and a brainstorming session, were refined to a 26 item assessment tool through the use of Delphi methodology. The tool was tested using videos of standardized resuscitations. A psychometric assessment of the evidence for instrument validity and reliability was undertaken. The performance of 30 residents on two videotaped scenarios was assessed by 4 pediatricians using the tool, with 12 items assessing 'leadership and communication skills' (LCS) and 14 items assessing 'knowledge and clinical skills' (KCS). The instrument showed evidence of reliability; the Cronbach's alpha and generalizability co-efficients for the overall instrument were α=0.818 and Ep(2)=0.76, for LCS were α=0.827 and Ep(2)=0.844, and for KCS were α=0.673 and Ep(2)=0.482. While validity was initially established through literature review and brainstorming by the panel of experts, it was further built through the high strength of correlation between global scores and scores for overall performance (r=0.733), LCS (r=0.718) and KCS (r=0.662) as well as the factor analysis which accounted for 40.2% of the variance. The results of the study demonstrate that the instrument is a valid and reliable tool to evaluate pediatric resuscitation team leader competence. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

  19. Reliability and Validity of the Greek Migraine Disability Assessment (MIDAS) Questionnaire.

    PubMed

    Oikonomidi, Theodora; Vikelis, Michail; Artemiadis, Artemios; Chrousos, George P; Darviri, Christina

    2018-03-01

    The Migraine Disability Assessment (MIDAS) Questionnaire is a reliable and valid instrument for migraine-related disability. Such a tool is needed to quantify migraine-related disability in the Greek population. This validation study aims to assess the test-retest reliability, internal consistency, item discriminant and convergent validity of the Greek translation of the MIDAS. Adults diagnosed with migraine completed the MIDAS Questionnaire on two occasions 3 weeks apart to assess reliability, and completed the RAND-36 to assess validity. Participants (n = 152) had a median MIDAS score of 24 and mostly severe disability (58% were grade IV). The test-retest reliability analysis (N = 59) revealed excellent reliability for the total score. Internal consistency was α = 0.71 for initial and α = 0.82 for retest completion. For item discriminant validity, the correlations between each question and the total score were significant, with high correlations for questions 2-5 (range 0.67 ≤ r ≤ 0.79; p < 0.01). For convergent validity, there was significant negative correlation between the total score and all RAND-36 subscales except for 'emotional wellbeing'. The negative correlation indicates that patients with a lower degree of disability according to their MIDAS score tended to have better wellbeing. Psychometric properties are comparable with those of other published validation studies of the MIDAS and the original. Findings on question 1 show that missing work/school days may be closely related with increased affect issues. The Greek version of the MIDAS Questionnaire has good reliability and validity. This study allowed for cross-cultural comparability of research findings.

  20. Design and validation of a questionnaire to assess organizational culture in French hospital wards.

    PubMed

    Saillour-Glénisson, F; Domecq, S; Kret, M; Sibe, M; Dumond, J P; Michel, P

    2016-09-17

    Although many organizational culture questionnaires have been developed, there is a lack of any validated multidimensional questionnaire assessing organizational culture at hospital ward level and adapted to health care context. Facing the lack of an appropriate tool, a multidisciplinary team designed and validated a dimensional organizational culture questionnaire for healthcare settings to be administered at ward level. A database of organizational culture items and themes was created after extensive literature review. Items were regrouped into dimensions and subdimensions (classification validated by experts). Pre-test and face validation was conducted with 15 health care professionals. In a stratified cluster random sample of hospitals, the psychometric validation was conducted in three phases on a sample of 859 healthcare professionals from 36 multidisciplinary medicine services: 1) the exploratory phase included a description of responses' saturation levels, factor and correlations analyses and an internal consistency analysis (Cronbach's alpha coefficient); 2) confirmatory phase used the Structural Equation Modeling (SEM); 3) reproducibility was studied by a test-retest. The overall response rate was 80 %; the completion average was 97 %. The metrological results were: a global Cronbach's alpha coefficient of 0.93, higher than 0.70 for 12 sub-dimensions; all Dillon-Goldstein's rho coefficients higher than 0.70; an excellent quality of external model with a Goodness of Fitness (GoF) criterion of 0.99. Seventy percent of the items had a reproducibility ranging from moderate (Intra-Class Coefficient between 50 and 70 % for 25 items) to good (ICC higher than 70 % for 33 items). COMEt (Contexte Organisationnel et Managérial en Etablissement de Santé) questionnaire is a validated multidimensional organizational culture questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first dimensional organizational culture questionnaire

  1. Development of a research ethics knowledge and analytical skills assessment tool.

    PubMed

    Taylor, Holly A; Kass, Nancy E; Ali, Joseph; Sisson, Stephen; Bertram, Amanda; Bhan, Anant

    2012-04-01

    The goal of this project was to develop and validate a new tool to evaluate learners' knowledge and skills related to research ethics. A core set of 50 questions from existing computer-based online teaching modules were identified, refined and supplemented to create a set of 74 multiple-choice, true/false and short answer questions. The questions were pilot-tested and item discrimination was calculated for each question. Poorly performing items were eliminated or refined. Two comparable assessment tools were created. These assessment tools were administered as a pre-test and post-test to a cohort of 58 Indian junior health research investigators before and after exposure to a new course on research ethics. Half of the investigators were exposed to the course online, the other half in person. Item discrimination was calculated for each question and Cronbach's α for each assessment tool. A final version of the assessment tool that incorporated the best questions from the pre-/post-test phase was used to assess retention of research ethics knowledge and skills 3 months after course delivery. The final version of the REKASA includes 41 items and had a Cronbach's α of 0.837. The results illustrate, in one sample of learners, the successful, systematic development and use of a knowledge and skills assessment tool in research ethics capable of not only measuring basic knowledge in research ethics and oversight but also assessing learners' ability to apply ethics knowledge to the analytical task of reasoning through research ethics cases, without reliance on essay or discussion-based examination. These promising preliminary findings should be confirmed with additional groups of learners.

  2. Quantitative Analysis of the Rubric as an Assessment Tool: An Empirical Study of Student Peer-Group Rating

    ERIC Educational Resources Information Center

    Hafner, John C.; Hafner, Patti M.

    2003-01-01

    Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool "in the hands of the students." This study focuses on the validity and reliability of the rubric as…

  3. Development of a music therapy assessment tool for patients in low awareness states.

    PubMed

    Magee, Wendy L

    2007-01-01

    People in low awareness states following profound brain injury typically demonstrate subtle changes in functional behaviors which challenge the sensitivity of measurement tools. Failure to identify and measure changes in functioning can lead to misdiagnosis and withdrawal of treatment with this population. Thus, the development of tools which are sensitive to responsiveness is of central concern. As the auditory modality has been found to be particularly sensitive in identifying responses indicating awareness, a convincing case can be made for music therapy as a treatment medium. However, little has been recommended about protocols for intervention or tools for measuring patient responses within the music therapy setting. This paper presents the rationale for an assessment tool specifically designed to measure responses in the music therapy setting with patients who are diagnosed as minimally conscious or in a vegetative state. Developed over fourteen years as part of interdisciplinary assessment and treatment, the music therapy assessment tool for low awareness states (MATLAS) contains fourteen items which rate behavioral responses across a number of domains. The tool can provide important information for interdisciplinary assessment and treatment particularly in the auditory and communication domains. Recommendations are made for testing its reliability and validity through research.

  4. Validation of a Nutrition Screening Tool for Pediatric Patients with Cystic Fibrosis.

    PubMed

    Souza Dos Santos Simon, Miriam Isabel; Forte, Gabriele Carra; da Silva Pereira, Juliane; da Fonseca Andrade Procianoy, Elenara; Drehmer, Michele

    2016-05-01

    In cystic fibrosis (CF), nutrition diagnosis is of critical relevance because the early identification of nutrition-related compromise enables early, adequate intervention and, consequently, influences patient prognosis. Up to now, there has not been a validated nutrition screening tool that takes into consideration clinical variables. To validate a specific nutritional risk screening tool for patients with CF based on clinical variables, anthropometric parameters, and dietary intake. Cross-sectional study. The nutrition screening tool was compared with a risk screening tool proposed by McDonald and the Cystic Fibrosis Foundation criteria. Patients aged 6 to 18 years, with a diagnosis of CF confirmed by two determinations of elevated chloride level in sweat (sweat test) and/or by identification of two CF-associated genetic mutations who were receiving follow-up care through the outpatient clinic of a Cystic Fibrosis Treatment Center. Earlier identification of nutritional risk in CF patients aged 6 to 18 years when a new screening tool was applied. Agreement among the tested methods was assessed by means of the kappa coefficient for categorical variables. Sensitivity, specificity, and accuracy values were calculated. The significance level was set at 5% (P<0.05). Statistical analyses were carried out in PASW Statistics for Windows version 18.0 (2009, SPSS Inc). Eighty-two patients (49% men, aged 6 to 18 years) were enrolled in the study. The agreement between the proposed screening tool and the tool for screening nutritional risk for CF by the McDonald method was good (κ=0.804; P<0.001) and the sensitivity and specificity was 85% and 95%, respectively. Agreement with the Cystic Fibrosis Foundation criteria was lower (κ=0.418; P<0.001), and the sensitivity and specificity were both 72%. The proposed screening tool with defined clinical variables promotes earlier identification of nutritional risk in pediatric patients with CF. Copyright © 2016 Academy of Nutrition

  5. Objective Evaluation Tool for Texture-Modified Food (OET-TMF): Development of the Tool and Validation.

    PubMed

    Calleja-Fernández, Alicia; Pintor-de-la-Maza, Begoña; Vidal-Casariego, Alfonso; Cano-Rodríguez, Isidoro; Ballesteros-Pomar, María D

    2016-06-01

    Texture-modified diets (TMDs) should fulfil nutritional goals, guarantee homogenous texture, and meet food safety regulations. The food industry has created texture-modified food (TMF) that meets the TMD requirements of quality and safety for inpatients. To design and develop a tool that allows the objective selection of foodstuffs for TMDs that ensures nutritional requirements and swallowing safety of inpatients in order to improve their quality of life, especially regarding their food satisfaction. An evaluation tool was designed to objectively determine the adequacy of food included in the TMD menus of a hospital. The "Objective Evaluation Tool for Texture-Modified Food" (OET-TMF) consists of seven items that evaluate the food's nutritional quality (energy and protein input), presence of allergens, texture and viscosity, cooking, storage type, useful life, and patient acceptance. The total score ranged from 0 to 64 and was divided into four categories: high quality, good quality, medium quality, and low quality. Studying four different commercial TMFs contributed to the validation of the tool. All the evaluated products scored between high and good regarding quality. There was a tendency (p = 0.077) towards higher consumption and a higher overall quality of the product obtained with the OET-TMF. The product that scored highest with the tool was the best accepted; the product with the lowest score had the highest rate of refusal. The OET-TMF allows for the objective discrimination of the quality of TMF. In addition, it shows a certain relationship between the observed and assessed quality intake.

  6. Comparing Quality of Public Primary Care between Hong Kong and Shanghai Using Validated Patient Assessment Tools

    PubMed Central

    Wei, Xiaolin; Li, Haitao; Yang, Nan; Wong, Samuel Y. S.; Owolabi, Onikepe; Xu, Jianguang; Shi, Leiyu; Tang, Jinling; Li, Donald; Griffiths, Sian M.

    2015-01-01

    Objectives Primary care is the key element of health reform in China. The objective of this study was to compare patient assessed quality of public primary care between Hong Kong, a city with established primary care environment influenced by its colonial history, and Shanghai, a city leading primary care reform in Mainland China; and to measure the equity of care in the two cities. Methods Cross sectional stratified random sampling surveys were conducted in 2011. Data were collected from 1,994 respondents in Hong Kong and 811 respondents in Shanghai. A validated Chinese version of the primary care assessment tool was employed to assess perceived quality of primary care with respect to socioeconomic characteristics and health status. Results We analyzed 391 and 725 respondents in Hong Kong and Shanghai, respectively, who were regular public primary care users. Respondents in Hong Kong reported significant lower scores in first contact accessibility (1.59 vs. 2.15), continuity of care (2.33 vs. 3.10), coordination of information (2.84 vs. 3.64), comprehensiveness service availability (2.43 vs. 3.31), comprehensiveness service provided (2.11 vs. 2.40), and the total score (23.40 vs. 27.40), but higher scores in first contact utilization (3.15 vs. 2.54) and coordination of services (2.67 vs. 2.40) when compared with those in Shanghai. Respondents with higher income reported a significantly higher total primary care score in Hong Kong, but not in Shanghai. Conclusions Respondents in Shanghai reported better quality of public primary care than those in Hong Kong, while quality of public primary care tended to be more equitable in Shanghai. PMID:25826616

  7. Validation of learning assessments: A primer.

    PubMed

    Peeters, Michael J; Martin, Beth A

    2017-09-01

    The Accreditation Council for Pharmacy Education's Standards 2016 has placed greater emphasis on validating educational assessments. In this paper, we describe validity, reliability, and validation principles, drawing attention to the conceptual change that highlights one validity with multiple evidence sources; to this end, we recommend abandoning historical (confusing) terminology associated with the term validity. Further, we describe and apply Kane's framework (scoring, generalization, extrapolation, and implications) for the process of validation, with its inferences and conclusions from varied uses of assessment instruments by different colleges and schools of pharmacy. We then offer five practical recommendations that can improve reporting of validation evidence in pharmacy education literature. We describe application of these recommendations, including examples of validation evidence in the context of pharmacy education. After reading this article, the reader should be able to understand the current concept of validation, and use a framework as they validate and communicate their own institution's learning assessments. Copyright © 2017 Elsevier Inc. All rights reserved.

  8. Validation of a motivation survey tool for pharmacy students: Exploring a link to professional identity development.

    PubMed

    Mylrea, Martina F; Sen Gupta, Tarun; Glass, Beverley D

    2017-09-01

    Self-determination theory (SDT), which describes a continuum of motivation regulators, is proposed as an appropriate framework to study pharmacy student motivation. The aim was to develop a Pharmacy Motivation Scale (Pharm-S) to determine motivation regulators in undergraduate students and explore a possible link to professional identity development. The Pharm-S was adapted from the SDT-based, Sports Motivation Scale (SMS-II), and administered to undergraduate students in an Australian pharmacy course. Convergent validity was assessed by conducting a correlation analysis between the Pharm-S and MacLeod Clark Professional Identity Scale (MCPIS-9). Face, content and construct validity were established for the Pharm-S through the analysis of 327 survey responses. Factor analysis extracted four of the six theoretical subscales as proposed by SDT (variance explained: 65.7%). Support for the SDT structure was confirmed by high factor loadings in each of the subscales and acceptable reliability coefficients. Subscale correlations revealed a simplex pattern, supporting the presence of a motivation continuum, as described by SDT. A moderate positive correlation (0.64) between Pharm-S responses and the validated professional identity instrument, MCPIS-9, indicated a possible link between levels of motivation and professional identity. and conclusions: Content and structural validity and internal consistency of the Pharm-S confirmed the reliability of the Pharm-S as a valid tool to assess motivational regulators. Pharm-S and the MCPIS-9 were positively correlated, lending support to a link between motivation and professional identity. This suggests a potential role for the Pharm-S as a valid tool to measure pharmacy student professional identity development. Crown Copyright © 2017. Published by Elsevier Inc. All rights reserved.

  9. The Dutch motor skills assessment as tool for talent development in table tennis: a reproducibility and validity study.

    PubMed

    Faber, Irene R; Nijhuis-Van Der Sanden, Maria W G; Elferink-Gemser, Marije T; Oosterveld, Frits G J

    2015-01-01

    A motor skills assessment could be helpful in talent development by estimating essential perceptuo-motor skills of young players, which are considered requisite to develop excellent technical and tactical qualities. The Netherlands Table Tennis Association uses a motor skills assessment in their talent development programme consisting of eight items measuring perceptuo-motor skills specific to table tennis under varying conditions. This study aimed to investigate this assessment regarding its reproducibility, internal consistency, underlying dimensions and concurrent validity in 113 young table tennis players (6-10 years). Intraclass correlation coefficients of six test items met the criteria of 0.7 with coefficients of variation between 3% and 8%. Cronbach's alpha valued 0.853 for internal consistency. The principal components analysis distinguished two conceptually meaningful factors: "ball control" and "gross motor function." Concurrent validity analyses demonstrated moderate associations between the motor skills assessment's results and national ranking; boys r = -0.53 (P < 0.001) and girls r = -0.45 (P = 0.015). In conclusion, this evaluation demonstrated six test items with acceptable reproducibility, good internal consistency and good prospects for validity. Two test items need revision to upgrade reproducibility. Since the motor skills assessment seems to be a reproducible, objective part of a talent development programme, more longitudinal studies are required to investigate its predictive validity.

  10. Validation of biomarkers of food intake-critical assessment of candidate biomarkers.

    PubMed

    Dragsted, L O; Gao, Q; Scalbert, A; Vergères, G; Kolehmainen, M; Manach, C; Brennan, L; Afman, L A; Wishart, D S; Andres Lacueva, C; Garcia-Aloy, M; Verhagen, H; Feskens, E J M; Praticò, G

    2018-01-01

    Biomarkers of food intake (BFIs) are a promising tool for limiting misclassification in nutrition research where more subjective dietary assessment instruments are used. They may also be used to assess compliance to dietary guidelines or to a dietary intervention. Biomarkers therefore hold promise for direct and objective measurement of food intake. However, the number of comprehensively validated biomarkers of food intake is limited to just a few. Many new candidate biomarkers emerge from metabolic profiling studies and from advances in food chemistry. Furthermore, candidate food intake biomarkers may also be identified based on extensive literature reviews such as described in the guidelines for Biomarker of Food Intake Reviews (BFIRev). To systematically and critically assess the validity of candidate biomarkers of food intake, it is necessary to outline and streamline an optimal and reproducible validation process. A consensus-based procedure was used to provide and evaluate a set of the most important criteria for systematic validation of BFIs. As a result, a validation procedure was developed including eight criteria, plausibility, dose-response, time-response, robustness, reliability, stability, analytical performance, and inter-laboratory reproducibility. The validation has a dual purpose: (1) to estimate the current level of validation of candidate biomarkers of food intake based on an objective and systematic approach and (2) to pinpoint which additional studies are needed to provide full validation of each candidate biomarker of food intake. This position paper on biomarker of food intake validation outlines the second step of the BFIRev procedure but may also be used as such for validation of new candidate biomarkers identified, e.g., in food metabolomic studies.

  11. Development and validation of a tool to evaluate the quality of medical education websites in pathology.

    PubMed

    Alyusuf, Raja H; Prasad, Kameshwar; Abdel Satir, Ali M; Abalkhail, Ali A; Arora, Roopa K

    2013-01-01

    The exponential use of the internet as a learning resource coupled with varied quality of many websites, lead to a need to identify suitable websites for teaching purposes. The aim of this study is to develop and to validate a tool, which evaluates the quality of undergraduate medical educational websites; and apply it to the field of pathology. A tool was devised through several steps of item generation, reduction, weightage, pilot testing, post-pilot modification of the tool and validating the tool. Tool validation included measurement of inter-observer reliability; and generation of criterion related, construct related and content related validity. The validated tool was subsequently tested by applying it to a population of pathology websites. Reliability testing showed a high internal consistency reliability (Cronbach's alpha = 0.92), high inter-observer reliability (Pearson's correlation r = 0.88), intraclass correlation coefficient = 0.85 and κ =0.75. It showed high criterion related, construct related and content related validity. The tool showed moderately high concordance with the gold standard (κ =0.61); 92.2% sensitivity, 67.8% specificity, 75.6% positive predictive value and 88.9% negative predictive value. The validated tool was applied to 278 websites; 29.9% were rated as recommended, 41.0% as recommended with caution and 29.1% as not recommended. A systematic tool was devised to evaluate the quality of websites for medical educational purposes. The tool was shown to yield reliable and valid inferences through its application to pathology websites.

  12. Translation and Validation of the Dutch Western Ontario Meniscal Evaluation Tool.

    PubMed

    van der Wal, Robert J P; Heemskerk, Bastiaan T J; van Arkel, Ewoud R A; Mokkink, Lidwine B; Thomassen, Bregje J W

    2017-05-01

    The aim of this study was to translate the Western Ontario Meniscal Evaluation Tool (WOMET) into Dutch, to evaluate the content validity, construct validity, reliability, and responsiveness, and to determine the minimal important change (MIC) of the Dutch version. The WOMET was translated into Dutch according to a standardized forward-backward translation protocol. A total of 86 patients (51 males, 35 females, median age 52 years [interquartile range, 43-60 years]) with isolated meniscal pathology were included. The WOMET was completed three times; at baseline, around 2 weeks, and after 3 months from the baseline. Knee injury and osteoarthritis outcome score, International Knee Documentation Committee subjective knee form, short-form 36, and an anchor question were also answered. There were good results for content validity (floor and ceiling effects [< 15%]), construct validity (79% of the predefined hypotheses were confirmed), internal consistency (Cronbach α = 0.87, 0.79, and 0.86 for each subscale score), test-retest reliability (intraclass correlation coefficient = 0.78 for total WOMET score), and responsiveness (79% of the predefined hypotheses were confirmed). The smallest detectable change and MIC for the Dutch WOMET are 20.5 and 14.7, respectively. The Dutch version of the WOMET is valid and reliable for assessing health-related quality of life in patients with meniscal pathology. Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

  13. A new assessment tool for patients with multiple sclerosis from Spanish-speaking countries: validation of the Brief International Cognitive Assessment for MS (BICAMS) in Argentina.

    PubMed

    Vanotti, Sandra; Smerbeck, Audrey; Benedict, Ralph H B; Caceres, Fernando

    2016-10-01

    The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) is an international assessment tool for monitoring cognitive function in multiple sclerosis (MS) patients. BICAMS comprises the Symbol Digit Modalities Test (SDMT), the California Verbal Learning Test - Second Edition (CVLT II) and the Brief Visuospatial Memory Test - Revised (BVMT-R). Our objective was to validate and assess the reliability of BICAMS as applied in Argentina and to obtain normative data in Spanish for this population. The sample composed of 50 MS patients and 100 healthy controls (HC). In order to test its reliability, BICAMS was re-administered in a subset of 25 patients. The sample's average age was 43.42 ± 10.17 years old, and average years of schooling were 14.86 ± 2.78. About 74% of the participants were women. The groups did not differ in age, years of schooling, or gender. The MS group performed significantly worse than the HC group across the three neuropsychological tests, yielding the following Cohen's d values: SDMT: .85; CVLT I: .87; and BVMT-R: .40. The mean raw scores for Argentina normative data were as follows: SDMT: 56.71 ± 10.85; CVLT I: 60.88 ± 10.46; and BVMT-R: 23.44 ± 5.84. Finally, test-retest reliability coefficients for each test were as follows: SDMT: r = .95; CVLT I: r = .87; and BVMT-R: r = .82. This BICAMS version is reliable and useful as a monitoring tool for identifying MS patients with cognitive impairment.

  14. Development, validation and psychometric properties of a diagnostic/prognostic tool for breakthrough pain in mixed chronic-pain patients.

    PubMed

    Samolsky Dekel, Boaz Gedaliahu; Remondini, Francesca; Gori, Alberto; Vasarri, Alessio; Di Nino, GianFranco; Melotti, Rita Maria

    2016-02-01

    Breakthrough pain (BTP) shows variable prevalence in different clinical contexts of cancer and non-cancer patients. BTP diagnostic tools with demonstrated reliability, validation and prognostic capability are lacking. We report the development, psychometric and validation properties of a diagnostic/prognostic tool, the IQ-BTP, for BTP recognition, its likelihood and clinical features among chronic-pain (CP) patients. n=120 consecutive mixed cancer/non-cancer CP in/outpatients. Development, psychometric analyses and formal validation included: Face/Content validity (by 'experts' opinion and assessing the relationship between the IQ-BTP classes and criteria derived from BTP operational-case-definition); Construct validity, by Principle Component Analysis (PCA); and the strength of Spearman correlation between IQ-BTP classes and the Brief Pain Inventory (BPI) items; Reliability, by Cronbach's alpha statistics. Associations with clinical/demographic moderators were assessed applying χ(2) analysis. Potential-BTP was found in 36.7% of patients (38.4% of non-cancer and 32.4% of cancer patients). Among these the likelihood for BTP diagnosis was 'high' in 25%, 'intermediate' in 41% and, 'low' 34% of patients. Analyses showed significant differences between IQ-BTP classes and between the latter BPI pain-item scores. Correlation between IQ-BTP classes and BPI items was moderate. PCA and scree test identified 3 components accounting for 62.3% of the variance. Cronbach's alpha was 0.71. The IQ-BTP showed satisfactory psychometric and validation properties. With adequate feasibility it enabled the allocating of cancer/non-cancer CP patients in three prognostic classes. Results are sufficient to warrant a subsequent impact study of the IQ-BTP as prognostic model and screening tool for BTP in both CP populations. Copyright © 2016 Elsevier B.V. All rights reserved.

  15. Reliable and valid tools for measuring surgeons' teaching performance: residents' vs. self evaluation.

    PubMed

    Boerebach, Benjamin C M; Arah, Onyebuchi A; Busch, Olivier R C; Lombarts, Kiki M J M H

    2012-01-01

    In surgical education, there is a need for educational performance evaluation tools that yield reliable and valid data. This paper describes the development and validation of robust evaluation tools that provide surgeons with insight into their clinical teaching performance. We investigated (1) the reliability and validity of 2 tools for evaluating the teaching performance of attending surgeons in residency training programs, and (2) whether surgeons' self evaluation correlated with the residents' evaluation of those surgeons. We surveyed 343 surgeons and 320 residents as part of a multicenter prospective cohort study of faculty teaching performance in residency training programs. The reliability and validity of the SETQ (System for Evaluation Teaching Qualities) tools were studied using standard psychometric techniques. We then estimated the correlations between residents' and surgeons' evaluations. The response rate was 87% among surgeons and 84% among residents, yielding 2625 residents' evaluations and 302 self evaluations. The SETQ tools yielded reliable and valid data on 5 domains of surgical teaching performance, namely, learning climate, professional attitude towards residents, communication of goals, evaluation of residents, and feedback. The correlations between surgeons' self and residents' evaluations were low, with coefficients ranging from 0.03 for evaluation of residents to 0.18 for communication of goals. The SETQ tools for the evaluation of surgeons' teaching performance appear to yield reliable and valid data. The lack of strong correlations between surgeons' self and residents' evaluations suggest the need for using external feedback sources in informed self evaluation of surgeons. Copyright © 2012 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

  16. Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate-the learning curve for endoscopy training

    PubMed Central

    Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

    2014-01-01

    BACKGROUND: Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. OBJECTIVE: To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. METHODS: The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists/surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The ‘Endoscopy Task Load Index’ was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. RESULTS: From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. DISCUSSION: The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers. PMID:24619638

  17. Validity and reliability of balance assessment software using the Nintendo Wii balance board: usability and validation

    PubMed Central

    2014-01-01

    Background A balance test provides important information such as the standard to judge an individual’s functional recovery or make the prediction of falls. The development of a tool for a balance test that is inexpensive and widely available is needed, especially in clinical settings. The Wii Balance Board (WBB) is designed to test balance, but there is little software used in balance tests, and there are few studies on reliability and validity. Thus, we developed a balance assessment software using the Nintendo Wii Balance Board, investigated its reliability and validity, and compared it with a laboratory-grade force platform. Methods Twenty healthy adults participated in our study. The participants participated in the test for inter-rater reliability, intra-rater reliability, and concurrent validity. The tests were performed with balance assessment software using the Nintendo Wii balance board and a laboratory-grade force platform. Data such as Center of Pressure (COP) path length and COP velocity were acquired from the assessment systems. The inter-rater reliability, the intra-rater reliability, and concurrent validity were analyzed by an intraclass correlation coefficient (ICC) value and a standard error of measurement (SEM). Results The inter-rater reliability (ICC: 0.89-0.79, SEM in path length: 7.14-1.90, SEM in velocity: 0.74-0.07), intra-rater reliability (ICC: 0.92-0.70, SEM in path length: 7.59-2.04, SEM in velocity: 0.80-0.07), and concurrent validity (ICC: 0.87-0.73, SEM in path length: 5.94-0.32, SEM in velocity: 0.62-0.08) were high in terms of COP path length and COP velocity. Conclusion The balance assessment software incorporating the Nintendo Wii balance board was used in our study and was found to be a reliable assessment device. In clinical settings, the device can be remarkably inexpensive, portable, and convenient for the balance assessment. PMID:24912769

  18. Linguistic validation of the Alberta Context Tool and two measures of research use, for German residential long term care.

    PubMed

    Hoben, Matthias; Bär, Marion; Mahler, Cornelia; Berger, Sarah; Squires, Janet E; Estabrooks, Carole A; Kruse, Andreas; Behrens, Johann

    2014-01-31

    To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks' Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity-an initial stage of validity, necessary before more advanced validity assessment. We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants' answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants. Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups. Cognitive debriefing is essential to detect and respond to problematic instrument items, particularly when translating

  19. Validity of Dietary Assessment in Athletes: A Systematic Review

    PubMed Central

    Beck, Kathryn L.; Gifford, Janelle A.; Slater, Gary; Flood, Victoria M.; O’Connor, Helen

    2017-01-01

    Dietary assessment methods that are recognized as appropriate for the general population are usually applied in a similar manner to athletes, despite the knowledge that sport-specific factors can complicate assessment and impact accuracy in unique ways. As dietary assessment methods are used extensively within the field of sports nutrition, there is concern the validity of methodologies have not undergone more rigorous evaluation in this unique population sub-group. The purpose of this systematic review was to compare two or more methods of dietary assessment, including dietary intake measured against biomarkers or reference measures of energy expenditure, in athletes. Six electronic databases were searched for English-language, full-text articles published from January 1980 until June 2016. The search strategy combined the following keywords: diet, nutrition assessment, athlete, and validity; where the following outcomes are reported but not limited to: energy intake, macro and/or micronutrient intake, food intake, nutritional adequacy, diet quality, or nutritional status. Meta-analysis was performed on studies with sufficient methodological similarity, with between-group standardized mean differences (or effect size) and 95% confidence intervals (CI) being calculated. Of the 1624 studies identified, 18 were eligible for inclusion. Studies comparing self-reported energy intake (EI) to energy expenditure assessed via doubly labelled water were grouped for comparison (n = 11) and demonstrated mean EI was under-estimated by 19% (−2793 ± 1134 kJ/day). Meta-analysis revealed a large pooled effect size of −1.006 (95% CI: −1.3 to −0.7; p < 0.001). The remaining studies (n = 7) compared a new dietary tool or instrument to a reference method(s) (e.g., food record, 24-h dietary recall, biomarker) as part of a validation study. This systematic review revealed there are limited robust studies evaluating dietary assessment methods in athletes. Existing literature

  20. Facial expression: An under-utilised tool for the assessment of welfare in mammals.

    PubMed

    Descovich, Kris A; Wathan, Jennifer; Leach, Matthew C; Buchanan-Smith, Hannah M; Flecknell, Paul; Farningham, David; Vick, Sarah-Jane

    2017-01-01

    Animal welfare is a key issue for industries that use or impact upon animals. The accurate identification of welfare states is particularly relevant to the field of bioscience, where the 3Rs framework encourages refinement of experimental procedures involving animal models. The assessment and improvement of welfare states in animals depends on reliable and valid measurement tools. Behavioral measures (activity, attention, posture and vocalization) are frequently used because they are immediate and non-invasive, however no single indicator can yield a complete picture of the internal state of an animal. Facial expressions are extensively studied in humans as a measure of psychological and emotional experiences but are infrequently used in animal studies, with the exception of emerging research on pain behavior. In this review, we discuss current evidence for facial representations of underlying affective states, and how communicative or functional expressions can be useful within welfare assessments. Validated tools for measuring facial movement are outlined, and the potential of expressions as honest signals is discussed, alongside other challenges and limitations to facial expression measurement within the context of animal welfare. We conclude that facial expression determination in animals is a useful but underutilized measure that complements existing tools in the assessment of welfare.

  1. [Emotional Intelligence Index: a tool for the routine assessment of mental health promotion programs in schools].

    PubMed

    Veltro, Franco; Ialenti, Valentina; Morales García, Manuel Alejandro; Gigantesco, Antonella

    2016-01-01

    After critical examination of several aspects relating to the evaluation of some dimensions of emotional intelligence through self-assessment tools, is described the procedure of construction and validation of an Index for its measurement, conceived only for the routine assessment of health promotion programs mental in schools that include among their objectives the improvement of emotional intelligence specifically "outcome-oriented". On the basis of the two most common international tools, are listed 27 items plus 6 of control, illustrated two Focus Group (FG) of students (face validity). The scale obtained by FG was administered to 300 students, and the results were submitted to factorial analysis (construct validity). It was also evaluated the internal consistency with Cronbach's Alpha and studied concurrent validity with the emotional quotient inventory, a scale of perceived self-efficacy and a stress test rating. From the analysis of FG all the original items were modified, deleted 4, and reduced the encoding system from 6 to 4 levels of Likert scale. Of the 23 items included in the analysis have emerged five factors (intra-psychic dimension, interpersonal, impulsivity, adaptive coping, sense of self-efficacy) for a total of 15 items. Very satisfactory were the results of the validation process of internal consistency (0.72) and the concurrent validity. The results are positive. It is obtained in fact the shortest routine assessment tool currently available in Italy which constitutes a real Index, for which compilation are required on average 3 minutes. Is emphasized the characteristic of an Index, and not of questionnaire or interview for clinical use, highlighting the only specific use for mental health promotion programs in schools.

  2. Development and Validation of a Standardized Tool for Prioritization of Information Sources.

    PubMed

    Akwar, Holy; Kloeze, Harold; Mukhi, Shamir

    2016-01-01

    To validate the utility and effectiveness of a standardized tool for prioritization of information sources for early detection of diseases. The tool was developed with input from diverse public health experts garnered through survey. Ten raters used the tool to evaluate ten information sources and reliability among raters was computed. The Proc mixed procedure with random effect statement and SAS Macros were used to compute multiple raters' Fleiss Kappa agreement and Kendall's Coefficient of Concordance. Ten disparate information sources evaluated obtained the following composite scores: ProMed 91%; WAHID 90%; Eurosurv 87%; MediSys 85%; SciDaily 84%; EurekAl 83%; CSHB 78%; GermTrax 75%; Google 74%; and CBC 70%. A Fleiss Kappa agreement of 50.7% was obtained for ten information sources and 72.5% for a sub-set of five sources rated, which is substantial agreement validating the utility and effectiveness of the tool. This study validated the utility and effectiveness of a standardized criteria tool developed to prioritize information sources. The new tool was used to identify five information sources suited for use by the KIWI system in the CEZD-IIR project to improve surveillance of infectious diseases. The tool can be generalized to situations when prioritization of numerous information sources is necessary.

  3. Development and validation of a tool to evaluate the quality of medical education websites in pathology

    PubMed Central

    Alyusuf, Raja H.; Prasad, Kameshwar; Abdel Satir, Ali M.; Abalkhail, Ali A.; Arora, Roopa K.

    2013-01-01

    Background: The exponential use of the internet as a learning resource coupled with varied quality of many websites, lead to a need to identify suitable websites for teaching purposes. Aim: The aim of this study is to develop and to validate a tool, which evaluates the quality of undergraduate medical educational websites; and apply it to the field of pathology. Methods: A tool was devised through several steps of item generation, reduction, weightage, pilot testing, post-pilot modification of the tool and validating the tool. Tool validation included measurement of inter-observer reliability; and generation of criterion related, construct related and content related validity. The validated tool was subsequently tested by applying it to a population of pathology websites. Results and Discussion: Reliability testing showed a high internal consistency reliability (Cronbach's alpha = 0.92), high inter-observer reliability (Pearson's correlation r = 0.88), intraclass correlation coefficient = 0.85 and κ =0.75. It showed high criterion related, construct related and content related validity. The tool showed moderately high concordance with the gold standard (κ =0.61); 92.2% sensitivity, 67.8% specificity, 75.6% positive predictive value and 88.9% negative predictive value. The validated tool was applied to 278 websites; 29.9% were rated as recommended, 41.0% as recommended with caution and 29.1% as not recommended. Conclusion: A systematic tool was devised to evaluate the quality of websites for medical educational purposes. The tool was shown to yield reliable and valid inferences through its application to pathology websites. PMID:24392243

  4. The development of a supportive care needs assessment tool for Indigenous people with cancer

    PubMed Central

    2012-01-01

    Background Little is known about the supportive care needs of Indigenous people with cancer and to date, existing needs assessment tools have not considered cultural issues for this population. We aimed to adapt an existing supportive care needs assessment tool for use with Indigenous Australians with cancer. Methods Face-to-face interviews with Indigenous cancer patients (n = 29) and five focus groups with Indigenous key-informants (n = 23) were conducted to assess the face and content validity, cultural acceptability, utility and relevance of the Supportive Care Needs Survey - Short Form 34 (SCNS-SF34) for use with Indigenous patients with cancer. Results All items from the SCNS-SF34 were shortened and changed to use more appropriate language (e.g. the word 'anxiety' was substituted with 'worry'). Seven questions were omitted (e.g. items on death and future considerations) as they were deemed culturally inappropriate or irrelevant and 12 items were added (e.g. accessible transport). Optional instructions were added before the sexual items. The design and response format of the SCNS-SF34 was modified to make it easier to use for Indigenous cancer patients. Given the extensive modifications to the SCNS-SF34 and the liklihood of a different factor structure we consider this tool to be a new tool rather than a modification. The Supportive care needs assessment tool for Indigenous people (SCNAT-IP) shows promising face and content validity and will be useful in informing services where they need to direct their attention for these patients. Conclusions Indigenous people with cancer have language, customs and specific needs that are not accommodated within the standard SCNS-SF34. Our SCNAT-IP improves acceptability, relevance and face validity for Indigenous-specific concerns. Our SCNAT-IP will allow screening for supportive care needs that are specific to Indigenous cancer patients' and greatly inform targeted policy development and practice. PMID:22817614

  5. The Assessment of Afterschool Program Practices Tool (APT): Findings from the APT Validation Study

    ERIC Educational Resources Information Center

    Tracy, Allison; Surr, Wendy; Richer, Amanda

    2012-01-01

    The Assessment of Afterschool Program Practices Tool ("APT"), developed by the National Institute of Out-of-School Time (NIOST), is an observational instrument designed to measure the aspects of afterschool program quality that research suggests contribute to the 21st century skills, attitudes, and behaviors youth need to be successful…

  6. Assessing college-level learning difficulties and "at riskness" for learning disabilities and ADHD: development and validation of the learning difficulties assessment.

    PubMed

    Kane, Steven T; Walker, John H; Schmidt, George R

    2011-01-01

    This article describes the development and validation of the Learning Difficulties Assessment (LDA), a normed and web-based survey that assesses perceived difficulties with reading, writing, spelling, mathematics, listening, concentration, memory, organizational skills, sense of control, and anxiety in college students. The LDA is designed to (a) map individual learning strengths and weaknesses, (b) provide users with a comparative sense of their academic skills, (c) integrate research in user-interface design to assist those with reading and learning challenges, and (d) identify individuals who may be at risk for learning disabilities and attention-deficit/hyperactivity disorder (ADHD) and who should thus be further assessed. Data from a large-scale 5-year study describing the instrument's validity as a screening tool for learning disabilities and ADHD are presented. This article also describes unique characteristics of the LDA including its user-interface design, normative characteristics, and use as a no-cost screening tool for identifying college students at risk for learning disorders and ADHD.

  7. Validation of self assessment patient knowledge questionnaire for heart failure patients.

    PubMed

    Lainscak, Mitja; Keber, Irena

    2005-12-01

    Several studies showed insufficient knowledge and poor compliance to non-pharmacological management in heart failure patients. Only a limited number of validated tools are available to assess their knowledge. The aim of the study was to test our 10-item Patient knowledge questionnaire. The Patient knowledge questionnaire was administered to 42 heart failure patients from Heart failure clinic and to 40 heart failure patients receiving usual care. Construct validity (Pearson correlation coefficient), internal consistency (Cronbach alpha), reproducibility (Wilcoxon signed rank test), and reliability (chi-square test and Student's t-test for independent samples) were assessed. Overall score of the Patient knowledge questionnaire had the strongest correlation to the question about regular weighing (r=0.69) and the weakest to the question about presence of heart disease (r=0.33). There was a strong correlation between question about fluid retention and questions assessing regular weighing, (r=0.86), weight of one litre of water (r=0.86), and salt restriction (r=0.57). The Cronbach alpha was 0.74 and could be improved by exclusion of questions about clear explanation (Chronbach alpha 0.75), importance of fruit, soup, and vegetables (Chronbach alpha 0.75), and self adjustment of diuretic (Chronbach alpha 0.81). During reproducibility testing 91% to 98% of questions were answered equally. Patients from Heart failure clinic scored significantly better than patients receiving usual care (7.9 (1.3) vs. 5.7 (2.2), p<0.001). Patient knowledge questionnaire is a valid and reliable tool to measure knowledge of heart failure patients.

  8. Evaluation of the internal construct validity of the Personal Care Participation Assessment and Resource Tool (PC-PART) using Rasch analysis.

    PubMed

    Darzins, Susan; Imms, Christine; Di Stefano, Marilyn; Taylor, Nicholas F; Pallant, Julie F

    2014-11-05

    The Personal Care Participation Assessment and Resource Tool (PC-PART) is a 43-item, clinician-administered assessment, designed to identify patients' unmet needs (participation restrictions) in activities of daily living (ADL) required for community life. This information is important for identifying problems that need addressing to enable, for example, discharge from inpatient settings to community living. The objective of this study was to evaluate internal construct validity of the PC-PART using Rasch methods. Fit to the Rasch model was evaluated for 41 PC-PART items, assessing threshold ordering, overall model fit, individual item fit, person fit, internal consistency, Differential Item Functioning (DIF), targeting of items and dimensionality. Data used in this research were taken from admission data from a randomised controlled trial conducted at two publically funded inpatient rehabilitation units in Melbourne, Australia, with 996 participants (63% women; mean age 74 years) and with various impairment types. PC-PART items assessed as one scale, and original PC-PART domains evaluated as separate scales, demonstrated poor fit to the Rasch model. Adequate fit to the Rasch model was achieved in two newly formed PC-PART scales: Self-Care (16 items) and Domestic Life (14 items). Both scales were unidimensional, had acceptable internal consistency (PSI =0.85, 0.76, respectively) and well-targeted items. Rasch analysis did not support conventional summation of all PC-PART item scores to create a total score. However, internal construct validity of the newly formed PC-PART scales, Self-Care and Domestic Life, was supported. Their Rasch-derived scores provided interval-level measurement enabling summation of scores to form a total score on each scale. These scales may assist clinicians, managers and researchers in rehabilitation settings to assess and measure changes in ADL participation restrictions relevant to community living. Data used in this research were

  9. Inter-agency communication and operations capabilities during a hospital functional exercise: reliability and validity of a measurement tool.

    PubMed

    Savoia, Elena; Biddinger, Paul D; Burstein, Jon; Stoto, Michael A

    2010-01-01

    As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise. The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement. Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self reported scores and external evaluators' scores ranged from moderate to good. The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.

  10. A 10-minute measure of global cognition: Validation of the Brief Cognitive Assessment Tool for Schizophrenia (B-CATS).

    PubMed

    Hurford, Irene M; Ventura, Joseph; Marder, Stephen R; Reise, Steven P; Bilder, Robert M

    2018-05-01

    Schizophrenia is marked by a global cognitive impairment that contributes significantly to chronic disability and unemployment. As new treatments are developed for cognition in schizophrenia, clinicians require easily administered instruments to assess cognition. We previously developed a very brief cognitive battery (Bell et al., 2005). The Brief Cognitive Assessment Tool for Schizophrenia (B-CATS) was developed specifically to provide clinicians with a way to assess cognition in their patients with schizophrenia. Here, we report the results of a validity study comparing B-CATS to a larger neurocognitive battery, the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) battery. Outpatients with schizophrenia (N=91) were administered the B-CATS and the non-overlapping tests of the MATRICS battery at two time points separated by 1month. They were also administered the UCSD Performance-based Skills Assessment-Brief (UPSA-B), a measure of functional capacity. The B-CATS has an administration time of approximately 10min. It demonstrates good test-retest reliability and internal consistency. It correlates 0.76 (p<0.01) with the MATRICS battery. The shorter B-CATS and the MATRICS battery correlate with the UPSA-B at 0.50 and 0.58 respectively. A 10-minute version of the B-CATS correlates highly with the "gold standard" neurocognitive battery that has an administration time of over 60min. Both measures correlate moderately with a measure of functional capacity. This brief battery was designed to allow clinicians to monitor cognitive change and better inform treatment decisions. Copyright © 2017 Elsevier B.V. All rights reserved.

  11. Commercially available gaming systems as clinical assessment tools to improve value in the orthopaedic setting: a systematic review.

    PubMed

    Ruff, Jessica; Wang, Tiffany L; Quatman-Yates, Catherine C; Phieffer, Laura S; Quatman, Carmen E

    2015-02-01

    Commercially available gaming systems (CAGS) such as the Wii Balance Board (WBB) and Microsoft Xbox with Kinect (Xbox Kinect) are increasingly used as balance training and rehabilitation tools. The purpose of this review was to answer the question, "Are commercially available gaming systems valid and reliable instruments for use as clinical diagnostic and functional assessment tools in orthopaedic settings?" and provide a summary of relevant studies, identify their strengths and weaknesses, and generate conclusions regarding general validity/reliability of WBB and Xbox Kinect in orthopaedics. A systematic search was performed using MEDLINE (1996-2013) and Scopus (1996-2013). Inclusion criteria were minimum of 5 subjects, full manuscript provided in English or translated, and studies incorporating investigation of CAG measurement properties. Exclusion criteria included reviews, systematic reviews, summary/clinical commentaries, or case studies; conference proceedings/presentations; cadaveric studies; studies of non-reversible, non-orthopaedic-related musculoskeletal disease; non-human trials; and therapeutic studies not reporting comparative evaluation to already established functional assessment criteria. All studies meeting inclusion and exclusion criteria were appraised for quality by two independent reviewers. Evidence levels (I-V) were assigned to each study based on established methodological criteria. 3 Level II, 7 level III, and 1 Level IV studies met inclusion criteria and provided information related to the use of the WBB and Xbox Kinect as clinical assessment tools in the field of orthopaedics. Studies have used the WBB in a variety of clinical applications, including the measurement of center of pressure (COP), measurement of medial-to-lateral (M/L) or anterior-to-posterior (A/P) symmetry, assessment anatomic landmark positioning, and assessment of fall risk. However, no uniform protocols or outcomes were used to evaluate the quality of the WBB as a

  12. The ELPAT living organ donor Psychosocial Assessment Tool (EPAT): from 'what' to 'how' of psychosocial screening - a pilot study.

    PubMed

    Massey, Emma K; Timmerman, Lotte; Ismail, Sohal Y; Duerinckx, Nathalie; Lopes, Alice; Maple, Hannah; Mega, Inês; Papachristou, Christina; Dobbels, Fabienne

    2018-01-01

    Thorough psychosocial screening of donor candidates is required in order to minimize potential negative consequences and to strive for optimal safety within living donation programmes. We aimed to develop an evidence-based tool to standardize the psychosocial screening process. Key concepts of psychosocial screening were used to structure our tool: motivation and decision-making, personal resources, psychopathology, social resources, ethical and legal factors and information and risk processing. We (i) discussed how each item per concept could be measured, (ii) reviewed and rated available validated tools, (iii) where necessary developed new items, (iv) assessed content validity and (v) pilot-tested the new items. The resulting ELPAT living organ donor Psychosocial Assessment Tool (EPAT) consists of a selection of validated questionnaires (28 items in total), a semi-structured interview (43 questions) and a Red Flag Checklist. We outline optimal procedures and conditions for implementing this tool. The EPAT and user manual are available from the authors. Use of this tool will standardize the psychosocial screening procedure ensuring that no psychosocial issues are overlooked and ensure that comparable selection criteria are used and facilitate generation of comparable psychosocial data on living donor candidates. © 2017 Steunstichting ESOT.

  13. Psychometric properties of the Peer Proficiency Assessment (PEPA): a tool for evaluation of undergraduate peer counselors' motivational interviewing fidelity.

    PubMed

    Mastroleo, Nadine R; Mallett, Kimberly A; Turrisi, Rob; Ray, Anne E

    2009-09-01

    Despite the expanding use of undergraduate student peer counseling interventions aimed at reducing college student drinking, few programs evaluate peer counselors' competency to conduct these interventions. The present research describes the development and psychometric assessments of the Peer Proficiency Assessment (PEPA), a new tool for examining Motivational Interviewing adherence in undergraduate student peer delivered interventions. Twenty peer delivered sessions were evaluated by master and undergraduate student coders using a cross-validation design to examine peer based alcohol intervention sessions. Assessments revealed high inter-rater reliability between student and master coders and good correlations between previously established fidelity tools. Findings lend support for the use of the PEPA to examine peer counselor competency. The PEPA, training for use, inter-rater reliability information, construct and predictive validity, and tool usefulness are described.

  14. Validity and reliability of wii fit balance board for the assessment of balance of healthy young adults and the elderly.

    PubMed

    Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

    2013-10-01

    [Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86-0.99) for the elderly people and positive correlations (r = 0.58-0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability.

  15. Validity and Reliability of Wii Fit Balance Board for the Assessment of Balance of Healthy Young Adults and the Elderly

    PubMed Central

    Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

    2013-01-01

    [Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86–0.99) for the elderly people and positive correlations (r = 0.58–0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability. PMID:24259769

  16. An evaluation tool for Myofascial Adhesions in Patients after Breast Cancer (MAP-BC evaluation tool): Concurrent, face and content validity.

    PubMed

    De Groef, An; Van Kampen, Marijke; Moortgat, Peter; Anthonissen, Mieke; Van den Kerckhove, Eric; Christiaens, Marie-Rose; Neven, Patrick; Geraerts, Inge; Devoogdt, Nele

    2018-01-01

    To investigate the concurrent, face and content validity of an evaluation tool for Myofascial Adhesions in Patients after Breast Cancer (MAP-BC evaluation tool). 1) Concurrent validity of the MAP-BC evaluation tool was investigated by exploring correlations (Spearman's rank Correlation Coefficient) between the subjective scores (0 -no adhesions to 3 -very strong adhesions) of the skin level using the MAP-BC evaluation tool and objective elasticity parameters (maximal skin extension and gross elasticity) generated by the Cutometer Dual MPA 580. Nine different examination points on and around the mastectomy scar were evaluated. 2) Face and content validity were explored by questioning therapists experienced with myofascial therapy in breast cancer patients about the comprehensibility and comprehensiveness of the MAP-BC evaluation tool. 1) Only three meaningful correlations were found on the mastectomy scar. For the most lateral examination point on the mastectomy scar a moderate negative correlation (-0.44, p = 0.01) with the maximal skin extension and a moderate positive correlation with the resistance versus ability of returning or 'gross elasticity' (0.42, p = 0.02) were found. For the middle point on the mastectomy scar an almost moderate positive correlation with gross elasticity was found as well (0.38, p = 0.04) 2) Content and face validity have been found to be good. Eighty-nine percent of the respondent found the instructions understandable and 98% found the scoring system obvious. Thirty-seven percent of the therapists suggested to add the possibility to evaluate additional anatomical locations in case of reconstructive and/or bilateral surgery. The MAP-BC evaluation tool for myofascial adhesions in breast cancer patients has good face and content validity. Evidence for good concurrent validity of the skin level was found only on the mastectomy scar itself.

  17. Data Center IT Equipment Energy Assessment Tools: Current State of Commercial Tools, Proposal for a Future Set of Assessment Tools

    DOE Office of Scientific and Technical Information (OSTI.GOV)

    Radhakrishnan, Ben D.

    2012-06-30

    This research project, which was conducted during the Summer and Fall of 2011, investigated some commercially available assessment tools with a focus on IT equipment to see if such tools could round out the DC Pro tool suite. In this research, the assessment capabilities of the various tools were compiled to help make “non-biased” information available to the public. This research should not be considered to be exhaustive on all existing vendor tools although a number of vendors were contacted. Large IT equipment OEM’s like IBM and Dell provide their proprietary internal automated software which does not work on anymore » other IT equipment. However, found two companies with products that showed promise in performing automated assessments for IT equipment from different OEM vendors. This report documents the research and provides a list of software products reviewed, contacts and websites, product details, discussions with specific companies, a set of recommendations, and next steps. As a result of this research, a simple 3-level approach to an IT assessment tool is proposed along with an example of an assessment using a simple IT equipment data collection tool (Level 1, spreadsheet). The tool has been reviewed with the Green Grid and LBNL staff. The initial feedback has been positive although further refinement to the tool will be necessary. Proposed next steps include a field trial of at least two vendors’ software in two different data centers with an objective to prove the concept, ascertain the extent of energy and computational assessment, ease of installation and opportunities for continuous improvement. Based on the discussions, field trials (or case studies) are proposed with two vendors – JouleX (expected to be completed in 2012) and Sentilla.« less

  18. Perioperative Respiratory Adverse Events in Pediatric Ambulatory Anesthesia: Development and Validation of a Risk Prediction Tool.

    PubMed

    Subramanyam, Rajeev; Yeramaneni, Samrat; Hossain, Mohamed Monir; Anneken, Amy M; Varughese, Anna M

    2016-05-01

    Perioperative respiratory adverse events (PRAEs) are the most common cause of serious adverse events in children receiving anesthesia. Our primary aim of this study was to develop and validate a risk prediction tool for the occurrence of PRAE from the onset of anesthesia induction until discharge from the postanesthesia care unit in children younger than 18 years undergoing elective ambulatory anesthesia for surgery and radiology. The incidence of PRAE was studied. We analyzed data from 19,059 patients from our department's quality improvement database. The predictor variables were age, sex, ASA physical status, morbid obesity, preexisting pulmonary disorder, preexisting neurologic disorder, and location of ambulatory anesthesia (surgery or radiology). Composite PRAE was defined as the presence of any 1 of the following events: intraoperative bronchospasm, intraoperative laryngospasm, postoperative apnea, postoperative laryngospasm, postoperative bronchospasm, or postoperative prolonged oxygen requirement. Development and validation of the risk prediction tool for PRAE were performed using a split sampling technique to split the database into 2 independent cohorts based on the year when the patient received ambulatory anesthesia for surgery and radiology using logistic regression. A risk score was developed based on the regression coefficients from the validation tool. The performance of the risk prediction tool was assessed by using tests of discrimination and calibration. The overall incidence of composite PRAE was 2.8%. The derivation cohort included 8904 patients, and the validation cohort included 10,155 patients. The risk of PRAE was 3.9% in the development cohort and 1.8% in the validation cohort. Age ≤ 3 years (versus >3 years), ASA physical status II or III (versus ASA physical status I), morbid obesity, preexisting pulmonary disorder, and surgery (versus radiology) significantly predicted the occurrence of PRAE in a multivariable logistic regression

  19. The Youth Throwing Score: Validating Injury Assessment in Young Baseball Players.

    PubMed

    Ahmad, Christopher S; Padaki, Ajay S; Noticewala, Manish S; Makhni, Eric C; Popkin, Charles A

    2017-02-01

    Epidemic levels of shoulder and elbow injuries have been reported recently in youth and adolescent baseball players. Despite the concerning frequency of these injuries, no instrument has been validated to assess upper extremity injury in this patient population. Purpose/Hypothesis: The purpose of this study was to validate an upper extremity assessment tool specifically designed for young baseball players. We hypothesized that this tool will be both reliable and valid. Cohort study (diagnosis); Level of evidence, 2. The Youth Throwing Score (YTS) was constructed by an interdisciplinary team of providers and coaches as a tool to assess upper extremity injury in youth and adolescent baseball players (age range, 10-18 years). The psychometric properties of the test were then determined. A total of 223 players completed the final survey. The players' mean age was 14.3 ± 2.7 years. Pilot analysis showed that none of the 14 questions received a mean athlete importance rating less than 3 of 5, and the final survey read at a Flesch-Kincaid level of 4.1, which is appropriate for patients aged 9 years and older. The players self-assigned their injury status, resulting in a mean instrument score of 59.7 ± 8.4 for the 148 players "playing without pain," 42.0 ± 11.5 for the 60 players "playing with pain," and 40.4 ± 10.5 for the 15 players "not playing due to pain." Players playing without pain scored significantly higher than those playing with pain and those not playing due to pain ( P < .001). Psychometric analysis showed a test-retest intraclass correlation coefficient of 0.90 and a Cronbach alpha intra-item reliability coefficient of 0.93, indicating excellent reliability and internal consistency. Pearson correlation coefficients of 0.65, 0.62, and 0.31 were calculated between the YTS and the Pediatric Outcomes Data Collection Instrument sports/physical functioning module, the Kerlan-Jobe Orthopaedic Clinic Shoulder and Elbow score, and the Quick Disabilities of the

  20. [Translation and validation of the Spanish version of the EAT-10 (Eating Assessment Tool-10) for the screening of dysphagia].

    PubMed

    Burgos, R; Sarto, B; Segurola, H; Romagosa, A; Puiggrós, C; Vázquez, C; Cárdenas, G; Barcons, N; Araujo, K; Pérez-Portabella, C

    2012-01-01

    The Eating Assessment Tool-10 (EAT-10) is a self-administered, analogical, direct-scoring screening tool for dysphagia. To translate and adapt the EAT-10 into Spanish, and to evaluate its psychometric properties. After the translation and back-translation process of the EAT-10 ES, a prospective study was performed in adult patients with preserved cognitive and functional abilities. Patients in 3 clinical situations, diagnosed with dysphagia (DD), patients at risk of dysphagia (RD), and patients not at risk of dysphagia (SRD) were recruited from 3 settings: a hospital Nutritional Support Unit (USN), a nursing home (RG) and primary care centre (CAP). Patients completed the EAT-10 ES during a single visit. Both patients and researchers completed a specific questionnaire regarding EAT-10 ES' comprehension. 65 patients were included (age 75 ± 9.1 y), 52.3% women. Mean time of administration was 3.8 ± 1.7 minutes. 95.4% of patients considered that all tool items were comprehensible and 72.3% found it easy to assign scores. EAT-10 ES' internal consistency, Cronbach's Alpha coefficient was 0.87. A high correlation was observed between all tool items and global scores (p < 0.001). Mean score for patients in group DD was 15 ± 8.9 points, 6.7 ± 7.7 points in group RD, and 2 ± 3.1 points in group SRD. Male patients, previously diagnosed of dysphagia or patients from the NSU showed significantly higher scores on the EAT-10 ES (p < 0.001). EAT-10 ES has proven to be reliable, valid and to have internal consistency. Is it an easy-to-understand tool that can be completed quickly, making it useful for the screening of dysphagia in routine clinical practice.

  1. Additional Support for the Information Systems Analyst Exam as a Valid Program Assessment Tool

    ERIC Educational Resources Information Center

    Carpenter, Donald A.; Snyder, Johnny; Slauson, Gayla Jo; Bridge, Morgan K.

    2011-01-01

    This paper presents a statistical analysis to support the notion that the Information Systems Analyst (ISA) exam can be used as a program assessment tool in addition to measuring student performance. It compares ISA exam scores earned by students in one particular Computer Information Systems program with scores earned by the same students on the…

  2. The GRACE Checklist: A Validated Assessment Tool for High Quality Observational Studies of Comparative Effectiveness.

    PubMed

    Dreyer, Nancy A; Bryant, Allison; Velentgas, Priscilla

    2016-10-01

    a predictor of quality in all 4 trees. When a composite outcome of the 3 quality measures was used, the GRACE Checklist showed high sensitivity and specificity (71.43% and 80.95%, respectively). The GRACE Checklist stands out from other consensus-driven and expert guidance documents because of its extensive validation efforts. This most recent work shows that the checklist has strong sensitivity and specificity, increasing its utility as a screening tool to identify high-quality observational comparative effectiveness research worthy of in-depth review and applicability for decision support. No outside funding supported this research. All authors are full-time employees of Quintiles, which provides research and consulting services to the biopharmaceutical industry. The authors have no other disclosures to report. Two of the 3 CART trees were presented at the International Society of Pharmacepidemiology in 2015 ("Article Citations per Year" and "Journal Impact Factor"). The original validation study was published in the March 2014 issue of the Journal of Managed Care & Specialty Pharmacy. The checklist questions and scoring were included using a table that was originally published by this journal in 2014. Study concept and design were primarily contributed by Dreyer and Velentgas, along with Bryant. Bryant took the lead in data collection and analysis, along with Dreyer and Velentgas, and data interpretation was performed by Dreyer, Velentgas, and Bryant. The manuscript was written and revised primarily by Dreyer, along with Bryant and Velentgas.

  3. A tool to assess sex-gender when selecting health research projects.

    PubMed

    Tomás, Concepción; Yago, Teresa; Eguiluz, Mercedes; Samitier, M A Luisa; Oliveros, Teresa; Palacios, Gemma

    2015-04-01

    To validate the questionnaire "Gender Perspective in Health Research" (GPIHR) to assess the inclusion of gender perspective in research projects. Validation study in two stages. Feasibility was analysed in the first, and reliability, internal consistence and validity in the second. Aragón Institute of Health Science, Aragón, Spain. GPIHR was applied to 118 research projects funded in national and international competitive tenders from 2003 to 2012. Analysis of inter- and intra-observer reliability with Kappa index and internal consistency with Cronbach's alpha. Content validity analysed through literature review and construct validity with an exploratory factor analysis. Validated GPIHR has 10 questions: 3 in the introduction, 1 for objectives, 3 for methodology and 3 for research purpose. Average time of application was 13min Inter-observer reliability (Kappa) varied between 0.35 and 0.94 and intra-observer between 0.40 and 0.94. Theoretical construct is supported in the literature. Factor analysis identifies three levels of GP inclusion: "difference by sex", "gender sensitive" and "feminist research" with an internal consistency of 0.64, 0.87 and 0.81, respectively, which explain 74.78% of variance. GPIHR questionnaire is a valid tool to assess GP and useful for those researchers who would like to include GP in their projects. Copyright © 2014 Elsevier España, S.L.U. All rights reserved.

  4. Exploring Assessment Tools for Research and Evaluation in Astronomy Education and Outreach

    NASA Astrophysics Data System (ADS)

    Buxner, S. R.; Wenger, M. C.; Dokter, E. F. C.

    2011-09-01

    The ability to effectively measure knowledge, attitudes, and skills in formal and informal educational settings is an important aspect of astronomy education research and evaluation. Assessments may take the form of interviews, observations, surveys, exams, or other probes to help unpack people's understandings or beliefs. In this workshop, we discussed characteristics of a variety of tools that exist to assess understandings of different concepts in astronomy as well as attitudes towards science and science teaching; these include concept inventories, surveys, interview protocols, observation protocols, card sorting, reflection videos, and other methods currently being used in astronomy education research and EPO program evaluations. In addition, we discussed common questions in the selection of assessment tools including issues of reliability and validity, time to administer, format of implementation, analysis, and human subject concerns.

  5. Validity of assessing child feeding with virtual reality.

    PubMed

    Persky, Susan; Goldring, Megan R; Turner, Sara A; Cohen, Rachel W; Kistler, William D

    2018-04-01

    Assessment of parents' child feeding behavior is challenging, and there is need for additional methodological approaches. Virtual reality technology allows for the creation of behavioral measures, and its implementation overcomes several limitations of existing methods. This report evaluates the validity and usability of the Virtual Reality (VR) Buffet among a sample of 52 parents of children aged 3-7. Participants served a meal of pasta and apple juice in both a virtual setting and real-world setting (counterbalanced and separated by a distractor task). They then created another meal for their child, this time choosing from the full set of food options in the VR Buffet. Finally, participants completed a food estimation task followed by a questionnaire, which assessed their perceptions of the VR Buffet. Results revealed that the amount of virtual pasta served by parents correlated significantly with the amount of real pasta they served, r s  = 0.613, p < .0001, as did served amounts of virtual and real apple juice, r s  = 0.822, p < .0001. Furthermore, parents' perception of the calorie content of chosen foods was significantly correlated with observed calorie content (r s  = 0.438, p = .002), and parents agreed that they would feed the meal they created to their child (M = 4.43, SD = 0.82 on a 1-5 scale). The data presented here demonstrate that parent behavior in the VR Buffet is highly related to real-world behavior, and that the tool is well-rated by parents. Given the data presented and the potential benefits of the abundant behavioral data the VR Buffet can provide, we conclude that it is a valid and needed addition to the array of tools for assessing feeding behavior. Published by Elsevier Ltd.

  6. Knowing Every Child: Validation of the Holistic Student Assessment (HSA) as a Measure of Social-Emotional Development.

    PubMed

    Malti, Tina; Zuffianò, Antonio; Noam, Gil G

    2018-04-01

    Knowing every child's social-emotional development is important as it can support prevention and intervention approaches to meet the developmental needs and strengths of children. Here, we discuss the role of social-emotional assessment tools in planning, implementing, and evaluating preventative strategies to promote mental health in all children and adolescents. We, first, selectively review existing tools and identify current gaps in the measurement literature. Next, we introduce the Holistic Student Assessment (HSA), a tool that is based in our social-emotional developmental theory, The Clover Model, and designed to measure social-emotional development in children and adolescents. Using a sample of 5946 students (51% boys, M age  = 13.16 years), we provide evidence for the psychometric validity of the self-report version of the HSA. First, we document the theoretically expected 7-dimension factor structure in a calibration sub-sample (n = 984) and cross-validate its structure in a validation sub-sample (n = 4962). Next, we show measurement invariance across development, i.e., late childhood (9- to 11-year-olds), early adolescence (12- to 14-year-olds), and middle adolescence (15- to 18-year-olds), and evidence for the HSA's construct validity in each age group. The findings support the robustness of the factor structure and confirm its developmental sensitivity. Structural equation modeling validity analysis in a multiple-group framework indicates that the HSA is associated with mental health in expected directions across ages. Overall, these findings show the psychometric properties of the tool, and we discuss how social-emotional tools such as the HSA can guide future research and inform large-scale dissemination of preventive strategies.

  7. ExEP yield modeling tool and validation test results

    NASA Astrophysics Data System (ADS)

    Morgan, Rhonda; Turmon, Michael; Delacroix, Christian; Savransky, Dmitry; Garrett, Daniel; Lowrance, Patrick; Liu, Xiang Cate; Nunez, Paul

    2017-09-01

    EXOSIMS is an open-source simulation tool for parametric modeling of the detection yield and characterization of exoplanets. EXOSIMS has been adopted by the Exoplanet Exploration Programs Standards Definition and Evaluation Team (ExSDET) as a common mechanism for comparison of exoplanet mission concept studies. To ensure trustworthiness of the tool, we developed a validation test plan that leverages the Python-language unit-test framework, utilizes integration tests for selected module interactions, and performs end-to-end crossvalidation with other yield tools. This paper presents the test methods and results, with the physics-based tests such as photometry and integration time calculation treated in detail and the functional tests treated summarily. The test case utilized a 4m unobscured telescope with an idealized coronagraph and an exoplanet population from the IPAC radial velocity (RV) exoplanet catalog. The known RV planets were set at quadrature to allow deterministic validation of the calculation of physical parameters, such as working angle, photon counts and integration time. The observing keepout region was tested by generating plots and movies of the targets and the keepout zone over a year. Although the keepout integration test required the interpretation of a user, the test revealed problems in the L2 halo orbit and the parameterization of keepout applied to some solar system bodies, which the development team was able to address. The validation testing of EXOSIMS was performed iteratively with the developers of EXOSIMS and resulted in a more robust, stable, and trustworthy tool that the exoplanet community can use to simulate exoplanet direct-detection missions from probe class, to WFIRST, up to large mission concepts such as HabEx and LUVOIR.

  8. Reliability and validity of procedure-based assessments in otolaryngology training.

    PubMed

    Awad, Zaid; Hayden, Lindsay; Robson, Andrew K; Muthuswamy, Keerthini; Tolley, Neil S

    2015-06-01

    To investigate the reliability and construct validity of procedure-based assessment (PBA) in assessing performance and progress in otolaryngology training. Retrospective database analysis using a national electronic database. We analyzed PBAs of otolaryngology trainees in North London from core trainees (CTs) to specialty trainees (STs). The tool contains six multi-item domains: consent, planning, preparation, exposure/closure, technique, and postoperative care, rated as "satisfactory" or "development required," in addition to an overall performance rating (pS) of 1 to 4. Individual domain score, overall calculated score (cS), and number of "development-required" items were calculated for each PBA. Receiver operating characteristic analysis helped determine sensitivity and specificity. There were 3,152 otolaryngology PBAs from 46 otolaryngology trainees analyzed. PBA reliability was high (Cronbach's α 0.899), and sensitivity approached 99%. cS correlated positively with pS and level in training (rs : +0.681 and +0.324, respectively). ST had higher cS and pS than CT (93% ± 0.6 and 3.2 ± 0.03 vs. 71% ± 3.1 and 2.3 ± 0.08, respectively; P < .001). cS and pS increased from CT1 to ST8 showing construct validity (rs : +0.348 and +0.354, respectively; P < .001). The technical skill domain had the highest utilization (98% of PBAs) and was the best predictor of cS and pS (rs : +0.96 and +0.66, respectively). PBA is reliable and valid for assessing otolaryngology trainees' performance and progress at all levels. It is highly sensitive in identifying competent trainees. The tool is used in a formative and feedback capacity. The technical domain is the best predictor and should be given close attention. NA. © 2014 The American Laryngological, Rhinological and Otological Society, Inc.

  9. Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

    PubMed

    Hobbelen, Johannes S M; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A

    2008-08-01

    Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability and validity of the PAI. A three-phase cross-sectional survey was conducted. In the first two phases, the PAI was developed and validated. In the third phase, the inter-observer reliability and feasibility of the instrument was tested. The original PAI consisted of five criteria that all needed to be met in order to make the diagnosis. On the basis of a qualitative analysis, one criterion was reformulated and another was removed. Following this, inter-observer reliability between the two assessors resulted in an improvement of Cohen's kappa from 0.532 in the initial phase to 0.677 in the second phase. This improvement was substantiated in the third phase by two independent assessors with Cohen's kappa ranging from 0.625 to 1. The PAI is a reliable and valid assessment tool for diagnosing paratonia in elderly people with dementia that can be applied easily in daily practice.

  10. Clinical instruments: reliability and validity critical appraisal.

    PubMed

    Brink, Yolandi; Louw, Quinette A

    2012-12-01

    RATIONALE, AIM AND OBJECTIVES: There is a lack of health care practitioners using objective clinical tools with sound psychometric properties. There is also a need for researchers to improve their reporting of the validity and reliability results of these clinical tools. Therefore, to promote the use of valid and reliable tools or tests for clinical evaluation, this paper reports on the development of a critical appraisal tool to assess the psychometric properties of objective clinical tools. A five-step process was followed to develop the new critical appraisal tool: (1) preliminary conceptual decisions; (2) defining key concepts; (3) item generation; (4) assessment of face validity; and (5) formulation of the final tool. The new critical appraisal tool consists of 13 items, of which five items relate to both validity and reliability studies, four items to validity studies only and four items to reliability studies. The 13 items could be scored as 'yes', 'no' or 'not applicable'. This critical appraisal tool will aid both the health care practitioner to critically appraise the relevant literature and researchers to improve the quality of reporting of the validity and reliability of objective clinical tools. © 2011 Blackwell Publishing Ltd.

  11. Pilot Use of a Novel Tool to Assess Neurosurgical Capacity in Uganda.

    PubMed

    Ploss, Brittany; Abdelgadir, Jihad; Smith, Emily R; Fuller, Anthony; Nickenig Vissoci, Joao Ricardo; Muhindo, Alex; Galukande, Moses; Haglund, Michael M

    2017-12-01

    There is a significant burden of unmet surgical need in many low- and middle-income countries (>80% in parts of Africa). This need is even larger for specialties such as neurosurgery. Surgical capacity tools have been developed and used to assess needs and plan for resource allocation. This study piloted a new tool to assess neurosurgical capacity and describes its use. A surgical capacity tool was adapted to assess neurosurgical capacity. An expert panel of neurosurgeons and researchers reviewed the Surgeons OverSeas PIPES (personnel, infrastructure, procedures, equipment, and supplies) assessment and added additional items essential to perform common neurosurgery procedures. This tool was then piloted at 3 public hospitals in Uganda and each hospital was given a score of neurosurgical capacity. At 1 hospital, 3 respondents were asked to answer the survey to assess reliability. The hospital with the largest neurosurgery caseload and 5 neurosurgeons scored the highest on our survey, followed by a regional hospital with 1 practicing neurosurgeon. The third hospital, without a neurosurgeon, scored the lowest on the scale. At the hospital that completed the reliability assessment, scores were varied between respondents. NeuroPIPES survey scores were in keeping with the number of neurosurgeons and respective caseloads of each hospital. However, the variation in scores between respondents at the same hospital suggests that adaptations could be made to the tool that may improve reliability and validity. The methodology used to create NeuroPIPES may be successfully applied to a variety of other surgical subspecialties for similar assessments. Copyright © 2017 Elsevier Inc. All rights reserved.

  12. Validation of a Tool Evaluating Educational Apps for Smart Education

    ERIC Educational Resources Information Center

    Lee, Jeong-Sook; Kim, Sung-Wan

    2015-01-01

    The purpose of this study is to develop and validate an evaluation tool of educational apps for smart education. Based on literature reviews, a potential model for evaluating educational apps was suggested. An evaluation tool consisting of 57 survey items was delivered to 156 students in middle and high schools. An exploratory factor analysis was…

  13. Self-Report Dietary Assessment Tools Used in Canadian Research: A Scoping Review123

    PubMed Central

    Vanderlee, Lana; Raffoul, Amanda; Stapleton, Jackie; Csizmadi, Ilona; Boucher, Beatrice A; Massarelli, Isabelle; Rondeau, Isabelle

    2017-01-01

    Choosing the most appropriate dietary assessment tool for a study can be a challenge. Through a scoping review, we characterized self-report tools used to assess diet in Canada to identify patterns in tool use and to inform strategies to strengthen nutrition research. The research databases Medline, PubMed, PsycINFO, and CINAHL were used to identify Canadian studies published from 2009 to 2014 that included a self-report assessment of dietary intake. The search elicited 2358 records that were screened to identify those that reported on self-report dietary intake among nonclinical, non-Aboriginal adult populations. A pool of 189 articles (reflecting 92 studies) was examined in-depth to assess the dietary assessment tools used. Food-frequency questionnaires (FFQs) and screeners were used in 64% of studies, whereas food records and 24-h recalls were used in 18% and 14% of studies, respectively. Three studies (3%) used a single question to assess diet, and for 3 studies the tool used was not clear. A variety of distinct FFQs and screeners, including those developed and/or adapted for use in Canada and those developed elsewhere, were used. Some tools were reported to have been evaluated previously in terms of validity or reliability, but details of psychometric testing were often lacking. Energy and fat were the most commonly studied, reported by 42% and 39% of studies, respectively. For ∼20% of studies, dietary data were used to assess dietary quality or patterns, whereas close to half assessed ≤5 dietary components. A variety of dietary assessment tools are used in Canadian research. Strategies to improve the application of current evidence on best practices in dietary assessment have the potential to support a stronger and more cohesive literature on diet and health. Such strategies could benefit from national and global collaboration. PMID:28298272

  14. Frameworks and tools for risk assessment of manufactured nanomaterials.

    PubMed

    Hristozov, Danail; Gottardo, Stefania; Semenzin, Elena; Oomen, Agnes; Bos, Peter; Peijnenburg, Willie; van Tongeren, Martie; Nowack, Bernd; Hunt, Neil; Brunelli, Andrea; Scott-Fordsmand, Janeck J; Tran, Lang; Marcomini, Antonio

    2016-10-01

    Commercialization of nanotechnologies entails a regulatory requirement for understanding their environmental, health and safety (EHS) risks. Today we face challenges to assess these risks, which emerge from uncertainties around the interactions of manufactured nanomaterials (MNs) with humans and the environment. In order to reduce these uncertainties, it is necessary to generate sound scientific data on hazard and exposure by means of relevant frameworks and tools. The development of such approaches to facilitate the risk assessment (RA) of MNs has become a dynamic area of research. The aim of this paper was to review and critically analyse these approaches against a set of relevant criteria. The analysis concluded that none of the reviewed frameworks were able to fulfill all evaluation criteria. Many of the existing modelling tools are designed to provide screening-level assessments rather than to support regulatory RA and risk management. Nevertheless, there is a tendency towards developing more quantitative, higher-tier models, capable of incorporating uncertainty into their analyses. There is also a trend towards developing validated experimental protocols for material identification and hazard testing, reproducible across laboratories. These tools could enable a shift from a costly case-by-case RA of MNs towards a targeted, flexible and efficient process, based on grouping and read-across strategies and compliant with the 3R (Replacement, Reduction, Refinement) principles. In order to facilitate this process, it is important to transform the current efforts on developing databases and computational models into creating an integrated data and tools infrastructure to support the risk assessment and management of MNs. Copyright © 2016 Elsevier Ltd. All rights reserved.

  15. Validation of the Spanish Version of the Eating Assessment Tool-10 (EAT-10spa) in Colombia. A Blinded Prospective Cohort Study.

    PubMed

    Giraldo-Cadavid, Luis Fernando; Gutiérrez-Achury, Alejandra María; Ruales-Suárez, Karem; Rengifo-Varona, Maria Leonor; Barros, Camilo; Posada, Andrés; Romero, Carlos; Galvis, Ana María

    2016-06-01

    Dysphagia might affect 12 % of the general population, and its complications include pneumonia, malnutrition, social isolation, and death. No validated Spanish symptom survey exists to quantify dysphagia symptoms among Latin Americans. Therefore, we performed a prospective cohort study in a tertiary care university hospital to validate the Spanish version of the 10-Item Eating-Assessment-Tool (EAT-10spa) for use in Colombia. After an interdisciplinary committee of five bilingual specialists evaluated the EAT-10spa (translated and validated in Spain) and deemed it appropriate for the Colombian culture, its feasibility, reliability, validity, sensitivity to change, and diagnostic capacity were evaluated. As a reference standard, we used the flexible endoscopic evaluation of swallowing with sensory testing and a standardized clinical evaluation. All assessments were blinded. In total, 133 subjects were included (52 % women, mean age 55 years) and completed the EAT-10spa (median completion time: 2 min [IQR 1-3 min]), 39 % of whom had an elementary-level education. Cronbach's α coefficient: 0.91; test-retest intra-class correlation coefficient: 0.94. The Spearman's correlation coefficient of the EAT-10spa with the 8-point penetration-aspiration scale was 0.54 (P < 0.001). The area under the receiver-operating-characteristic-curve (AUC-ROC) for dysphagia and aspiration were 0.79 (P < 0.001) and 0.81 (P < 0.001), respectively. The best cut-off points for dysphagia and aspiration were EAT-10spa ≥2 (sensitivity 93.6 %, specificity 36.4 %) and EAT-10spa ≥4 (sensitivity 94.3 %, specificity 49.5 %), respectively. A reduction in the EAT-10 ≥3 was the best cut-off point for a clinically significant improvement (AUC-ROC 0.83; P < 0.0001). The EAT-10spa showed excellent psychometric properties and discriminatory capacity for use in Colombia.

  16. Vegetable behavioral tool demonstrates validity with MyPlate vegetable cups and carotenoid and inflammatory biomarkers.

    PubMed

    Townsend, Marilyn S; Shilts, Mical K; Styne, Dennis M; Drake, Christiana; Lanoue, Louise; Woodhouse, Leslie; Allen, Lindsay H

    2016-12-01

    Young children are not meeting recommendations for vegetable intake. Our objective is to provide evidence of validity and reliability for a pictorial vegetable behavioral assessment for use by federally funded community nutrition programs. Parent/child pairs (n=133) from Head Start and the Special Supplemental Nutrition Program for Women, Infants and Children [WIC] provided parent-administered vegetable tools, three child 24-hour diet recalls, child blood sample and measured heights/weights. The 10-item Focus on Veggies scale, with an alpha of .83 and a stability reliability coefficient of .74, was positively related to vegetables in cup equivalents [p≤.05]; dietary intakes of folate, vitamin C, β-carotene, potassium and magnesium [p≤.05-.01]; and soluble fiber [p≤.001]. The child vegetable scores were related to the parent's mediators [p≤.00001] and vegetable behaviors [p≤.00001]. Children's plasma inflammatory markers were negatively related to the 10 item scale [p≤.05] and are indicators of the child's health status. The positive relationship between the serum carotenoid index and a sub-scale of child vegetable behaviors offered additional support for criterion validity [p≤.05]. Finally, the inverse relationship of BMI-for-age percentile one year post baseline and a sub-scale of child vegetable behaviors supported the predictive validity [p≤.05]. Focus on Veggies, a simple assessment tool, can inform practitioners about the child's health status. A child with a high score, shows a healthful profile with a lower inflammation index, higher carotenoid index, lower BMI and higher vegetable intake. In conclusion, validity of Focus on Veggies has been demonstrated using vegetable cup equivalents and micronutrient intakes, anthropometry and blood biomarkers. Copyright © 2016 Elsevier Ltd. All rights reserved.

  17. Development of risk assessment tool for foundry workers.

    PubMed

    Mohan, G Madhan; Prasad, P S S; Mokkapati, Anil Kumar; Venkataraman, G

    2008-01-01

    Occupational ill-health and work-related disorders are predominant in manufacturing industries due to the inevitable presence of manual work even after several waves of industrial automation and technological advancements. Ergonomic risk factors and musculoskeletal disorders like low-back symptoms have been noted amongst foundry workers. The purpose of this study was to formulate and develop a Physical Effort Index to assess risk factor. The questionnaire tool applicable to foundry environment has been designed and validated. The data recorded through survey across the foundries has been subjected to regression analysis to correlate between proposed physical effort index and the standard Borg's Ratings of Perceived Exertion (RPE) scale. The physical efforts of sixty seven workers in various foundry shop floors were assessed subjectively. The 'Job factors' and 'Work environment' were the two major parameters considered in assessing the worker discomfort level at workplace. A relation between Borg's RPE scale and the above two parameters were arrived at, through regression analysis. The study demonstrates the prevalence of risk factors amongst foundry workers and the effectiveness of the proposed index in estimating the risk factor levels. RELEVANCE TO THE INDUSTRY: The proposed tool will assist foundry supervisors and managers to assess the risk factors and helps in better understanding of the workplace to avoid work-related disorders, ensuring better output.

  18. Social Validation of the New England Center for Children-Core Skills Assessment

    ERIC Educational Resources Information Center

    Dickson, Chata A.; MacDonald, Rebecca P. F.; Mansfield, Renee; Guilhardi, Paulo; Johnson, Cammarie; Ahearn, William H.

    2014-01-01

    We investigated the social validity of the NECC Core Skills Assessment (NECC-CSA) with parents and professionals as participants. The NECC-CSA is a measurement tool consisting of direct and indirect measures of skills important to all individuals with autism, across the lifespan. Participants (N = 245) were provided with a list of 66 skills, 47 of…

  19. The Feasibility of Standardised Geriatric Assessment Tools and Physical Exercises in Frail Older Adults.

    PubMed

    Jadczak, A D; Mahajan, N; Visvanathan, R

    2017-01-01

    Geriatric assessment tools are applicable to the general geriatric population; however, their feasibility in frail older adults is yet to be determined. The study aimed to determine the feasibility of standardised geriatric assessment tools and physical exercises in hospitalised frail older adults. Various assessment tools including the FRAIL Screen, the Charlson Comorbidity Index, the SF-36, the Trail Making Test (TMT), the Rapid Cognitive Screen, the Self Mini Nutritional Assessment (MNA-SF) and the Lawton iADL as well as standard physical exercises were assessed using observational protocols. The FRAIL Screen, MNA-SF, Rapid Cognitive Screen, Lawton iADL and the physical exercises were deemed to be feasible with only minor comprehension, execution and safety issues. The TMT was not considered to be feasible and the SF-36 should be replaced by its shorter form, the SF-12. In order to ensure the validity of these findings a study with a larger sample size should be undertaken.

  20. Development and Validation of Osteoporosis Risk-Assessment Model for Korean Men

    PubMed Central

    Oh, Sun Min; Song, Bo Mi; Nam, Byung-Ho; Rhee, Yumie; Moon, Seong-Hwan; Kim, Deog Young; Kang, Dae Ryong

    2016-01-01

    Purpose The aim of the present study was to develop an osteoporosis risk-assessment model to identify high-risk individuals among Korean men. Materials and Methods The study used data from 1340 and 1110 men ≥50 years who participated in the 2009 and 2010 Korean National Health and Nutrition Examination Survey, respectively, for development and validation of an osteoporosis risk-assessment model. Osteoporosis was defined as T score ≤-2.5 at either the femoral neck or lumbar spine. Performance of the candidate models and the Osteoporosis Self-assessment Tool for Asian (OSTA) was compared with sensitivity, specificity, and area under the receiver operating characteristics curve (AUC). A net reclassification improvement was further calculated to compare the developed Korean Osteoporosis Risk-Assessment Model for Men (KORAM-M) with OSTA. Results In the development dataset, the prevalence of osteoporosis was 8.1%. KORAM-M, consisting of age and body weight, had a sensitivity of 90.8%, a specificity of 42.4%, and an AUC of 0.666 with a cut-off score of -9. In the validation dataset, similar results were shown: sensitivity 87.9%, specificity 39.7%, and AUC 0.638. Additionally, risk categorization with KORAM-M showed improved reclassification over that of OSTA up to 22.8%. Conclusion KORAM-M can be simply used as a pre-screening tool to identify candidates for dual energy X-ray absorptiometry tests. PMID:26632400

  1. Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose.

    PubMed

    Loudon, Kirsty; Zwarenstein, Merrick; Sullivan, Frank; Donnan, Peter; Treweek, Shaun

    2013-04-27

    If you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention. The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters' subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of

  2. Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose

    PubMed Central

    2013-01-01

    Background If you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention. Methods The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters’ subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the

  3. Nursing Informatics Competencies: Psychometric Validation, Dissemination, and Maintenance of Self-Assessment Tool for Nurse Leaders.

    PubMed

    Collins, Sarah

    2016-01-01

    Due to rapid advances in technology, HIT competencies for nursing leaders require frequent attention and updating from experts in the field to ensure relevance to nursing leaders' work. This workshop will target nursing informatics researchers and leaders to: 1) learn methods and findings from a study validating a Self-Assessment Scale for Nursing Informatics Competencies for Nurse Leaders, 2) generate awareness of the Self-Assessment scale, 3) discuss strategies for maintenance of competencies overtime and 4) identify strategies to engage nursing leaders in this pursuit.

  4. The Learning Loss Scale as an Assessment Tool: An Empirical Examination of Convergent Validity with Performative Measures

    ERIC Educational Resources Information Center

    Hooker, John; Denker, Katherine

    2014-01-01

    Higher education has placed an increasingly greater value on assessment. The Learning Loss Scale may be an appropriate tool to assess learning across disciplines. In this paper, we review the culture of assessment, conceptualizations of cognitive learning, the Learning Loss Scale, and a theoretical explanation, and then we test this measure to…

  5. Reliability and validity of play-based assessments of motor and cognitive skills for infants and young children: a systematic review.

    PubMed

    O'Grady, Michael G; Dusing, Stacey C

    2015-01-01

    Play is vital for development. Infants and children learn through play. Traditional standardized developmental tests measure whether a child performs individual skills within controlled environments. Play-based assessments can measure skill performance during natural, child-driven play. The purpose of this study was to systematically review reliability, validity, and responsiveness of all play-based assessments that quantify motor and cognitive skills in children from birth to 36 months of age. Studies were identified from a literature search using PubMed, ERIC, CINAHL, and PsycINFO databases and the reference lists of included papers. Included studies investigated reliability, validity, or responsiveness of play-based assessments that measured motor and cognitive skills for children to 36 months of age. Two reviewers independently screened 40 studies for eligibility and inclusion. The reviewers independently extracted reliability, validity, and responsiveness data. They examined measurement properties and methodological quality of the included studies. Four current play-based assessment tools were identified in 8 included studies. Each play-based assessment tool measured motor and cognitive skills in a different way during play. Interrater reliability correlations ranged from .86 to .98 for motor development and from .23 to .90 for cognitive development. Test-retest reliability correlations ranged from .88 to .95 for motor development and from .45 to .91 for cognitive development. Structural validity correlations ranged from .62 to .90 for motor development and from .42 to .93 for cognitive development. One study assessed responsiveness to change in motor development. Most studies had small and poorly described samples. Lack of transparency in data management and statistical analysis was common. Play-based assessments have potential to be reliable and valid tools to assess cognitive and motor skills, but higher-quality research is needed. Psychometric properties

  6. Validation of balance-quality assessment using a modified bathroom scale.

    PubMed

    Hewson, D J; Duchêne, J; Hogrel, J-Y

    2015-02-01

    The balance quality tester (BQT), based on a standard electronic bathroom scale has been developed in order to assess balance quality. The BQT includes automatic detection of the person to be tested by means of an infrared detector and bluetooth communication capability for remote assessment when linked to a long-distance communication device such as a mobile phone. The BQT was compared to a standard force plate for validity and agreement. The two most widely reported parameters in balance literature, the area of the centre of pressure (COP) displacement and the velocity of the COP displacement, were compared for 12 subjects, each of whom was tested on ten occasions on each of the 2 days. No significant differences were observed between the BQT and the force plate for either of the two parameters. In addition a high level of agreement was observed between both devices. The BQT is a valid device for remote assessment of balance quality, and could provide a useful tool for long-term monitoring of people with balance problems, particularly during home monitoring.

  7. Validity of the Kinect for Gait Assessment: A Focused Review

    PubMed Central

    Springer, Shmuel; Yogev Seligmann, Galit

    2016-01-01

    Gait analysis may enhance clinical practice. However, its use is limited due to the need for expensive equipment which is not always available in clinical settings. Recent evidence suggests that Microsoft Kinect may provide a low cost gait analysis method. The purpose of this report is to critically evaluate the literature describing the concurrent validity of using the Kinect as a gait analysis instrument. An online search of PubMed, CINAHL, and ProQuest databases was performed. Included were studies in which walking was assessed with the Kinect and another gold standard device, and consisted of at least one numerical finding of spatiotemporal or kinematic measures. Our search identified 366 papers, from which 12 relevant studies were retrieved. The results demonstrate that the Kinect is valid only for some spatiotemporal gait parameters. Although the kinematic parameters measured by the Kinect followed the trend of the joint trajectories, they showed poor validity and large errors. In conclusion, the Kinect may have the potential to be used as a tool for measuring spatiotemporal aspects of gait, yet standardized methods should be established, and future examinations with both healthy subjects and clinical participants are required in order to integrate the Kinect as a clinical gait analysis tool. PMID:26861323

  8. Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study

    PubMed Central

    Cho, Hwayoung; Liu, Jianfang

    2018-01-01

    Background Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. Objective The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. Methods A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. Results The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. Conclusions The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. PMID:29305343

  9. Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study.

    PubMed

    Schnall, Rebecca; Cho, Hwayoung; Liu, Jianfang

    2018-01-05

    Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. ©Rebecca Schnall, Hwayoung Cho, Jianfang Liu. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 05.01.2018.

  10. Validation of a Dietary Screening Tool in a Middle-Aged Appalachian Population

    PubMed Central

    Ventura Marra, Melissa; Johnson, Elizabeth J.

    2018-01-01

    Proactive nutrition screening is an effective public health strategy for identifying and targeting individuals who could benefit from making dietary improvements for primary and secondary prevention of disease. The Dietary Screening Tool (DST) was developed and validated to assess nutritional risk among rural older adults. The purpose of this study was to evaluate the utility and validity of the DST to identify nutritional risk in middle-aged adults. This cross-sectional study in middle-aged adults (45–64 year olds, n = 87) who reside in Appalachia, examined nutritional status using an online health survey, biochemical measures, anthropometry, and three representative 24-h dietary recalls. The Healthy Eating Index (HEI) was calculated to describe overall diet quality. Adults identified by the DST with a nutrition risk had lower HEI scores (50 vs. 64, p < 0.001) and were much more likely to also be considered at dietary risk by the HEI (OR 11.6; 3.2–42.6) when compared to those not at risk. Those at risk had higher energy-adjusted total fat, saturated fat, and added sugar intakes and lower intakes of dietary fiber, and several micronutrients than those classified as not at risk by the DST. Similarly, the at-risk group had significantly lower serum levels of α-carotene, β-carotene, cryptoxanthin, lutein, and zeaxanthin but did not differ in retinol or methylmalonic acid compared with those not at risk. The DST is a valid tool to identify middle-aged adults with nutritional risk. PMID:29534543

  11. An international road map to improve pain assessment in people with impaired cognition: the development of the Pain Assessment in Impaired Cognition (PAIC) meta-tool.

    PubMed

    Corbett, Anne; Achterberg, Wilco; Husebo, Bettina; Lobbezoo, Frank; de Vet, Henrica; Kunz, Miriam; Strand, Liv; Constantinou, Marios; Tudose, Catalina; Kappesser, Judith; de Waal, Margot; Lautenbacher, Stefan

    2014-12-10

    Pain is common in people with dementia, yet identification is challenging. A number of pain assessment tools exist, utilizing observation of pain-related behaviours, vocalizations and facial expressions. Whilst they have been developed robustly, these often lack sufficient evidence of psychometric properties, like reliability, face and construct validity, responsiveness and usability, and are not internationally implemented. The EU-COST initiative "Pain in impaired cognition, especially dementia" aims to combine the expertise of clinicians and researchers to address this important issue by building on previous research in the area, identifying existing pain assessment tools for dementia, and developing consensus for items for a new universal meta-tool for use in research and clinical settings. This paper reports on the initial phase of this collaboration task. All existing observational pain behaviour tools were identified and elements categorised using a three-step reduction process. Selection and refinement of items for the draft Pain Assessment in Impaired Cognition (PAIC) meta-tool was achieved through scrutiny of the evidence, consensus of expert opinion, frequency of use and alignment with the American Geriatric Society guidelines. The main aim of this process was to identify key items with potential empirical, rather than theoretical value to take forward for testing. 12 eligible assessment tools were identified, and pain items categorised according to behaviour, facial expression and vocalisation according to the AGS guidelines (Domains 1 - 3). This has been refined to create the PAIC meta-tool for validation and further refinement. A decision was made to create a supporting comprehensive toolkit to support the core assessment tool to provide additional resources for the assessment of overlapping symptoms in dementia, including AGS domains four to six, identification of specific types of pain and assessment of duration and location of pain. This

  12. Conceptual assessment tool for advanced undergraduate electrodynamics

    NASA Astrophysics Data System (ADS)

    Baily, Charles; Ryan, Qing X.; Astolfi, Cecilia; Pollock, Steven J.

    2017-12-01

    As part of ongoing investigations into student learning in advanced undergraduate courses, we have developed a conceptual assessment tool for upper-division electrodynamics (E&M II): the Colorado UppeR-division ElectrodyNamics Test (CURrENT). This is a free response, postinstruction diagnostic with 6 multipart questions, an optional 3-question preinstruction test, and accompanying grading rubrics. The instrument's development was guided by faculty-consensus learning goals and research into common student difficulties. It can be used to gauge the effectiveness of transformed pedagogy, and to gain insights into student thinking in the covered topic areas. We present baseline data representing 500 students across 9 institutions, along with validity, reliability, and discrimination measures of the instrument and scoring rubric.

  13. Assessment Methodology for Process Validation Lifecycle Stage 3A.

    PubMed

    Sayeed-Desta, Naheed; Pazhayattil, Ajay Babu; Collins, Jordan; Chen, Shu; Ingram, Marzena; Spes, Jana

    2017-07-01

    The paper introduces evaluation methodologies and associated statistical approaches for process validation lifecycle Stage 3A. The assessment tools proposed can be applied to newly developed and launched small molecule as well as bio-pharma products, where substantial process and product knowledge has been gathered. The following elements may be included in Stage 3A: number of 3A batch determination; evaluation of critical material attributes, critical process parameters, critical quality attributes; in vivo in vitro correlation; estimation of inherent process variability (IPV) and PaCS index; process capability and quality dashboard (PCQd); and enhanced control strategy. US FDA guidance on Process Validation: General Principles and Practices, January 2011 encourages applying previous credible experience with suitably similar products and processes. A complete Stage 3A evaluation is a valuable resource for product development and future risk mitigation of similar products and processes. Elements of 3A assessment were developed to address industry and regulatory guidance requirements. The conclusions made provide sufficient information to make a scientific and risk-based decision on product robustness.

  14. The application of health literacy measurement tools (collective or individual domains) in assessing chronic disease management: a systematic review protocol.

    PubMed

    Shum, Jessica; Poureslami, Iraj; Doyle-Waters, Mary M; FitzGerald, J Mark

    2016-06-07

    The term "health literacy" (HL) was first coined in 1974, and its most common definition is currently defined as a person's ability to access, understand, evaluate, communicate, and use health information to make decisions for one's health. The previous systematic reviews assessing the effect of existing HL measurement tools on health outcomes have simply searched for the term "health literacy" only to identify measures instead of incorporating either one or more of the five domains in their search. Furthermore, as the domain "use" is fairly new, few studies have actually assessed this domain. In this protocol, we propose to identify and assess HL measures that applied the mentioned five domains either collectively or individually in assessing chronic disease management, in particular for asthma and chronic obstructive pulmonary disease (COPD). The ultimate goal is to provide recommendations towards the development and validation of a patient-centric HL measurement tool for the two diseases. A comprehensive, electronic search will be conducted to identify potential studies dating from 1974 to 2016 from databases such as Embase, MEDLINE, CINAHL, Cochrane Central Register of Controlled Trials, Web of Science, ERIC, PsycINFO, and HAPI. Database searches will be complemented with grey literature. Two independent reviewers will perform tool selection, study selection, data extraction, and quality assessment using pre-designed study forms. Any disagreement will be resolved through discussion or a third reviewer. Only studies that have developed or validated HL measurement tools (including one or more of the five domains mentioned above) among asthma and COPD patients will be included. Information collected from the studies will include instrument details such as versions, purpose, underlying constructs, administration, mapping of items onto the five domains, internal structure, scoring, response processes, standard error of measurement (SEM), correlation with other

  15. Cross Cultural Adaptation, Validity, and Reliability of the Farsi Breastfeeding Attrition Prediction Tools in Iranian Pregnant Women

    PubMed Central

    Mortazavi, Forough; Mousavi, Seyed Abbas; Chaman, Reza; Khosravi, Ahmad; Janke, Jill R.

    2015-01-01

    Background: The rate of exclusive breastfeeding in Iran is decreasing. The breastfeeding attrition prediction tools (BAPT) have been validated and used in predicting premature weaning. Objectives: We aimed to translate the BAPT into Farsi, assess its content validity, and examine its reliability and validity to identify exclusive breastfeeding discontinuation in Iran. Materials and Methods: The BAPT was translated into Farsi and the content validity of the Farsi version of the BAPT was assessed. It was administered to 356 pregnant women in the third trimester of pregnancy, who were residents of a city in northeast of Iran. The structural integrity of the four-factor model was assessed in confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). Reliability was assessed using Cronbach’s alpha coefficient and item-subscale correlations. Validity was assessed using the known-group comparison (128 with vs. 228 without breastfeeding experience) and predictive validity (80 successes vs. 265 failures in exclusive breastfeeding). Results: The internal consistency of the whole instrument (49 items) was 0.775. CFA provided an acceptable fit to the a priori four-factor model (Chi-square/df = 1.8, Root Mean Square Error of Approximation (RMSEA) = 0.049, Standardized Root Mean Square Residual (SRMR) = 0.064, Comparative Fit Index (CFI) = 0.911). The difference in means of breastfeeding control (BFC) between the participants with and without breastfeeding experience was significant (P < 0.001). In addition, the total score of BAPT and the score of Breast Feeding Control (BFC) subscale were higher in women who were on exclusive breastfeeding than women who were not, at four months postpartum (P < 0.05). Conclusions: This study validated the Farsi version of BAPT. It is useful for researchers who want to use it in Iran to identify women at higher risks of Exclusive Breast Feeding (EBF) discontinuation. PMID:26019910

  16. A comparison of sleep assessment tools by nurses and patients in critical care.

    PubMed

    Richardson, Annette; Crow, Wendy; Coghill, Elaine; Turnock, Christopher

    2007-09-01

    The aim of this critical care sleep assessment pilot study was to evaluate the usefulness of three sleep assessment tools to identify which, if any, provided the closest comparison between the nurses' judgement and the patients' experience of their sleep. The study objectives were to: (i) compare patients' and nurses' assessment of sleep using three different rating tools. (ii) Ascertain patients' preferences with non-interventional, user friendly, practical tools in critical care. (iii) Recommend changes and improvements to the way that sleep is assessed and documented. Sleep is important for promoting critical care recovery and sleep disturbance is known to cause irritability, aggression and increased stress levels. The availability and use of valid critical care sleep assessment tools is limited. A descriptive comparative study using three sleep assessment-rating scales were constructed to provide easy to understand tools for completion by both patients and nurses in critical care. Structured interviews were undertaken with 82 patients and 82 nurses using a convenience sample from four multispecialty critical care units in one large teaching trust. Patients were included in the study if they met a list of pre-defined criteria to obtain responses from lucid orientated patients. No tool produced a close association between the nurses' assessment of the patients sleep and the patients' assessment of their sleep. Patients found two of the three tools easy to use when rating their sleep. Discussion. Objective invasive measurements of sleep as well as complex subjective tools appear inappropriate to be used as a part of daily critical care practice. The application of simple rating scores has a high degree of error when nurses assess patients' sleep, even though high levels of patient observation and assessment are practiced in critical care. More research is needed to examine the assessment of sleep in critical care, particularly linking rating scales to alternative

  17. Development and validation of the Surgical Outcome Risk Tool (SORT)

    PubMed Central

    Protopapa, K L; Simpson, J C; Smith, N C E; Moonesinghe, S R

    2014-01-01

    Background Existing risk stratification tools have limitations and clinical experience suggests they are not used routinely. The aim of this study was to develop and validate a preoperative risk stratification tool to predict 30-day mortality after non-cardiac surgery in adults by analysis of data from the observational National Confidential Enquiry into Patient Outcome and Death (NCEPOD) Knowing the Risk study. Methods The data set was split into derivation and validation cohorts. Logistic regression was used to construct a model in the derivation cohort to create the Surgical Outcome Risk Tool (SORT), which was tested in the validation cohort. Results Prospective data for 19 097 cases in 326 hospitals were obtained from the NCEPOD study. Following exclusion of 2309, details of 16 788 patients were analysed (derivation cohort 11 219, validation cohort 5569). A model of 45 risk factors was refined on repeated regression analyses to develop a model comprising six variables: American Society of Anesthesiologists Physical Status (ASA-PS) grade, urgency of surgery (expedited, urgent, immediate), high-risk surgical specialty (gastrointestinal, thoracic, vascular), surgical severity (from minor to complex major), cancer and age 65 years or over. In the validation cohort, the SORT was well calibrated and demonstrated better discrimination than the ASA-PS and Surgical Risk Scale; areas under the receiver operating characteristic (ROC) curve were 0·91 (95 per cent c.i. 0·88 to 0·94), 0·87 (0·84 to 0·91) and 0·88 (0·84 to 0·92) respectively (P < 0·001). Conclusion The SORT allows rapid and simple data entry of six preoperative variables, and provides a percentage mortality risk for individuals undergoing surgery. PMID:25388883

  18. The web-buffet--development and validation of an online tool to measure food choice.

    PubMed

    Bucher, Tamara; Keller, Carmen

    2015-08-01

    To date, no data exist on the agreement of food choice measured using an online tool with subsequent actual consumption. This needs to be shown before food choice, measured by means of an online tool, is used as a dependent variable to examine intake in the general population. A 'web-buffet' was developed to assess food choice. Choice was measured as planned meal composition from photographic material; respondents chose preferred foods and proportions for a main meal (out of a possible 144 combinations) online and the validity was assessed by comparison of a meal composed from a web-buffet with actual food intake 24-48 h later. Furthermore, correlations of food preferences, energy needs and health interest with meals chosen from the web-buffet were analysed. Students: n 106 (Study I), n 32 (Study II). Meals chosen from the web-buffet (mean = 2998 kJ, SD = 471 kJ) agreed with actual consumption (rs = 0.63, P < 0.001) but were on average 367 kJ (10.5%) lower in energy than consumed meals (mean = 3480 kJ, SD = 755 kJ). Preferences were highly associated with chosen amounts and health interest was negatively correlated with the energy selected (rs = -0.40, P<0.001). Meal composition choice in the web-buffet agrees sufficiently well with actual intake to measure food choice as a dependent variable in online surveys. However, we found an average underestimation of subsequent consumption. High correlations of preferences with chosen amounts and an inverse association of health interest with total energy further indicate the validity of the tool. Applications in behavioural nutrition research are discussed.

  19. Evaluation of tools used to measure calcium and/or dairy consumption in adults.

    PubMed

    Magarey, Anthea; Baulderstone, Lauren; Yaxley, Alison; Markow, Kylie; Miller, Michelle

    2015-05-01

    To identify and critique tools for the assessment of Ca and/or dairy intake in adults, in order to ascertain the most accurate and reliable tools available. A systematic review of the literature was conducted using defined inclusion and exclusion criteria. Articles reporting on originally developed tools or testing the reliability or validity of existing tools that measure Ca and/or dairy intake in adults were included. Author-defined criteria for reporting reliability and validity properties were applied. Studies conducted in Western countries. Adults. Thirty papers, utilising thirty-six tools assessing intake of dairy, Ca or both, were identified. Reliability testing was conducted on only two dairy and five Ca tools, with results indicating that only one dairy and two Ca tools were reliable. Validity testing was conducted for all but four Ca-only tools. There was high reliance in validity testing on lower-order tests such as correlation and failure to differentiate between statistical and clinically meaningful differences. Results of the validity testing suggest one dairy and five Ca tools are valid. Thus one tool was considered both reliable and valid for the assessment of dairy intake and only two tools proved reliable and valid for the assessment of Ca intake. While several tools are reliable and valid, their application across adult populations is limited by the populations in which they were tested. These results indicate a need for tools that assess Ca and/or dairy intake in adults to be rigorously tested for reliability and validity.

  20. [Assessing work-related stress: an Italian adaptation of the HSE Management Standards Work-Related Stress Indicator Tool].

    PubMed

    Marcatto, Francesco; D'Errico, Giuseppe; Di Blas, Lisa; Ferrante, Donatella

    2011-01-01

    The aim of this paper is to present a preliminary validation of an Italian adaptation of the HSE Management Standards Work-Related Stress Indicator Tool (IT), an instrument for assessing work-related stress at the organizational level, originally developed in Britain by the Health and Safety Executive. A scale that assesses the physical work environment has been added to the original version of the IT. 190 employees of the University of Trieste have been enrolled in the study. A confirmatory analysis showed a satisfactory fit of the eight-factors structure of the instrument. Further psychometric analysis showed adequate internal consistency of the IT scales and good criterion validity, as evidenced by the correlations with self-perception of stress, work satisfaction and motivation. In conclusion, the Indicator Tool proved to be a valid and reliable instrument for the assessment of work-related stress at the organizational level, and it is also compatible with the instructions provided by the Ministry of Labour and Social Policy (Circular letter 18/11/2010).

  1. Psychometric properties of Malay neuropsychiatry unit cognitive assessment tool among Alzheimer's disease patients in comparison to Malay Montreal Cognitive Assessment.

    PubMed

    Thong, Kai Shin; Chee, Kok Yoon; Ng, Chong Guan; Walterfang, Mark; Velakoulis, Dennis

    2016-09-01

    This study aims to establish psychometric properties of the Malay Neuropsychiatry Unit Cognitive Assessment Tool (Malay NuCOG) in Alzheimer's disease. NuCOG was translated to Malay language and compared with Montreal Cognitive Assessment Tool on 80 individuals. The Malay NuCOG showed good internal consistency and reliability (Cronbach's alpha = 0.895). It demonstrated 100% sensitivity and 87.5% specificity at the cutoff score of 78.50/100. The Malay NuCOG is a valid and reliable cognitive instrument that is sensitive and specific for the detection of dementia and has clinical advantages in its ability to examine individual cognitive domains. © 2015 Wiley Publishing Asia Pty Ltd.

  2. Deconstruction Rapid Assessment Tool

    EPA Pesticide Factsheets

    Deconstruction Rapid Assessment Tool (EPA 905-F-15-001) instructions, form and spreadsheet for assessing and triaging structures being considered for deconstruction. Promote environmental stewardship and economic revitalization through deconstruction.

  3. Development and Validation of a Practical Instrument for Injury Prevention: The Occupational Safety and Health Monitoring and Assessment Tool (OSH-MAT).

    PubMed

    Sun, Yi; Arning, Martin; Bochmann, Frank; Börger, Jutta; Heitmann, Thomas

    2018-06-01

    The Occupational Safety and Health Monitoring and Assessment Tool (OSH-MAT) is a practical instrument that is currently used in the German woodworking and metalworking industries to monitor safety conditions at workplaces. The 12-item scoring system has three subscales rating technical, organizational, and personnel-related conditions in a company. Each item has a rating value ranging from 1 to 9, with higher values indicating higher standard of safety conditions. The reliability of this instrument was evaluated in a cross-sectional survey among 128 companies and its validity among 30,514 companies. The inter-rater reliability of the instrument was examined independently and simultaneously by two well-trained safety engineers. Agreement between the double ratings was quantified by the intraclass correlation coefficient and absolute agreement of the rating values. The content validity of the OSH-MAT was evaluated by quantifying the association between OSH-MAT values and 5-year average injury rates by Poisson regression analysis adjusted for the size of the companies and industrial sectors. The construct validity of OSH-MAT was examined by principle component factor analysis. Our analysis indicated good to very good inter-rater reliability (intraclass correlation coefficient = 0.64-0.74) of OSH-MAT values with an absolute agreement of between 72% and 81%. Factor analysis identified three component subscales that met exactly the structure theory of this instrument. The Poisson regression analysis demonstrated a statistically significant exposure-response relationship between OSH-MAT values and the 5-year average injury rates. These analyses indicate that OSH-MAT is a valid and reliable instrument that can be used effectively to monitor safety conditions at workplaces.

  4. The Computerized Perceptual Motor Skills Assessment: A new visual perceptual motor skills evaluation tool for children in early elementary grades.

    PubMed

    Howe, Tsu-Hsin; Chen, Hao-Ling; Lee, Candy Chieh; Chen, Ying-Dar; Wang, Tien-Ni

    2017-10-01

    Visual perceptual motor skills have been proposed as underlying courses of handwriting difficulties. However, there is no evaluation tool currently available to assess these skills comprehensively and to serve as a sensitive measure. The purpose of this study was to validate the Computerized Perceptual Motor Skills Assessment (CPMSA), a newly developed evaluation tool for children in early elementary grades. Its test-retest reliability, concurrent validity, discriminant validity, and responsiveness were examined in 43 typically developing children and 26 children with handwriting difficulty. The CPMSA demonstrated excellent reliability across all subtests with intra-class correlation coefficients (ICCs)≥0.80. Significant moderate correlations between the domains of the CPMSA and corresponding gold standards including Beery VMI, the TVPS-3, and the eye-hand coordination subtest of the DTVP-2 demonstrated good concurrent validity. In addition, the CPMSA showed evidence of discriminant validity in samples of children with and without handwriting difficulty. This article provides evidence in support of the CPMSA. The CPMSA is a reliable, valid, and promising measure of visual perceptual motor skills for children in early elementary grades. Directions for future study and improvements to the assessment are discussed. Copyright © 2017. Published by Elsevier Ltd.

  5. Spreadsheet Assessment Tool v. 2.4

    DOE Office of Scientific and Technical Information (OSTI.GOV)

    Allen, David J.; Martinez, Ruben

    2016-03-03

    The Spreadsheet Assessment Tool (SAT) is an easy to use, blast assessment tool that is intended to estimate the potential risk due to an explosive attack on a blood irradiator. The estimation of risk is based on the methodology, assumptions, and results of a detailed blast effects assessment study that is summarized in Sandia National Laboratories Technical Report SAND2015-6166. Risk as defined in the report and as used in the SAT is: "The potential risk of creating an air blast-induced vent opening at a buildings envelope surface". Vent openings can be created at a buildings envelope through the failure ofmore » an exterior building component—like a wall, window, or door—due to an explosive sabotage of an irradiator within the building. To estimate risk, the tool requires that users obtain and input information pertaining to the building's characteristics and the irradiator location. The tool also suggests several prescriptive mitigation strategies that can be considered to reduce risk. Given the variability in civilian building construction practices, the input parameters used by this tool may not apply to all buildings being assessed. The tool should not be used as a substitute for engineering judgment. The tool is intended for assessment purposes only.« less

  6. Validation of an observation tool to assess physical activity-promoting physical education lessons in high schools: SOFIT.

    PubMed

    Fairclough, Stuart J; Weaver, R Glenn; Johnson, Siobhan; Rawlinson, Jack

    2018-05-01

    SOFIT+ is an observation tool to measure teacher practices related to moderate-to-vigorous physical activity (MVPA) promotion during physical education (PE). The objective of the study was to examine the validity of SOFIT+ during high school PE lessons. This cross-sectional, observational study tested the construct validity of SOFIT+ in boys' and girls' high school PE lessons. Twenty-one PE lessons were video-recorded and retrospectively coded using SOFIT+. Students wore hip-mounted accelerometers during lessons as an objective measure of MVPA. Multinomial logistic regression was used to estimate the likelihood of students engaging in MVPA during different teacher practices represented by observed individual codes and a combined SOFIT+ index-score. Fourteen individual SOFIT+ variables demonstrated a statistically significant relationship with girls' and boys' MVPA. Observed lesson segments identified as high MVPA-promoting were related to an increased likelihood of girls engaging in 5-10 (OR=2.86 [95% CI 2.41-3.40]), 15-25 (OR=7.41 [95% CI 6.05-9.06]), and 30-40 (OR=22.70 [95% CI 16.97-30.37])s of MVPA. For boys, observed high-MVPA promoting segments were related to an increased likelihood of engaging in 5-10 (OR=1.71 [95% CI 1.45-2.01]), 15-25 (OR=2.69 [95% CI 2.31-3.13]) and 30-40 (OR=4.26 [95% CI 3.44-5.29])s of MVPA. Teacher practices during high school PE lessons are significantly related to students' participation in MVPA. SOFIT+ is a valid and reliable tool to examine relationships between PE teacher practices and student MVPA during PE. Copyright © 2017 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

  7. cluML: A markup language for clustering and cluster validity assessment of microarray data.

    PubMed

    Bolshakova, Nadia; Cunningham, Pádraig

    2005-01-01

    cluML is a new markup language for microarray data clustering and cluster validity assessment. The XML-based format has been designed to address some of the limitations observed in traditional formats, such as inability to store multiple clustering (including biclustering) and validation results within a dataset. cluML is an effective tool to support biomedical knowledge representation in gene expression data analysis. Although cluML was developed for DNA microarray analysis applications, it can be effectively used for the representation of clustering and for the validation of other biomedical and physical data that has no limitations.

  8. Assessment Tools for Adult Education.

    ERIC Educational Resources Information Center

    Shefrin, Carol; Shafer, Dehra; Forlizzi, Lori

    The Assessment Tools for Adult Education project was designed to provide training and support to staff of the Pennsylvania Bureau of Adult Basic and Literacy Education (ABLE) funded programs to help them use assessment tools and procedures to document the learning gains of the adult students they serve. The following candidate assessment…

  9. Validation and Development of a Modified Breast Graded Prognostic Assessment As a Tool for Survival in Patients With Breast Cancer and Brain Metastases.

    PubMed

    Subbiah, Ishwaria M; Lei, Xiudong; Weinberg, Jeffrey S; Sulman, Erik P; Chavez-MacGregor, Mariana; Tripathy, Debu; Gupta, Rohan; Varma, Ankur; Chouhan, Jay; Guevarra, Richard P; Valero, Vicente; Gilbert, Mark R; Gonzalez-Angulo, Ana M

    2015-07-10

    Several indices have been developed to predict overall survival (OS) in patients with breast cancer with brain metastases, including the breast graded prognostic assessment (breast-GPA), comprising age, tumor subtype, and Karnofsky performance score. However, number of brain metastases-a highly relevant clinical variable-is less often incorporated into the final model. We sought to validate the existing breast-GPA in an independent larger cohort and refine it integrating number of brain metastases. Data were retrospectively gathered from a prospectively maintained institutional database. Patients with newly diagnosed brain metastases from 1996 to 2013 were identified. After validating the breast-GPA, multivariable Cox regression and recursive partitioning analysis led to the development of the modified breast-GPA. The performances of the breast-GPA and modified breast-GPA were compared using the concordance index. In our cohort of 1,552 patients, the breast-GPA was validated as a prognostic tool for OS (P < .001). In multivariable analysis of the breast-GPA and number of brain metastases (> three v ≤ three), both were independent predictors of OS. We therefore developed the modified breast-GPA integrating a fourth clinical parameter. Recursive partitioning analysis reinforced the prognostic significance of these four factors. Concordance indices were 0.78 (95% CI, 0.77 to 0.80) and 0.84 (95% CI, 0.83 to 0.85) for the breast-GPA and modified breast-GPA, respectively (P < .001). The modified breast-GPA incorporates four simple clinical parameters of high prognostic significance. This index has an immediate role in the clinic as a formative part of the clinician's discussion of prognosis and direction of care and as a potential patient selection tool for clinical trials. © 2015 by American Society of Clinical Oncology.

  10. Linguistic validation of the Alberta Context Tool and two measures of research use, for German residential long term care

    PubMed Central

    2014-01-01

    Background To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks’ Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity–an initial stage of validity, necessary before more advanced validity assessment. Methods We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants’ answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants. Results Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups. Conclusion Cognitive debriefing is essential to detect and respond to problematic instrument

  11. Measuring Decision-Making During Thyroidectomy: Validity Evidence for a Web-Based Assessment Tool.

    PubMed

    Madani, Amin; Gornitsky, Jordan; Watanabe, Yusuke; Benay, Cassandre; Altieri, Maria S; Pucher, Philip H; Tabah, Roger; Mitmaker, Elliot J

    2018-02-01

    Errors in judgment during thyroidectomy can lead to recurrent laryngeal nerve injury and other complications. Despite the strong link between patient outcomes and intraoperative decision-making, methods to evaluate these complex skills are lacking. The purpose of this study was to develop objective metrics to evaluate advanced cognitive skills during thyroidectomy and to obtain validity evidence for them. An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from four institutions completed a 33-item assessment, developed based on a cognitive task analysis and expert Delphi consensus. Sixteen items required subjects to make annotations on still frames of thyroidectomy videos, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test," VCT). Seven items were short answer (SA), requiring users to type their answers, and scores were automatically calculated based on their similarity to a pre-populated repertoire of correct responses. Test-retest reliability, internal consistency, and correlation of scores with self-reported experience and training level (novice, intermediate, expert) were calculated. Twenty-eight subjects (10 endocrine surgeons and otolaryngologists, 18 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.96; n = 10) and internal consistency (Cronbach's α = 0.93). The assessment demonstrated significant differences between novices, intermediates, and experts in total score (p < 0.01), VCT score (p < 0.01) and SA score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.95, p < 0.01), between total case number and VCT score (ρ = 0.93, p < 0.01), and between total case number and SA score (ρ = 0.83, p < 0.01). This study describes the development of novel metrics and provides validity evidence for an interactive Web-based platform

  12. Tools used to assess medical students competence in procedural skills at the end of a primary medical degree: a systematic review.

    PubMed

    Morris, Marie C; Gallagher, Tom K; Ridgway, Paul F

    2012-01-01

    The objective was to systematically review the literature to identify and grade tools used for the end point assessment of procedural skills (e.g., phlebotomy, IV cannulation, suturing) competence in medical students prior to certification. The authors searched eight bibliographic databases electronically - ERIC, Medline, CINAHL, EMBASE, Psychinfo, PsychLIT, EBM Reviews and the Cochrane databases. Two reviewers independently reviewed the literature to identify procedural assessment tools used specifically for assessing medical students within the PRISMA framework, the inclusion/exclusion criteria and search period. Papers on OSATS and DOPS were excluded as they focused on post-registration assessment and clinical rather than simulated competence. Of 659 abstracted articles 56 identified procedural assessment tools. Only 11 specifically assessed medical students. The final 11 studies consisted of 1 randomised controlled trial, 4 comparative and 6 descriptive studies yielding 12 heterogeneous procedural assessment tools for analysis. Seven tools addressed four discrete pre-certification skills, basic suture (3), airway management (2), nasogastric tube insertion (1) and intravenous cannulation (1). One tool used a generic assessment of procedural skills. Two tools focused on postgraduate laparoscopic skills and one on osteopathic students and thus were not included in this review. The levels of evidence are low with regard to reliability - κ = 0.65-0.71 and minimum validity is achieved - face and content. In conclusion, there are no tools designed specifically to assess competence of procedural skills in a final certification examination. There is a need to develop standardised tools with proven reliability and validity for assessment of procedural skills competence at the end of medical training. Medicine graduates must have comparable levels of procedural skills acquisition entering the clinical workforce irrespective of the country of training.

  13. Measuring the food service environment: development and implementation of assessment tools.

    PubMed

    Minaker, Leia M; Raine, Kim D; Cash, Sean B

    2009-01-01

    The food environment is increasingly being implicated in the obesity epidemic, though few reported measures of it exist. In order to assess the impact of the food environment on food intake, valid measures must be developed and tested. The current study describes the development of a food service environment assessment tool and its implementation in a community setting. A descriptive study with mixed qualitative and quantitative methods at a large, North American university campus was undertaken. Measures were developed on the basis of a conceptual model of nutrition environments. Measures of community nutrition environment were the number, type and hours of operation of each food service outlet on campus. Measures of consumer nutrition environment were food availability, food affordability, food promotion and nutrition information availability. Seventy-five food service outlets within the geographic boundaries were assessed. Assessment tools could be implemented in a reasonable amount of time and showed good face and content validity. The food environments were described and measures were grouped so that food service outlet types could be compared in terms of purchasing convenience, cost/value, healthy food promotion and health. Food service outlet types that scored higher in purchasing convenience and cost/value tended to score lower in healthy food promotion and health. This study adds evidence that food service outlet types that are convenient to consumers and supply high value (in terms of calories per dollar) tend to be less health-promoting. Results from this study also suggest the possibility of characterizing the food environment according to the type of food service outlet observed.

  14. When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments.

    PubMed

    Cook, David A; Kuper, Ayelet; Hatala, Rose; Ginsburg, Shiphra

    2016-10-01

    Quantitative scores fail to capture all important features of learner performance. This awareness has led to increased use of qualitative data when assessing health professionals. Yet the use of qualitative assessments is hampered by incomplete understanding of their role in forming judgments, and lack of consensus in how to appraise the rigor of judgments therein derived. The authors articulate the role of qualitative assessment as part of a comprehensive program of assessment, and translate the concept of validity to apply to judgments arising from qualitative assessments. They first identify standards for rigor in qualitative research, and then use two contemporary assessment validity frameworks to reorganize these standards for application to qualitative assessment.Standards for rigor in qualitative research include responsiveness, reflexivity, purposive sampling, thick description, triangulation, transparency, and transferability. These standards can be reframed using Messick's five sources of validity evidence (content, response process, internal structure, relationships with other variables, and consequences) and Kane's four inferences in validation (scoring, generalization, extrapolation, and implications). Evidence can be collected and evaluated for each evidence source or inference. The authors illustrate this approach using published research on learning portfolios.The authors advocate a "methods-neutral" approach to assessment, in which a clearly stated purpose determines the nature of and approach to data collection and analysis. Increased use of qualitative assessments will necessitate more rigorous judgments of the defensibility (validity) of inferences and decisions. Evidence should be strategically sought to inform a coherent validity argument.

  15. Tensions Between Validity and Outcomes: Teacher Assessment of Written Work of Recently Arrived Immigrant ESL Students

    ERIC Educational Resources Information Center

    Arkoudis, Sophie; O'Loughlin, Kieran

    2004-01-01

    This article reports on a collaborative study involving ESL teachers in an Australian English Language Centre as they work through some of their concerns about reliability and validity in their assessment practices. The focus of this article is on how teachers work with the Curriculum Standards Framework (CSF) as an assessment tool. The discussion…

  16. Performance comparison of SNP detection tools with illumina exome sequencing data—an assessment using both family pedigree information and sample-matched SNP array data

    PubMed Central

    Yi, Ming; Zhao, Yongmei; Jia, Li; He, Mei; Kebebew, Electron; Stephens, Robert M.

    2014-01-01

    To apply exome-seq-derived variants in the clinical setting, there is an urgent need to identify the best variant caller(s) from a large collection of available options. We have used an Illumina exome-seq dataset as a benchmark, with two validation scenarios—family pedigree information and SNP array data for the same samples, permitting global high-throughput cross-validation, to evaluate the quality of SNP calls derived from several popular variant discovery tools from both the open-source and commercial communities using a set of designated quality metrics. To the best of our knowledge, this is the first large-scale performance comparison of exome-seq variant discovery tools using high-throughput validation with both Mendelian inheritance checking and SNP array data, which allows us to gain insights into the accuracy of SNP calling through such high-throughput validation in an unprecedented way, whereas the previously reported comparison studies have only assessed concordance of these tools without directly assessing the quality of the derived SNPs. More importantly, the main purpose of our study was to establish a reusable procedure that applies high-throughput validation to compare the quality of SNP discovery tools with a focus on exome-seq, which can be used to compare any forthcoming tool(s) of interest. PMID:24831545

  17. Quantitative analysis of the rubric as an assessment tool: an empirical study of student peer-group rating

    NASA Astrophysics Data System (ADS)

    Hafner, John C.; Hafner, Patti M.

    2003-12-01

    Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool in the hands of the students. This study focuses on the validity and reliability of the rubric as an assessment tool for student peer-group evaluation in an effort to further explore the use and effectiveness of the rubric. A total of 1577 peer-group ratings using a rubric for an oral presentation was used in this 3-year study involving 107 college biology students. A quantitative analysis of the rubric used in this study shows that it is used consistently by both students and the instructor across the study years. Moreover, the rubric appears to be 'gender neutral' and the students' academic strength has no significant bearing on the way that they employ the rubric. A significant, one-to-one relationship (slope = 1.0) between the instructor's assessment and the students' rating is seen across all years using the rubric. A generalizability study yields estimates of inter-rater reliability of moderate values across all years and allows for the estimation of variance components. Taken together, these data indicate that the general form and evaluative criteria of the rubric are clear and that the rubric is a useful assessment tool for peer-group (and self-) assessment by students. To our knowledge, these data provide the first statistical documentation of the validity and reliability of the rubric for student peer-group assessment.

  18. Case finding of lifestyle and mental health disorders in primary care: validation of the ‘CHAT’ tool

    PubMed Central

    Goodyear-Smith, Felicity; Coupe, Nicole M; Arroll, Bruce; Elley, C Raina; Sullivan, Sean; McGill, Anne-Thea

    2008-01-01

    Background Primary care is accessible and ideally placed for case finding of patients with lifestyle and mental health risk factors and subsequent intervention. The short self-administered Case-finding and Help Assessment Tool (CHAT) was developed for lifestyle and mental health assessment of adult patients in primary health care. This tool checks for tobacco use, alcohol and other drug misuse, problem gambling, depression, anxiety and stress, abuse, anger problems, inactivity, and eating disorders. It is well accepted by patients, GPs and nurses. Aim To assess criterion-based validity of CHAT against a composite gold standard. Design of study Conducted according to the Standards for Reporting of Diagnostic Accuracy statement for diagnostic tests. Setting Primary care practices in Auckland, New Zealand. Method One thousand consecutive adult patients completed CHAT and a composite gold standard. Sensitivities, specificities, positive and negative predictive values, and likelihood ratios were calculated. Results Response rates for each item ranged from 79.6 to 99.8%. CHAT was sensitive and specific for almost all issues screened, except exercise and eating disorders. Sensitivity ranged from 96% (95% confidence interval [CI] = 87 to 99%) for major depression to 26% (95% CI = 22 to 30%) for exercise. Specificity ranged from 97% (95% CI = 96 to 98%) for problem gambling and problem drug use to 40% (95% CI = 36 to 45%) for exercise. All had high likelihood ratios (3–30), except exercise and eating disorders. Conclusion CHAT is a valid and acceptable case-finding tool for most common lifestyle and mental health conditions. PMID:18186993

  19. Validation of a virtual reality balance module for use in clinical concussion assessment and management.

    PubMed

    Teel, Elizabeth F; Slobounov, Semyon M

    2015-03-01

    To determine the criterion and content validity of a virtual reality (VR) balance module for use in clinical practice. Retrospective, VR balance module completed by participants during concussion baseline or assessment testing session. A Pennsylvania State University research laboratory. A total of 60 control and 28 concussed students and athletes from the Pennsylvania State University. None. This study examined: (1) the relationship between VR composite balance scores (final, stationary, yaw, pitch, and roll) and area of the center-of-pressure (eyes open and closed) scores and (2) group differences (normal volunteers and concussed student-athletes) on VR composite balance scores. With the exception of the stationary composite score, all other VR balance composite scores were significantly correlated with the center of pressure data obtained from a force platform. Significant correlations ranged from r = -0.273 to -0.704 for the eyes open conditions and from r = -0.353 to -0.876 for the eyes closed condition. When examining group differences on the VR balance composite modules, the concussed group did significantly (P < 0.01) worse on all measures compared with the control group. The VR balance module met or exceeded the criterion and content validity standard set by the current balance tools and may be appropriate for use in a clinical concussion setting. Virtual reality balance module is a valid tool for concussion assessment in clinical settings. This novel type of balance assessment may be more sensitive to concussion diagnoses, especially later (7-10 days) in the recovery phase than current clinical balance tools.

  20. Evaluation of Phosphorus Site Assessment Tools: Lessons from the USA.

    PubMed

    Sharpley, Andrew; Kleinman, Peter; Baffaut, Claire; Beegle, Doug; Bolster, Carl; Collick, Amy; Easton, Zachary; Lory, John; Nelson, Nathan; Osmond, Deanna; Radcliffe, David; Veith, Tamie; Weld, Jennifer

    2017-11-01

    Critical source area identification through phosphorus (P) site assessment is a fundamental part of modern nutrient management planning in the United States, yet there has been only sparse testing of the many versions of the P Index that now exist. Each P site assessment tool was developed to be applicable across a range of field conditions found in a given geographic area, making evaluation extremely difficult. In general, evaluation with in-field monitoring data has been limited, focusing primarily on corroborating manure and fertilizer "source" factors. Thus, a multiregional effort (Chesapeake Bay, Heartland, and Southern States) was undertaken to evaluate P Indices using a combination of limited field data, as well as output from simulation models (i.e., Agricultural Policy Environmental eXtender, Annual P Loss Estimator, Soil and Water Assessment Tool [SWAT], and Texas Best Management Practice Evaluation Tool [TBET]) to compare against P Index ratings. These comparisons show promise for advancing the weighting and formulation of qualitative P Index components but require careful vetting of the simulation models. Differences among regional conclusions highlight model strengths and weaknesses. For example, the Southern States region found that, although models could simulate the effects of nutrient management on P runoff, they often more accurately predicted hydrology than total P loads. Furthermore, SWAT and TBET overpredicted particulate P and underpredicted dissolved P, resulting in correct total P predictions but for the wrong reasons. Experience in the United States supports expanded regional approaches to P site assessment, assuming closely coordinated efforts that engage science, policy, and implementation communities, but limited scientific validity exists for uniform national P site assessment tools at the present time. Copyright © by the American Society of Agronomy, Crop Science Society of America, and Soil Science Society of America, Inc.

  1. Development of the Canadian Physiotherapy Assessment of Clinical Performance: A New Tool to Assess Physiotherapy Students' Performance in Clinical Education.

    PubMed

    Mori, Brenda; Brooks, Dina; Norman, Kathleen E; Herold, Jodi; Beaton, Dorcas E

    2015-08-01

    To develop the first draft of a Canadian tool to assess physiotherapy (PT) students' performance in clinical education (CE). Phase 1: to gain consensus on the items within the new tool, the number and placement of the comment boxes, and the rating scale; Phase 2: to explore the face and content validity of the draft tool. Phase 1 used the Delphi method; Phase 2 used cognitive interviewing methods with recent graduates and clinical instructors (CIs) and detailed interviews with clinical education and measurement experts. Consensus was reached on the first draft of the new tool by round 3 of the Delphi process, which was completed by 21 participants. Interviews were completed with 13 CIs, 6 recent graduates, and 7 experts. Recent graduates and CIs were able to interpret the tool accurately, felt they could apply it to a recent CE experience, and provided suggestions to improve the draft. Experts provided salient advice. The first draft of a new tool to assess PT students in CE, the Canadian Physiotherapy Assessment of Clinical Performance (ACP), was developed and will undergo further development and testing, including national consultation with stakeholders. Data from Phase 2 will contribute to developing an online education module for CIs and students.

  2. Characterizing the literature on validity and assessment in medical education: a bibliometric study.

    PubMed

    Young, Meredith; St-Onge, Christina; Xiao, Jing; Vachon Lachiver, Elise; Torabi, Nazi

    2018-05-23

    Assessment in Medical Education fills many roles and is under constant scrutiny. Assessments must be of good quality, and supported by validity evidence. Given the high-stakes consequences of assessment, and the many audiences within medical education (e. g., training level, specialty-specific), we set out to document the breadth, scope, and characteristics of the literature reporting on validation of assessments within medical education. Searches in Medline (Ovid), Web of Science, ERIC, EMBASE (Ovid), and PsycINFO (Ovid) identified articles reporting on assessment of learners in medical education published since 1999. Included articles were coded for geographic origin, journal, journal category, targeted assessment, and authors. A map of collaborations between prolific authors was generated. A total of 2,863 articles were included. The majority of articles were from the United States, with Canada producing the most articles per medical school. Most articles were published in journals with medical categorizations (73.1% of articles), but Medical Education was the most represented journal (7.4% of articles). Articles reported on a variety of assessment tools and approaches, and 89 prolific authors were identified, with a total of 228 collaborative links. Literature reporting on validation of assessments in medical education is heterogeneous. Literature is produced by a broad array of authors and collaborative networks, reported to a broad audience, and is primarily generated in North American and European contexts. Our findings speak to the heterogeneity of the medical education literature on assessment validation, and suggest that this heterogeneity may stem, at least in part, from differences in constructs measured, assessment purposes, or conceptualizations of validity.

  3. Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly.

    PubMed

    Brouillette, Robert M; Foil, Heather; Fontenot, Stephanie; Correro, Anthony; Allen, Ray; Martin, Corby K; Bruce-Keller, Annadora J; Keller, Jeffrey N

    2013-01-01

    While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these "next-generation" assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST) in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, p<0.0001) and multiple measures of processing speed and attention (Digit Span: r = 0.427, p<0.0001; Trail Making Test: r = -0.651, p<0.00001; Digit Symbol Test: r = 0.508, p<0.0001). The CST was not correlated with naming and verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming) or memory tasks (Logical Memory Test). Test re-test reliability was observed to be significant (r = 0.726; p = 0.02). Together, these data are the first to demonstrate the feasibility, reliability, and validity of using a smartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries of cognitive

  4. [Validity of the suicidality assessment instrument PRISM-S (Pictoral Representation of Illness Self Measure - Suicidality)].

    PubMed

    Ring, Mariann; Harbauer, Gregor; Haas, Sebastian; Schuetz, Christopher; Andreae, Andreas; Maercker, Andreas; Ajdacic-Gross, Vladeta

    2014-01-01

    In routine clinical practice the assessment of suicidality proves to be difficult and complex. The aim of the present study was to examine if PRISM can be used to measure validly the person's subjectively perceived suicidality. The nonverbal visualization technique PRISM (Pictoral Representation of Illness and Self Measure) has been developed by Büchi et al. (2002) to evaluate the perceived burden of suffering due to physical illness. The adapted version of PRISM used in our study is called PRISM-S (Pictoral Representation of Illness and Self Measure - Suicidality). 156 eligible inpatients, admitted voluntarily to the crisis intervention centre Winterthur, participated in the study. We used as gold standards the well established assessment tools the Beck Scale of Suicide Ideation (BSS) and the Depressive Symptome Inventory - Subscale (DSI-SS). The results showed high correlations between PRISM-S and the BSS (r = - 0,73) and the DSI-SS scores (r = - 0,76). Clinicians, general practitioners, psychiatrists and psychologists receive with PRISM-S a valid suicidality assessment tool that is very brief and easy to administer in clinical settings.

  5. Validation of a Brief Questionnaire Against Direct Observation to Assess Adolescents' School Lunchtime Beverage Consumption.

    PubMed

    Grummon, Anna H; Hampton, Karla E; Hecht, Amelie; Oliva, Ariana; McCulloch, Charles E; Brindis, Claire D; Patel, Anisha I

    Beverage consumption is an important determinant of youth health outcomes. Beverage interventions often occur in schools, yet no brief validated questionnaires exist to assess whether these efforts improve in-school beverage consumption. This study validated a brief questionnaire to assess beverage consumption during school lunch. Researchers observed middle school students' (n = 25) beverage consumption during school lunchtime using a standardized tool. After lunch, students completed questionnaires regarding their lunchtime beverage consumption. Kappa statistics compared self-reported with observed beverage consumption across 15 beverage categories. Eight beverages showed at least fair agreement (kappa [κ] > 0.20) for both type and amount consumed, with most showing substantial agreement (κ > 0.60). One beverage had high raw agreement but κ < 0.20. Six beverages had too few ratings to compute κ's. This brief questionnaire was useful for assessing school lunchtime consumption of many beverages and provides a low-cost tool for evaluating school-based beverage interventions. Copyright © 2017 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

  6. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation.

    PubMed

    Benedict, Ralph H B; Amato, Maria Pia; Boringa, Jan; Brochet, Bruno; Foley, Fred; Fredrikson, Stan; Hamalainen, Paivi; Hartung, Hans; Krupp, Lauren; Penner, Iris; Reder, Anthony T; Langdon, Dawn

    2012-07-16

    An international expert consensus committee recently recommended a brief battery of tests for cognitive evaluation in multiple sclerosis. The Brief International Cognitive Assessment for MS (BICAMS) battery includes tests of mental processing speed and memory. Recognizing that resources for validation will vary internationally, the committee identified validation priorities, to facilitate international acceptance of BICAMS. Practical matters pertaining to implementation across different languages and countries were discussed. Five steps to achieve optimal psychometric validation were proposed. In Step 1, test stimuli should be standardized for the target culture or language under consideration. In Step 2, examiner instructions must be standardized and translated, including all information from manuals necessary for administration and interpretation. In Step 3, samples of at least 65 healthy persons should be studied for normalization, matched to patients on demographics such as age, gender and education. The objective of Step 4 is test-retest reliability, which can be investigated in a small sample of MS and/or healthy volunteers over 1-3 weeks. Finally, in Step 5, criterion validity should be established by comparing MS and healthy controls. At this time, preliminary studies are underway in a number of countries as we move forward with this international assessment tool for cognition in MS.

  7. Computational assessment of hemodynamics-based diagnostic tools using a database of virtual subjects: Application to three case studies.

    PubMed

    Willemet, Marie; Vennin, Samuel; Alastruey, Jordi

    2016-12-08

    Many physiological indexes and algorithms based on pulse wave analysis have been suggested in order to better assess cardiovascular function. Because these tools are often computed from in-vivo hemodynamic measurements, their validation is time-consuming, challenging, and biased by measurement errors. Recently, a new methodology has been suggested to assess theoretically these computed tools: a database of virtual subjects generated using numerical 1D-0D modeling of arterial hemodynamics. The generated set of simulations encloses a wide selection of healthy cases that could be encountered in a clinical study. We applied this new methodology to three different case studies that demonstrate the potential of our new tool, and illustrated each of them with a clinically relevant example: (i) we assessed the accuracy of indexes estimating pulse wave velocity; (ii) we validated and refined an algorithm that computes central blood pressure; and (iii) we investigated theoretical mechanisms behind the augmentation index. Our database of virtual subjects is a new tool to assist the clinician: it provides insight into the physical mechanisms underlying the correlations observed in clinical practice. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

  8. Drug Target Validation Methods in Malaria - Protein Interference Assay (PIA) as a Tool for Highly Specific Drug Target Validation.

    PubMed

    Meissner, Kamila A; Lunev, Sergey; Wang, Yuan-Ze; Linzke, Marleen; de Assis Batista, Fernando; Wrenger, Carsten; Groves, Matthew R

    2017-01-01

    The validation of drug targets in malaria and other human diseases remains a highly difficult and laborious process. In the vast majority of cases, highly specific small molecule tools to inhibit a proteins function in vivo are simply not available. Additionally, the use of genetic tools in the analysis of malarial pathways is challenging. These issues result in difficulties in specifically modulating a hypothetical drug target's function in vivo. The current "toolbox" of various methods and techniques to identify a protein's function in vivo remains very limited and there is a pressing need for expansion. New approaches are urgently required to support target validation in the drug discovery process. Oligomerisation is the natural assembly of multiple copies of a single protein into one object and this self-assembly is present in more than half of all protein structures. Thus, oligomerisation plays a central role in the generation of functional biomolecules. A key feature of oligomerisation is that the oligomeric interfaces between the individual parts of the final assembly are highly specific. However, these interfaces have not yet been systematically explored or exploited to dissect biochemical pathways in vivo. This mini review will describe the current state of the antimalarial toolset as well as the potentially druggable malarial pathways. A specific focus is drawn to the initial efforts to exploit oligomerisation surfaces in drug target validation. As alternative to the conventional methods, Protein Interference Assay (PIA) can be used for specific distortion of the target protein function and pathway assessment in vivo. Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.org.

  9. A Novel Tool for Assessment of Emergency Medicine Resident Skill in Determining Diagnosis and Management for Emergent Electrocardiograms: A Multicenter Study.

    PubMed

    Hartman, Nicholas D; Wheaton, Natasha B; Williamson, Kelly; Quattromani, Erin N; Branzetti, Jeremy B; Aldeen, Amer Z

    2016-12-01

    Reading emergent electrocardiograms (ECGs) is one of the emergency physician's most crucial tasks, yet no well-validated tool exists to measure resident competence in this skill. To assess validity of a novel tool measuring emergency medicine resident competency for interpreting, and responding to, critical ECGs. In addition, we aim to observe trends in this skill for resident physicians at different levels of training. This is a multi-center, prospective study of postgraduate year (PGY) 1-4 residents at five emergency medicine (EM) residency programs in the United States. An assessment tool was created that asks the physician to identify either the ECG diagnosis or the best immediate management. One hundred thirteen EM residents from five EM residency programs submitted completed assessment surveys, including 43 PGY-1s, 33 PGY-2s, and 37 PGY-3/4s. PGY-3/4s averaged 74.6% correct (95% confidence interval [CI] 70.9-78.4) and performed significantly better than PGY-1s, who averaged 63.2% correct (95% CI 58.0-68.3). PGY-2s averaged 69.0% (95% CI 62.2-73.7). Year-to-year differences were more pronounced in management than in diagnosis. Residency training in EM seems to be associated with improved ability to interpret "critical" ECGs as measured by our assessment tool. This lends validity evidence for the tool by correlating with a previously observed association between residency training and improved ECG interpretation. Resident skill in ECG interpretation remains less than ideal. Creation of this sort of tool may allow programs to assess resident performance as well as evaluate interventions designed to improve competency. Copyright © 2016 Elsevier Inc. All rights reserved.

  10. A novel cell culture model as a tool for forensic biology experiments and validations.

    PubMed

    Feine, Ilan; Shpitzen, Moshe; Roth, Jonathan; Gafny, Ron

    2016-09-01

    To improve and advance DNA forensic casework investigation outcomes, extensive field and laboratory experiments are carried out in a broad range of relevant branches, such as touch and trace DNA, secondary DNA transfer and contamination confinement. Moreover, the development of new forensic tools, for example new sampling appliances, by commercial companies requires ongoing validation and assessment by forensic scientists. A frequent challenge in these kinds of experiments and validations is the lack of a stable, reproducible and flexible biological reference material. As a possible solution, we present here a cell culture model based on skin-derived human dermal fibroblasts. Cultured cells were harvested, quantified and dried on glass slides. These slides were used in adhesive tape-lifting experiments and tests of DNA crossover confinement by UV irradiation. The use of this model enabled a simple and concise comparison between four adhesive tapes, as well as a straightforward demonstration of the effect of UV irradiation intensities on DNA quantity and degradation. In conclusion, we believe this model has great potential to serve as an efficient research tool in forensic biology. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

  11. Adaptation and cross-cultural validation of the United States Primary Care Assessment Tool (expanded version) for use in South Africa.

    PubMed

    Bresick, Graham; Sayed, Abdul-Rauf; le Grange, Cynthia; Bhagwan, Susheela; Manga, Nayna

    2015-06-19

    Measuring primary care is important for health sector reform. The Primary Care Assessment Tool (PCAT) measures performance of elements essential for cost-effective care. Following minor adaptations prior to use in Cape Town in 2011, a few findings indicated a need to improve the content and cross-cultural validity for wider use in South Africa (SA). This study aimed to validate the United States of America-developed PCAT before being used in a baseline measure of primary care performance prior to major reform. Public sector primary care clinics, users, practitioners and managers in urban and rural districts in the Western Cape Province. Face value evaluation of item phrasing and a combination of Delphi and Nominal Group Technique (NGT) methods with an expert panel and user focus group were used to obtain consensus on content relevant to SA. Original and new domains and items with > = 70% agreement were included in the South African version--ZA PCAT. All original PCAT domains achieved consensus on inclusion. One new domain, the primary healthcare (PHC) team, was added. Three of 95 original items achieved < 70% agreement, that is consensus to exclude as not relevant to SA; 19 new items were added. A few items needed minor rephrasing with local healthcare jargon. The demographic section was adapted to local socio-economic conditions. The adult PCAT was translated into isiXhosa and Afrikaans. The PCAT is a valid measure of primary care performance in SA. The PHC team domain is an important addition, given its emphasis in PHC re-engineering. A combination of Delphi and NGT methods succeeded in obtaining consensus on a multi-domain, multi-item instrument in a resource-constrained environment.

  12. The Predictive Validity of the Assessment of Basic Learning Abilities versus Parents' Predictions with Children with Autism

    ERIC Educational Resources Information Center

    Murphy, Colleen; Martin, Garry L.; Yu, C. T.

    2014-01-01

    The Assessment of Basic Learning Abilities (ABLA) is an empirically validated clinical tool for assessing the learning ability of persons with intellectual disabilities and children with autism. An ABLA tester uses standardized prompting and reinforcement procedures to attempt to teach, individually, each of six tasks, called levels, to a testee,…

  13. Content validation using an expert panel: assessment process for assistive technology adopted by farmers with disabilities.

    PubMed

    Mathew, S N; Field, W E; French, B F

    2011-07-01

    This article reports the use of an expert panel to perform content validation of an experimental assessment process for the safety of assistive technology (AT) adopted by farmers with disabilities. The validation process was conducted by a panel of six experts experienced in the subject matter, i.e., design, use, and assessment of AT for farmers with disabilities. The exercise included an evaluation session and two focus group sessions. The evaluation session consisted of using the assessment process under consideration by the panel to evaluate a set of nine ATs fabricated by a farmer on his farm site. The expert panel also participated in the focus group sessions conducted immediately before and after the evaluation session. The resulting data were analyzed using discursive analysis, and the results were incorporated into the final assessment process. The method and the results are presented with recommendations for the use of expert panels in research projects and validation of assessment tools.

  14. Development of the Hand Assessment for Infants: evidence of internal scale validity.

    PubMed

    Krumlinde-Sundholm, Lena; Ek, Linda; Sicola, Elisa; Sjöstrand, Lena; Guzzetta, Andrea; Sgandurra, Giuseppina; Cioni, Giovanni; Eliasson, Ann-Christin

    2017-12-01

    The aim of this study was to develop a descriptive and evaluative assessment of upper limb function for infants aged 3 to 12 months and to investigate its internal scale validity for use with infants at risk of unilateral cerebral palsy. The concepts of the test items and scoring criteria were developed. Internal scale validity and aspects of reliability were investigated on the basis of 156 assessments of infants at 3 to 12 months corrected age (mean 7.2mo, SD 2.5) with signs of asymmetric hand use. Rasch measurement model analysis and non-parametric statistics were used. The new test, the Hand Assessment for Infants (HAI), consists of 12 unimanual and five bimanual items, each scored on a 3-point rating scale. It demonstrated a unidimensional construct and good fit to the Rasch model requirements. The excellent person reliability enabled person separation to six significant ability strata. The HAI produced an interval-level measure of bilateral hand use as well as unimanual scores of each hand, allowing a quantification of possible asymmetry expressed as an asymmetry index. The HAI can be considered a valid assessment tool for measuring bilateral hand use and quantifying side difference between hands among infants at risk of developing unilateral cerebral palsy. The Hand Assessment for Infants (HAI) measures the use of both hands and quantifies a possible asymmetry of hand use. HAI is valid for infants at 3 to 12 months corrected age at risk of unilateral cerebral palsy. © 2017 Mac Keith Press.

  15. Field Validation of the Los Angeles Motor Scale as a Tool for Paramedic Assessment of Stroke Severity.

    PubMed

    Kim, Joon-Tae; Chung, Pil-Wook; Starkman, Sidney; Sanossian, Nerses; Stratton, Samuel J; Eckstein, Marc; Pratt, Frank D; Conwit, Robin; Liebeskind, David S; Sharma, Latisha; Restrepo, Lucas; Tenser, May-Kim; Valdes-Sueiras, Miguel; Gornbein, Jeffrey; Hamilton, Scott; Saver, Jeffrey L

    2017-02-01

    The Los Angeles Motor Scale (LAMS) is a 3-item, 0- to 10-point motor stroke-deficit scale developed for prehospital use. We assessed the convergent, divergent, and predictive validity of the LAMS when performed by paramedics in the field at multiple sites in a large and diverse geographic region. We analyzed early assessment and outcome data prospectively gathered in the FAST-MAG trial (Field Administration of Stroke Therapy-Magnesium phase 3) among patients with acute cerebrovascular disease (cerebral ischemia and intracranial hemorrhage) within 2 hours of onset, transported by 315 ambulances to 60 receiving hospitals. Among 1632 acute cerebrovascular disease patients (age 70±13 years, male 57.5%), time from onset to prehospital LAMS was median 30 minutes (interquartile range 20-50), onset to early postarrival (EPA) LAMS was 145 minutes (interquartile range 119-180), and onset to EPA National Institutes of Health Stroke Scale was 150 minutes (interquartile range 120-180). Between the prehospital and EPA assessments, LAMS scores were stable in 40.5%, improved in 37.6%, and worsened in 21.9%. In tests of convergent validity, against the EPA National Institutes of Health Stroke Scale, correlations were r=0.49 for the prehospital LAMS and r=0.89 for the EPA LAMS. Prehospital LAMS scores did diverge from the prehospital Glasgow Coma Scale, r=-0.22. Predictive accuracy (adjusted C statistics) for nondisabled 3-month outcome was as follows: prehospital LAMS, 0.76 (95% confidence interval 0.74-0.78); EPA LAMS, 0.85 (95% confidence interval 0.83-0.87); and EPA National Institutes of Health Stroke Scale, 0.87 (95% confidence interval 0.85-0.88). In this multicenter, prospective, prehospital study, the LAMS showed good to excellent convergent, divergent, and predictive validity, further establishing it as a validated instrument to characterize stroke severity in the field. © 2017 American Heart Association, Inc.

  16. Validation Analysis of a Geriatric Dehydration Screening Tool in Community-Dwelling and Institutionalized Elderly People

    PubMed Central

    Rodrigues, Susana; Silva, Joana; Severo, Milton; Inácio, Cátia; Padrão, Patrícia; Lopes, Carla; Carvalho, Joana; do Carmo, Isabel; Moreira, Pedro

    2015-01-01

    Dehydration is common among elderly people. The aim of this study was to perform validation analysis of a geriatric dehydration-screening tool (DST) in the assessment of hydration status in elderly people. This tool was based on the DST proposed by Vivanti et al., which is composed by 11 items (four physical signs of dehydration and seven questions about thirst sensation, pain and mobility), with four questions extra about drinking habits. The resulting questionnaire was evaluated in a convenience sample comprising institutionalized (n = 29) and community-dwelling (n = 74) elderly people. Urinary parameters were assessed (24-h urine osmolality and volume) and free water reserve (FWR) was calculated. Exploratory factor analysis was used to evaluate the scale’s dimensionality and Cronbach’s alpha was used to measure the reliability of each subscale. Construct’s validity was tested using linear regression to estimate the association between scores in each dimension and urinary parameters. Two factors emerged from factor analysis, which were named “Hydration Score” and “Pain Score”, and both subscales showed acceptable reliabilities. The “Hydration Score” was negatively associated with 24-h urine osmolality in community-dwelling; and the “Pain Score” was negatively associated with 24-h urine osmolality, and positively associated with 24-h urine volume and FWR in institutionalized elderly people. PMID:25739005

  17. Validation of a modified FRAX® tool for improving outpatient efficiency--part of the "Catch Before a Fall" initiative.

    PubMed

    Parker, Simon; Ciaccio, Maria; Cook, Erica; Davenport, Graham; Cooper, Alun; Grange, Simon; Smitham, Peter

    2015-01-01

    We have validated our touch-screen-modified FRAX® tool against the traditional healthcare professional-led questionnaire, demonstrating strong concordance between doctor- and patient-derived results. We will use this in outpatient clinics and general practice to increase our capture rate of at-risk patients, making valuable use of otherwise wasted patient waiting times. Outpatient clinics offer an opportunity to collect valuable health information from a captive population. We have previously developed a modified fracture risk assessment (FRAX®) tool, enabling patients to self-assess their osteoporotic fracture risk in a touch-screen computer format and demonstrated its acceptability with patients. We aim to validate the accuracy of our tool against the traditional questionnaire. Fifty patients over 50 years of age within the fracture clinic independently completed a paper equivalent of our touch-screen-modified FRAX® questionnaire. Responses were analysed against the traditional healthcare professional (HCP)-led questionnaire which was carried out afterwards. Correlation was assessed by sensitivity, specificity, Cohen's kappa statistic and Fisher's exact test for each potential FRAX® outcome of "treat", "measure BMD" and "lifestyle advice". Age range was 51-98 years. The FRAX® tool was completed by 88 % of patients; six patients lacked confidence in estimating either their height or weight. Following question adjustment according to patient response and feedback, our tool achieved >95 % sensitivity and specificity for the "treat" and "lifestyle advice" groups, and 79 % sensitivity and 100 % specificity in the "measure BMD" group. Cohen's kappa value ranged from 0.823 to 0.995 across all groups, demonstrating "very good" agreement for all. Fisher's exact test demonstrated significant concordance between doctor and patient decisions. Our modified tool provides a simple, accurate and reliable method for patients to self-report their own FRAX® score

  18. Assessing Face Validity of a Food Behavior Checklist for Limited-resource Filipinos

    PubMed Central

    Buchthal, Opal Vanessa; Tauyan, Socorro

    2015-01-01

    Abstract Diet-related chronic health conditions are prevalent in the Filipino American community; however, there is a lack of rigorously validated nutrition education evaluation tools in Tagalog for use in this population. This study aimed to develop and evaluate the face validity of a Tagalog-language food behavior checklist (FBC). A multi-step method was used, involving translation of questionnaire text from English to Tagalog by a team of professionals, creation of accompanying color photographs, cognitive testing with the target population, final review by the team of professionals, and assessment of readability. Subjects for cognitive testing were men (n=6) and women (n=14) 18 years or older in Hawai‘i who received or were eligible to receive Supplemental Nutrition Assistance Program (SNAP) benefits, self-identified as Filipino, and preferred Tagalog rather than English. Participants were recruited from churches, the Filipino Center, and other community sites. Cognitive interviews revealed several issues with text and photographs, such as preferences for specific terms, and images that did not adequately illustrate the text. Image changes were made to reflect items most commonly consumed. The team of professionals agreed with participant suggestions. Assessment of readability revealed a reading level appropriate for a low-literacy population of grade 5.9. The multi-step process, which allowed members of the target audience to reveal the appropriateness of the questionnaire, yielded a Tagalog-language FBC found to have adequate face validity. After further evaluation of validity and reliability, this tool may be used to evaluate behavior change resulting from the United States Department of Agriculture's (USDA) nutrition education programs. PMID:26535163

  19. The development and validation of a carer questionnaire to assess cognitive function in neuropsychiatric patients.

    PubMed

    Randhawa, Sharan; Walterfang, Mark; Miller, Kathryn; Scholes, Amelia; Mocellin, Ramon; Velakoulis, Dennis

    2007-07-01

    The carer history is an integral part of the assessment of patients with cognitive impairment. We aimed to develop a comprehensive yet concise carer questionnaire, the CogRisk, which captures actuarial risk variables for cognitive impairment in addition to key symptoms suggestive of cognitive decline in a number of cognitive domains, and to then assess its validity and reliability in a neuropsychiatric population. Carers of patients assessed for cognitive impairment completed the CogRisk, and patients were clinically assessed using the Mini-Mental State Examination (MMSE) and Neuropsychiatry Unit COGnitive assessment tool (NUCOG). Reliability was assessed using test-retest and interrater measures and measures of internal consistency. Construct and concurrent validity was assessed using correlation between total and subscale scores on the CogRisk, total scores on the NUCOG and MMSE, and subscale scores on the NUCOG. Predictive validity was determined using measures of sensitivity and specificity and using receiver operating characteristic (ROC) methods. The CogRisk was completed by all carers in less than 10 min. The total CogRisk score correlated significantly with total MMSE and NUCOG scores (r=-0.511 and -0.563, respectively) and remained highly significant when age and education were controlled for. Internal consistency of CogRisk items was high (alpha=0.943). Intrarater reliability of the CogRisk was high with an intraclass correlation coefficient of .978 (P<.001), and interrater reliability between carers was also high at 0.868 (P<.05). Sensitivity and specificity for the detection of dementia were .70 and .73, respectively, with area under the ROC curve not significantly different from that of the MMSE or NUCOG. The CogRisk is a brief carer-rated tool of a patient's cognitive functioning developed for use within a neuropsychiatric setting. It exhibited good concurrent validity, internal consistency, and interrater and intrarater reliability. The CogRisk also

  20. 33 CFR 101.510 - Assessment tools.

    Code of Federal Regulations, 2010 CFR

    2010-07-01

    ... may include: (a) DHS/TSA's vulnerability self-assessment tool located at http://www.tsa.gov/risk; and (b) USCG assessment tools, available from the cognizant COTP or at http://www.uscg.mil/hq/g-m/nvic...

  1. Development and initial validation of an instrument to assess stressors among South African sports coaches.

    PubMed

    Kubayi, Alliance; Toriola, Abel; Didymus, Faye

    2018-06-01

    The aim of this series of studies was to develop and initially validate an instrument to assess stressors among South African sports coaches. In study one, a preliminary pool of 45 items was developed based on existing literature and an expert panel was employed to assess the content validity and applicability of these items. In study two, the 32 items that were retained after study one were analysed using principal component analysis (PCA). The resultant factorial structure comprised four components: environmental stressors, performance stressors, task-related stressors, and athlete stressors. These four components were made up of 26 items and, together, the components and items comprised the provisional Stressors in Sports Coaching Questionnaire (SSCQ). The results show that the SSCQ demonstrates acceptable internal consistency (.73-.89). The findings provide preliminary evidence that SSCQ is a valid tool to assess stressors among South African sports coaches.

  2. Puzzle test: A tool for non-analytical clinical reasoning assessment.

    PubMed

    Monajemi, Alireza; Yaghmaei, Minoo

    2016-01-01

    Most contemporary clinical reasoning tests typically assess non-automatic thinking. Therefore, a test is needed to measure automatic reasoning or pattern recognition, which has been largely neglected in clinical reasoning tests. The Puzzle Test (PT) is dedicated to assess automatic clinical reasoning in routine situations. This test has been introduced first in 2009 by Monajemi et al in the Olympiad for Medical Sciences Students.PT is an item format that has gained acceptance in medical education, but no detailed guidelines exist for this test's format, construction and scoring. In this article, a format is described and the steps to prepare and administer valid and reliable PTs are presented. PT examines a specific clinical reasoning task: Pattern recognition. PT does not replace other clinical reasoning assessment tools. However, it complements them in strategies for assessing comprehensive clinical reasoning.

  3. Structural Validation of the Holistic Wellness Assessment

    ERIC Educational Resources Information Center

    Brown, Charlene; Applegate, E. Brooks; Yildiz, Mustafa

    2015-01-01

    The Holistic Wellness Assessment (HWA) is a relatively new assessment instrument based on an emergent transdisciplinary model of wellness. This study validated the factor structure identified via exploratory factor analysis (EFA), assessed test-retest reliability, and investigated concurrent validity of the HWA in three separate samples. The…

  4. Validity of Montreal Cognitive Assessment in non-english speaking patients with Parkinson's disease.

    PubMed

    Krishnan, Syam; Justus, Sunitha; Meluveettil, Radhamani; Menon, Ramshekhar N; Sarma, Sankara P; Kishore, Asha

    2015-01-01

    The Montreal Cognitive Assessment is a brief and easy screening tool for accurately testing cognitive dysfunction in Parkinson's disease. We tested its validity for use in non-English (Malayalam) speaking patients with Parkinson's disease. We developed a Malayalam (a south-Indian language) version of Montreal Cognitive Assessment and applied to 70 patients with Parkinson's disease and 60 age- and education-matched healthy controls. Metric properties were assessed, and the scores were compared with the performance in validated Malayalam versions of Mini Mental Status Examination and Addenbrooke's Cognitive Examination. The Montreal Cognitive Assessment-Malayalam showed good internal consistency and test-retest reliability and its scores correlated with Mini Mental Status Examination (patients: R = 0.70; P < 0.001; healthy controls: R = 0.26; P = 0.04) and Addenbrooke's Cognitive Examination (patients: R = 0.8; P < 0.001; healthy controls: R = 0.52; P < 0.001) scores. This study establishes the reliability of cross-cultural adaptation of Montreal Cognitive Assessment for assessing cognition in Malayalam-speaking Parkinson's disease patients for early screening and potential future interventions for cognitive dysfunction.

  5. Validity of the Microsoft Kinect for assessment of postural control.

    PubMed

    Clark, Ross A; Pua, Yong-Hao; Fortin, Karine; Ritchie, Callan; Webster, Kate E; Denehy, Linda; Bryant, Adam L

    2012-07-01

    Clinically feasible methods of assessing postural control such as timed standing balance and functional reach tests provide important information, however, they cannot accurately quantify specific postural control mechanisms. The Microsoft Kinect™ system provides real-time anatomical landmark position data in three dimensions (3D), and given that it is inexpensive, portable and simple to setup it may bridge this gap. This study assessed the concurrent validity of the Microsoft Kinect™ against a benchmark reference, a multiple-camera 3D motion analysis system, in 20 healthy subjects during three postural control tests: (i) forward reach, (ii) lateral reach, and (iii) single-leg eyes-closed standing balance. For the reach tests, the outcome measures consisted of distance reached and trunk flexion angle in the sagittal (forward reach) and coronal (lateral reach) planes. For the standing balance test the range and deviation of movement in the anatomical landmark positions for the sternum, pelvis, knee and ankle and the lateral and anterior trunk flexion angle were assessed. The Microsoft Kinect™ and 3D motion analysis systems had comparable inter-trial reliability (ICC difference=0.06±0.05; range, 0.00-0.16) and excellent concurrent validity, with Pearson's r-values >0.90 for the majority of measurements (r=0.96±0.04; range, 0.84-0.99). However, ordinary least products analyses demonstrated proportional biases for some outcome measures associated with the pelvis and sternum. These findings suggest that the Microsoft Kinect™ can validly assess kinematic strategies of postural control. Given the potential benefits it could therefore become a useful tool for assessing postural control in the clinical setting. Copyright © 2012 Elsevier B.V. All rights reserved.

  6. Validation of the German version of the Clinical Assessment Interview for Negative Symptoms (CAINS).

    PubMed

    Engel, Maike; Fritzsche, Anja; Lincoln, Tania Marie

    2014-12-15

    Validated assessment instruments could contribute to a better understanding and assessment of negative symptoms and advance treatment research. The aim of this study was to examine the psychometric properties of a German version of the Clinical Assessment Interview for Negative Symptoms (CAINS). In- and outpatients (N=53) with schizophrenia or schizoaffective disorder were assessed with standardized interviews and questionnaires on negative and positive symptoms and general psychopathology in schizophrenia, depression, the ability to experience anticipatory and consummatory pleasure, and global functioning. The results indicated good psychometric properties, high internal consistency and promising inter-rater agreement for the German version of the CAINS. The two-factor solution of the original version of the CAINS was confirmed, indicating good construct validity. Convergent validity was supported by significant correlations between the CAINS subscales with the negative symptom scale of the Positive and Negative Syndrome Scale, and with consummatory pleasure. The CAINS also exhibited discriminant validity indicated by its non-significant correlations with positive symptoms, general psychopathology and depression that are in line with the findings for the original version of the CAINS. In addition, the CAINS correlated moderately with global functioning. The German version of the CAINS appears to be a valid and suitable diagnostic tool for measuring negative symptoms in schizophrenia. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

  7. Development, Validation and Deployment of a Real Time 30 Day Hospital Readmission Risk Assessment Tool in the Maine Healthcare Information Exchange.

    PubMed

    Hao, Shiying; Wang, Yue; Jin, Bo; Shin, Andrew Young; Zhu, Chunqing; Huang, Min; Zheng, Le; Luo, Jin; Hu, Zhongkai; Fu, Changlin; Dai, Dorothy; Wang, Yicheng; Culver, Devore S; Alfreds, Shaun T; Rogow, Todd; Stearns, Frank; Sylvester, Karl G; Widen, Eric; Ling, Xuefeng B

    2015-01-01

    Identifying patients at risk of a 30-day readmission can help providers design interventions, and provide targeted care to improve clinical effectiveness. This study developed a risk model to predict a 30-day inpatient hospital readmission for patients in Maine, across all payers, all diseases and all demographic groups. Our objective was to develop a model to determine the risk for inpatient hospital readmission within 30 days post discharge. All patients within the Maine Health Information Exchange (HIE) system were included. The model was retrospectively developed on inpatient encounters between January 1, 2012 to December 31, 2012 from 24 randomly chosen hospitals, and then prospectively validated on inpatient encounters from January 1, 2013 to December 31, 2013 using all HIE patients. A risk assessment tool partitioned the entire HIE population into subgroups that corresponded to probability of hospital readmission as determined by a corresponding positive predictive value (PPV). An overall model c-statistic of 0.72 was achieved. The total 30-day readmission rates in low (score of 0-30), intermediate (score of 30-70) and high (score of 70-100) risk groupings were 8.67%, 24.10% and 74.10%, respectively. A time to event analysis revealed the higher risk groups readmitted to a hospital earlier than the lower risk groups. Six high-risk patient subgroup patterns were revealed through unsupervised clustering. Our model was successfully integrated into the statewide HIE to identify patient readmission risk upon admission and daily during hospitalization or for 30 days subsequently, providing daily risk score updates. The risk model was validated as an effective tool for predicting 30-day readmissions for patients across all payer, disease and demographic groups within the Maine HIE. Exposing the key clinical, demographic and utilization profiles driving each patient's risk of readmission score may be useful to providers in developing individualized post discharge

  8. Multifactorial screening for fall risk in community-dwelling older adults in the primary care office: development of the fall risk assessment & screening tool.

    PubMed

    Renfro, Mindy Oxman; Fehrer, Steven

    2011-01-01

    Unintentional falls is an increasing public health problem as incidence of falls rises and the population ages. The Centers for Disease Control and Prevention reports that 1 in 3 adults aged 65 years and older will experience a fall this year; 20% to 30% of those who fall will sustain a moderate to severe injury. Physical therapists caring for older adults are usually engaged with these patients after the first injury fall and may have little opportunity to abate fall risk before the injuries occur. This article describes the content selection and development of a simple-to-administer, multifactorial, Fall Risk Assessment & Screening Tool (FRAST), designed specifically for use in primary care settings to identify those older adults with high fall risk. Fall Risk Assessment & Screening Tool incorporates previously validated measures within a new multifactorial tool and includes targeted recommendations for intervention. Development of the multifactorial FRAST used a 5-part process: identification of significant fall risk factors, review of best evidence, selection of items, creation of the scoring grid, and development of a recommended action plan. Fall Risk Assessment & Screening Tool has been developed to assess fall risk in the target population of older adults (older than 65 years) living and ambulating independently in the community. Many fall risk factors have been considered and 15 items selected for inclusion. Fall Risk Assessment & Screening Tool includes 4 previously validated measures to assess balance, depression, falls efficacy, and home safety. Reliability and validity studies of FRAST are under way. Fall risk for community-dwelling older adults is an urgent, multifactorial, public health problem. Providing primary care practitioners (PCPs) with a very simple screening tool is imperative. Fall Risk Assessment & Screening Tool was created to allow for safe, quick, and low-cost administration by minimally trained office staff with interpretation and

  9. Making and Executing Decisions for Safe and Independent Living (MED-SAIL): development and validation of a brief screening tool.

    PubMed

    Mills, Whitney L; Regev, Tziona; Kunik, Mark E; Wilson, Nancy L; Moye, Jennifer; McCullough, Laurence B; Naik, Aanand D

    2014-03-01

    Older adults prefer to remain in their own homes for as long as possible. The purpose of this article is to describe the development and preliminary validation of Making and Executing Decisions for Safe and Independent Living (MED-SAIL), a brief screening tool for capacity to live safely and independently in the community. Prospective preliminary validation study. Outpatient geriatrics clinic located in a community-based hospital. Forty-nine community-dwelling older adults referred to the clinic for a comprehensive capacity assessment. We examined internal consistency, criterion-based validity, concurrent validity, and accuracy of classification for MED-SAIL. The items included in MED-SAIL demonstrated internal consistency (5 items; α = 0.85). MED-SAIL was significantly correlated with the Independent Living Scales (r = 0.573, p ≤0.001) and instrumental activities of daily living (r = 0.440, p ≤0.01). The Mann-Whitney U test revealed significant differences between the no capacity and partial/full capacity classifications on MED-SAIL (U(48) = 60.5, Z = -0.38, p <0.0001). The area under the curve was 0.864 (95% confidence interval: 0.84-0.99). This study demonstrated the validity of MED-SAIL as a brief screening tool to identify older adults with impaired capacity for remaining safe and independent in their current living environment. MED-SAIL is useful tool for health and social service providers in the community for the purpose of referral for definitive capacity evaluation. Published by Elsevier Inc.

  10. Validation of Assessment Vignettes and Scoring Rubric of Multicultural and International Competency in Faculty Teaching

    ERIC Educational Resources Information Center

    Henderson, Sheila J.; Horton, Ruth A.; Saito, Paul K.; Shorter-Gooden, Kumea

    2016-01-01

    The purpose of this research was to develop a new tool for assessing multicultural and international competency in faculty teaching through vignette scenarios of university classroom critical incidents--across disciplines of clinical and forensics psychology, business, and education. Construct and content validity of the initial draft vignettes…

  11. Development and validation of a fatigue assessment scale for U.S. construction workers.

    PubMed

    Zhang, Mingzong; Sparer, Emily H; Murphy, Lauren A; Dennerlein, Jack T; Fang, Dongping; Katz, Jeffrey N; Caban-Martinez, Alberto J

    2015-02-01

    To develop a fatigue assessment scale and test its reliability and validity for commercial construction workers. Using a two-phased approach, we first identified items (first phase) for the development of a Fatigue Assessment Scale for Construction Workers (FASCW) through review of existing scales in the scientific literature, key informant interviews (n = 11) and focus groups (three groups with six workers each) with construction workers. The second phase included assessment for the reliability, validity, and sensitivity of the new scale using a repeated-measures study design with a convenience sample of construction workers (n = 144). Phase one resulted in a 16-item preliminary scale that after factor analysis yielded a final 10-item scale with two sub-scales ("Lethargy" and "Bodily Ailment"). During phase two, the FASCW and its subscales demonstrated satisfactory internal consistency (alpha coefficients were FASCW [0.91], Lethargy [0.86] and Bodily Ailment [0.84]) and acceptable test-retest reliability (Pearson Correlations Coefficients: 0.59-0.68; Intraclass Correlation Coefficients: 0.74-0.80). Correlation analysis substantiated concurrent and convergent validity. A discriminant analysis demonstrated that the FASCW differentiated between groups with arthritis status and different work hours. The 10-item FASCW with good reliability and validity is an effective tool for assessing the severity of fatigue among construction workers. © 2015 Wiley Periodicals, Inc.

  12. Development and Validation of a Fatigue Assessment Scale for U.S. Construction Workers

    PubMed Central

    Zhang, Mingzong; Sparer, Emily H.; Murphy, Lauren A.; Dennerlein, Jack T.; Fang, Dongping; Katz, Jeffrey N.; Caban-Martinez, Alberto J.

    2015-01-01

    Objective To develop a fatigue assessment scale and test its reliability and validity for commercial construction workers. Methods Using a two-phased approach, we first identified items for the development of a Fatigue Assessment Scale for Construction Workers (FASCW) through review of existing scales in the scientific literature, key informant interviews (n=11) and focus groups (3 groups with 6 workers each) with construction workers. The second phase included assessment for the reliability, validity and sensitivity of the new scale using a repeated-measures study design with a convenience sample of construction workers (n=144). Results Phase one resulted in a 16-item preliminary scale that after factor analysis yielded a final 10-item scale with two sub-scales (“Lethargy” and “Bodily Ailment”).. During phase two, the FASCW and its subscales demonstrated satisfactory internal consistency (alpha coefficients were FASCW (0.91), Lethargy (0.86) and Bodily Ailment (0.84)) and acceptable test-retest reliability (Pearson Correlations Coefficients: 0.59–0.68; Intraclass Correlation Coefficients: 0.74–0.80). Correlation analysis substantiated concurrent and convergent validity. A discriminant analysis demonstrated that the FASCW differentiated between groups with arthritis status and different work hours. Conclusions The 10-item FASCW with good reliability and validity is an effective tool for assessing the severity of fatigue among construction workers. PMID:25603944

  13. Cross-cultural adaptation, reliability, and validity of the Persian version of the Cumberland Ankle Instability Tool.

    PubMed

    Hadadi, Mohammad; Ebrahimi Takamjani, Ismail; Ebrahim Mosavi, Mohammad; Aminian, Gholamreza; Fardipour, Shima; Abbasi, Faeze

    2017-08-01

    The purpose of the present study was to translate and to cross-culturally adapt the Cumberland Ankle Instability Tool (CAIT) into Persian language and to evaluate its psychometric properties. The International Quality of Life Assessment process was pursued to translate CAIT into Persian. Two groups of Persian-speaking individuals, 105 participants with a history of ankle sprain and 30 participants with no history of ankle sprain, were asked to fill out Persian version of CAIT (CAIT-P), Foot and Ankle Ability Measure (FAAM), and Visual Analog Scale (VAS). Data obtained from the first administration of CAIT were used to evaluate floor and ceiling effects, internal consistency, dimensionality, and criterion validity. To determine the test-retest reliability, 45 individuals re-filled CAIT 5-7 days after the first session. Cronbach's alpha was over the cutoff point of 0.70 for both ankles and in both groups. The intra-class correlation coefficient was high for right (0.95) and left (0.91) ankles. There was a strong correlation between each item and the total score of the CAIT-P. Although the CAIT-P had strong correlation with VAS, its correlation with both subscales of FAAM was moderate. The CAIT-P has good validity and reliability and it can be used by clinicians and researchers for identification and investigation of functional ankle instability. Implications for Rehabilitation Chronic ankle instability is one of the most common consequences of acute ankle sprain. Cumberland Ankle Instability Tool is an acceptable measure to determine functional ankle instability and its severity. The Persian version of Cumberland Ankle Instability Tool is a valid and reliable tool for clinical and research purpose in Persian-speaking individuals.

  14. Resource Assessment and Tools

    USDA-ARS?s Scientific Manuscript database

    Various resource assessment strategies and tools are needed to ensure bioenergy feedstock materials are produced, harvested, and transported in a sustainable manner. This presentation highlights research accomplishments by the USDA-ARS Renewable Energy Assessment Project (REAP) team along with our u...

  15. Development and Validation of a Parent Report Measure for Assessing Social-Emotional Competencies of Children and Adolescents

    ERIC Educational Resources Information Center

    Merrell, Kenneth W.; Felver-Gant, Josh C.; Tom, Karalyn M.

    2011-01-01

    Based on the premises that strength-based assessment of children and adolescents is an important emerging area, and that additional tools for this purpose are needed, this study details development and validation efforts on a new strength-based assessment: the Social-Emotional Assets and Resilience Scale, parent form (SEARS-P). Following careful…

  16. The Outcome and Assessment Information Set (OASIS): A Review of Validity and Reliability

    PubMed Central

    O’CONNOR, MELISSA; DAVITT, JOAN K.

    2015-01-01

    The Outcome and Assessment Information Set (OASIS) is the patient-specific, standardized assessment used in Medicare home health care to plan care, determine reimbursement, and measure quality. Since its inception in 1999, there has been debate over the reliability and validity of the OASIS as a research tool and outcome measure. A systematic literature review of English-language articles identified 12 studies published in the last 10 years examining the validity and reliability of the OASIS. Empirical findings indicate the validity and reliability of the OASIS range from low to moderate but vary depending on the item studied. Limitations in the existing research include: nonrepresentative samples; inconsistencies in methods used, items tested, measurement, and statistical procedures; and the changes to the OASIS itself over time. The inconsistencies suggest that these results are tentative at best; additional research is needed to confirm the value of the OASIS for measuring patient outcomes, research, and quality improvement. PMID:23216513

  17. Determining Historical Pesticide Deposition on Cape Cod through Sediment Core Analysis:A Validation of GIS as An Exposure Assessment Tool

    NASA Astrophysics Data System (ADS)

    Feingold, B. J.; Benoit, G.; Rudel, R.

    2006-12-01

    Geographic Information Systems (GIS) has emerged as a powerful tool to assess current and historical exposure to environmental pollutants. GIS aids in the visualization and understanding of associations between exposure to contaminants and disease. This study is an example of the bridge between environmental science and public health and of how new technology such as GIS can be incorporated into these fields to strengthen both the research and the communication of scientific results. It attempts to validate a GIS-based aerial drift model which predicts the residential exposure to and boundaries of historical organochlorine pesticide (OCP) drift from applications on cranberry bogs, tree pest sprayings and others by analytically quantifying the historical pesticide deposition in a transect of lakes radiating from a distinct spray source. This model was previously used to assess historical residential exposure to OCPs in an environmental epidemiological case-control study of breast cancer incidence on Cape Cod, MA, where the incidence rate of the disease is significantly higher than in the rest of the state. The model's validation in this current study is essential to establishing its predictive ability and thus, its further use. Ground truthing of the model was done through the collection and analysis of sediment cores along a transect of five hydrologically independent kettle ponds radiating from a distinct OCP tree-pest spray area. Measurements of OCP concentrations, total carbon and total organic carbon were determined, and dating of the sediments was completed using 210Pb and verified using 137Cs. Each 50-cm core was sliced into 25 2- cm sections for the analyses, creating a fine-scale depositional history in each pond. Information gathered from each core allows for the determination of the extent and degree of dissipation of individual spray events of a known source area and determine how well the model fits the actual data.

  18. The FLIR ONE thermal imager for the assessment of burn wounds: Reliability and validity study.

    PubMed

    Jaspers, M E H; Carrière, M E; Meij-de Vries, A; Klaessens, J H G M; van Zuijlen, P P M

    2017-11-01

    Objective measurement tools may be of great value to provide early and reliable burn wound assessment. Thermal imaging is an easy, accessible and objective technique, which measures skin temperature as an indicator of tissue perfusion. These thermal images might be helpful in the assessment of burn wounds. However, before implementation of a novel measurement tool into clinical practice is considered, it is appropriate to test its clinimetric properties (i.e. reliability and validity). The objective of this study was to assess the reliability and validity of the recently introduced FLIR ONE thermal imager. Two observers obtained thermal images of burn wounds in adult patients at day 1-3, 4-7 and 8-10 after burn. Subsequently, temperature differences between the burn wound and healthy skin (ΔT) were calculated on an iPad mini containing the FLIR Tools app. To assess reliability, ΔT values of both observers were compared by calculating the intraclass correlation coefficient (ICC) and measurement error parameters. To assess validity, the ΔT values of the first observer were compared to the registered healing time of the burn wounds, which was specified into three categories: (I) ≤14 days, (II) 15-21 days and (III) >21 days. The ability of the FLIR ONE to discriminate between healing ≤21 days and >21 days was evaluated by means of a receiver operating characteristic curve and an optimal ΔT cut-off value. Reliability: ICCs were 0.99 for each time point, indicating excellent reliability up to 10 days after burn. The standard error of measurement varied between 0.17-0.22°C. the area under the curve was calculated at 0.69 (95% CI 0.54-0.84). A cut-off value of -1.15°C shows a moderate discrimination between burn wound healing ≤21 days and >21 days (46% sensitivity; 82% specificity). Our results show that the FLIR ONE thermal imager is highly reliable, but the moderate validity calls for additional research. However, the FLIR ONE is pre-eminently feasible

  19. The Irvine, Beatties, and Bresnahan (IBB) Forelimb Recovery Scale: An Assessment of Reliability and Validity

    PubMed Central

    Irvine, Karen-Amanda; Ferguson, Adam R.; Mitchell, Kathleen D.; Beattie, Stephanie B.; Lin, Amity; Stuck, Ellen D.; Huie, J. Russell; Nielson, Jessica L.; Talbott, Jason F.; Inoue, Tomoo; Beattie, Michael S.; Bresnahan, Jacqueline C.

    2014-01-01

    The IBB scale is a recently developed forelimb scale for the assessment of fine control of the forelimb and digits after cervical spinal cord injury [SCI; (1)]. The present paper describes the assessment of inter-rater reliability and face, concurrent and construct validity of this scale following SCI. It demonstrates that the IBB is a reliable and valid scale that is sensitive to severity of SCI and to recovery over time. In addition, the IBB correlates with other outcome measures and is highly predictive of biological measures of tissue pathology. Multivariate analysis using principal component analysis (PCA) demonstrates that the IBB is highly predictive of the syndromic outcome after SCI (2), and is among the best predictors of bio-behavioral function, based on strong construct validity. Altogether, the data suggest that the IBB, especially in concert with other measures, is a reliable and valid tool for assessing neurological deficits in fine motor control of the distal forelimb, and represents a powerful addition to multivariate outcome batteries aimed at documenting recovery of function after cervical SCI in rats. PMID:25071704

  20. Validity and Interrater Reliability of the Visual Quarter-Waste Method for Assessing Food Waste in Middle School and High School Cafeteria Settings.

    PubMed

    Getts, Katherine M; Quinn, Emilee L; Johnson, Donna B; Otten, Jennifer J

    2017-11-01

    Measuring food waste (ie, plate waste) in school cafeterias is an important tool to evaluate the effectiveness of school nutrition policies and interventions aimed at increasing consumption of healthier meals. Visual assessment methods are frequently applied in plate waste studies because they are more convenient than weighing. The visual quarter-waste method has become a common tool in studies of school meal waste and consumption, but previous studies of its validity and reliability have used correlation coefficients, which measure association but not necessarily agreement. The aims of this study were to determine, using a statistic measuring interrater agreement, whether the visual quarter-waste method is valid and reliable for assessing food waste in a school cafeteria setting when compared with the gold standard of weighed plate waste. To evaluate validity, researchers used the visual quarter-waste method and weighed food waste from 748 trays at four middle schools and five high schools in one school district in Washington State during May 2014. To assess interrater reliability, researcher pairs independently assessed 59 of the same trays using the visual quarter-waste method. Both validity and reliability were assessed using a weighted κ coefficient. For validity, as compared with the measured weight, 45% of foods assessed using the visual quarter-waste method were in almost perfect agreement, 42% of foods were in substantial agreement, 10% were in moderate agreement, and 3% were in slight agreement. For interrater reliability between pairs of visual assessors, 46% of foods were in perfect agreement, 31% were in almost perfect agreement, 15% were in substantial agreement, and 8% were in moderate agreement. These results suggest that the visual quarter-waste method is a valid and reliable tool for measuring plate waste in school cafeteria settings. Copyright © 2017 Academy of Nutrition and Dietetics. Published by Elsevier Inc. All rights reserved.

  1. Validity, Reliability and Acceptability of the Team Standardized Assessment of Clinical Encounter Report*

    PubMed Central

    Wong, Camilla L.; Norris, Mireille; Sinha, Samir S.; Zorzitto, Maria L.; Madala, Sushma; Hamid, Jemila S.

    2016-01-01

    Background The Team Standardized Assessment of a Clinical Encounter Report (StACER) was designed for use in Geriatric Medicine residency programs to evaluate Communicator and Collaborator competencies. Methods The Team StACER was completed by two geriatricians and interdisciplinary team members based on observations during a geriatric medicine team meeting. Postgraduate trainees were recruited from July 2010–November 2013. Inter-rater reliability between two geriatricians and between all team members was determined. Internal consistency of items for the constructs Communicator and Collaborator competencies was calculated. Raters completed a survey previously administered to Canadian geriatricians to assess face validity. Trainees completed a survey to determine the usefulness of this instrument as a feedback tool. Results Thirty postgraduate trainees participated. The prevalence-adjusted bias-adjusted kappa range inter-rater reliability for Communicator and Collaborator items were 0.87–1.00 and 0.86–1.00, respectively. The Cronbach’s alpha coefficient for Communicator and Collaborator items was 0.997 (95% CI: 0.993–1.00) and 0.997 (95% CI: 0.997–1.00), respectively. The instrument lacked discriminatory power, as all trainees scored “meets requirements” in the overall assessment. Niney-three per cent and 86% of trainees found feedback useful for developing Communicator and Collaborator competencies, respectively. Conclusions The Team StACER has adequate inter-rater reliability and internal consistency. Poor discriminatory power and face validity challenge the merit of using this evaluation tool. Trainees felt the tool provided useful feedback on Collaborator and Communicator competencies. PMID:28050222

  2. Music Therapy Assessment Tool for Awareness in Disorders of Consciousness (MATADOC): Reliability and Validity of a Measure to Assess Awareness in Patients with Disorders of Consciousness.

    PubMed

    Magee, Wendy L; Siegert, Richard J; Taylor, Steve M; Daveson, Barbara A; Lenton-Smith, Gemma

    2016-01-01

    Prolonged Disorders of Consciousness (PDOC) describes a population where a consciousness disorder has persisted for at least four weeks post injury but is still under investigation. Complex motor, sensory, communication, and cognitive impairments cause challenges with diagnosis, assessment, and intervention planning. Developing sensitive, reliable, and valid measures is a central concern. The auditory modality is the most sensitive for identifying awareness; however, the current standardized behavioral measures fail to provide adequate screening and measurement of auditory responsiveness. The Music Therapy Assessment Tool for Awareness in Disorders of Consciousness (MATADOC) is a recently standardized measure for assessment with PDOC; however, psychometric values for two of its subscales require examination. To determine the measurement characteristics and properties of the MATADOC subscales two and three. In a convenience sample of 21participants with PDOC, a prospective repeated measures study examined inter-rater reliability (IRR) and test-retest reliability (TRR) for both subscales and internal consistency of subscale two. Overall, the items from the MATADOC subscales two and three demonstrated good agreement across and within assessors, with some variability on two identified items. The MATADOC is a standardized measure for assessment of auditory responsiveness in PDOC. Psychometric limitations for the two identified items may have resulted from variations in music therapist clinical experience and training, leading to variations in the administration and interpretation of PDOC patient responses to these two MATADOC assessment items. Although its psychometric properties could be improved, the MATADOC's clinimetric properties make it a valuable assessment to guide clinical work for patients with PDOC. © the American Music Therapy Association 2015. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

  3. Is the presence of a validated malnutrition screening tool associated with better nutritional care in hospitalized patients?

    PubMed

    Eglseer, Doris; Halfens, Ruud J G; Lohrmann, Christa

    2017-05-01

    The aims of this study were to evaluate the association between the use of clinical guidelines and the use of validated screening tools, evaluate the nutritional screening policy in hospitals, and examine the association between the use of validated screening tools and the prevalence of malnutrition and nutritional interventions in hospitalized patients. This was a cross-sectional, multicenter study. Data were collected using a standardized questionnaire on three levels: institution (presence of a guideline for malnutrition), department (use of a validated screening tool), and patient (e.g., malnutrition prevalence). In all, 53 hospitals with 5255 patients participated. About 45% of the hospitals indicated that they have guidelines for malnutrition. Of the departments surveyed, 38.6% used validated screening tools as part of a standard procedure. The nutritional status of 74.5% of the patients was screened during admission, mostly on the basis of clinical observation and patient weight. A validated screening tool was used for 21.2% of the patients. Significant differences between wards with and without validated screening tools were found with regard to malnutrition prevalence (P = 0.002) and the following interventions: referral to a dietitian (P < 0.001), provision of energy-enriched snacks (P = 0.038), adjustment of consistency (food/drinks; P = 0.004), monitoring of the nutritional intake (P = 0.001), and adjustment of the meal ambiance (P < 0.001). Nutritional screening with validated tools in hospitalized patients remains poor. Generally, the nutritional status of patients is screened with unreliable parameters such as clinical observation and body mass index. The results of the present study suggest that the use of validated malnutrition screening tools is associated with better nutritional care and lower malnutrition prevalence rates in hospitalized patients. Copyright © 2017 Elsevier Inc. All rights reserved.

  4. Youth Offender Care Needs Assessment Tool (YO-CNAT): an actuarial risk assessment tool for predicting problematic child-rearing situations in juvenile offenders on the basis of police records.

    PubMed

    van der Put, Claudia E; Stams, Geert Jan J M

    2013-12-01

    In the juvenile justice system, much attention is paid to estimating the risk for recidivism among juvenile offenders. However, it is also important to estimate the risk for problematic child-rearing situations (care needs) in juvenile offenders, because these problems are not always related to recidivism. In the present study, an actuarial care needs assessment tool for juvenile offenders, the Youth Offender Care Needs Assessment Tool (YO-CNAT), was developed to predict the probability of (a) a future supervision order imposed by the child welfare agency, (b) a future entitlement to care indicated by the youth care agency, and (c) future incidents involving child abuse, domestic violence, and/or sexual norm trespassing behavior at the juvenile's address. The YO-CNAT has been developed for use by the police and is based solely on information available in police registration systems. It is designed to assist a police officer without clinical expertise in making a quick assessment of the risk for problematic child-rearing situations. The YO-CNAT was developed on a sample of 1,955 juvenile offenders and was validated on another sample of 2,045 juvenile offenders. The predictive validity (area under the receiver-operating-characteristic curve) scores ranged between .70 (for predicting future entitlement to care) and .75 (for predicting future worrisome incidents at the juvenile's address); therefore, the predictive accuracy of the test scores of the YO-CNAT was sufficient to justify its use as a screening instrument for the police in deciding to refer a juvenile offender to the youth care agency for further assessment into care needs.

  5. Psychometric characterization of the obstetric communication assessment tool for medical education: a pilot study.

    PubMed

    Rodriguez, A Noel; DeWitt, Peter; Fisher, Jennifer; Broadfoot, Kirsten; Hurt, K Joseph

    2016-06-11

    To characterize the psychometric properties of a novel Obstetric Communication Assessment Tool (OCAT) in a pilot study of standardized difficult OB communication scenarios appropriate for undergraduate medical evaluation. We developed and piloted four challenging OB Standardized Patient (SP) scenarios in a sample of twenty-one third year OB/GYN clerkship students: Religious Beliefs (RB), Angry Father (AF), Maternal Smoking (MS), and Intimate Partner Violence (IPV). Five trained Standardized Patient Reviewers (SPRs) independently scored twenty-four randomized video-recorded encounters using the OCAT. Cronbach's alpha and Intraclass Correlation Coefficient-2 (ICC-2) were used to estimate internal consistency (IC) and inter-rater reliability (IRR), respectively. Systematic variation in reviewer scoring was assessed using the Stuart-Maxwell test. IC was acceptable to excellent with Cronbach's alpha values (and 95% Confidence Intervals [CI]): RB 0.91 (0.86, 0.95), AF 0.76 (0.62, 0.87), MS 0.91 (0.86, 0.95), and IPV 0.94 (0.91, 0.97). IRR was unacceptable to poor with ICC-2 values: RB 0.46 (0.40, 0.53), AF 0.48 (0.41, 0.54), MS 0.52 (0.45, 0.58), and IPV 0.67 (0.61, 0.72). Stuart-Maxwell analysis indicated systematic differences in reviewer stringency. Our initial characterization of the OCAT demonstrates important issues in communications assessment. We identify scoring inconsistencies due to differences in SPR rigor that require enhanced training to improve assessment reliability. We outline a rational process for initial communication tool validation that may be useful in undergraduate curriculum development, and acknowledge that rigorous validation of OCAT training and implementation is needed to create a valuable OB communication assessment tool.

  6. Enabling Research Tools for Sustained Climate Assessment

    NASA Technical Reports Server (NTRS)

    Leidner, Allison K.; Bosilovich, Michael G.; Jasinski, Michael F.; Nemani, Ramakrishna R.; Waliser, Duane Edward; Lee, Tsengdar J.

    2016-01-01

    The U.S. Global Change Research Program Sustained Assessment process benefits from long-term investments in Earth science research that enable the scientific community to conduct assessment-relevant science. To this end, NASA initiated several research programs over the past five years to support the Earth observation community in developing indicators, datasets, research products, and tools to support ongoing and future National Climate Assessments. These activities complement NASA's ongoing Earth science research programs. One aspect of the assessment portfolio funds four "enabling tools" projects at NASA research centers. Each tool leverages existing capacity within the center, but has developed tailored applications and products for National Climate Assessments. The four projects build on the capabilities of a global atmospheric reanalysis (MERRA-2), a continental U.S. land surface reanalysis (NCA-LDAS), the NASA Earth Exchange (NEX), and a Regional Climate Model Evaluation System (RCMES). Here, we provide a brief overview of each enabling tool, highlighting the ways in which it has advanced assessment science to date. We also discuss how the assessment community can access and utilize these tools for National Climate Assessments and other sustained assessment activities.

  7. PRA (Probabilistic Risk Assessments) Participation versus Validation

    NASA Technical Reports Server (NTRS)

    DeMott, Diana; Banke, Richard

    2013-01-01

    Probabilistic Risk Assessments (PRAs) are performed for projects or programs where the consequences of failure are highly undesirable. PRAs primarily address the level of risk those projects or programs posed during operations. PRAs are often developed after the design has been completed. Design and operational details used to develop models include approved and accepted design information regarding equipment, components, systems and failure data. This methodology basically validates the risk parameters of the project or system design. For high risk or high dollar projects, using PRA methodologies during the design process provides new opportunities to influence the design early in the project life cycle to identify, eliminate or mitigate potential risks. Identifying risk drivers before the design has been set allows the design engineers to understand the inherent risk of their current design and consider potential risk mitigation changes. This can become an iterative process where the PRA model can be used to determine if the mitigation technique is effective in reducing risk. This can result in more efficient and cost effective design changes. PRA methodology can be used to assess the risk of design alternatives and can demonstrate how major design changes or program modifications impact the overall program or project risk. PRA has been used for the last two decades to validate risk predictions and acceptability. Providing risk information which can positively influence final system and equipment design the PRA tool can also participate in design development, providing a safe and cost effective product.

  8. Development and Validation of the Life Sciences Assessment: A Measure of Preschool Children's Conceptions of Basic Life Sciences

    ERIC Educational Resources Information Center

    Maherally, Uzma Nooreen

    2014-01-01

    The purpose of this study was to develop and validate a science assessment tool termed the Life Sciences Assessment (LSA) in order to assess preschool children's conceptions of basic life sciences. The hypothesis was that the four sub-constructs, each of which can be measured through a series of questions on the LSA, will make a significant…

  9. The Development and Validation of the Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia

    PubMed Central

    Wong, Kok Thong; Low, Bee Yean

    2015-01-01

    Objectives To develop and validate Osteoporosis Prevention and Awareness Tool (OPAAT) in Malaysia. Methods The OPAAT was modified from the Malaysian Osteoporosis Knowledge Tool and developed from an exploratory study on patients. Face and content validity was established by an expert panel. The OPAAT consists of 30 items, categorized into three domains. A higher score indicates higher knowledge level. English speaking non-osteoporotic postmenopausal women ≥50 years of age and pharmacists were included in the study. Results A total of 203 patients and 31 pharmacists were recruited. Factor analysis extracted three domains. Flesch reading ease was 59.2. The mean±SD accuracy rate was 0.60±0.22 (range: 0.26-0.94). The Cronbach’s α for each domain ranged from 0.286-0.748. All items were highly correlated (Spearman’s rho: 0.761-0.990, p<0.05), with no significant change in the overall test-retest scores, indicating that OPAAT has achieved stable reliability. Pharmacists had higher knowledge score than patients (80.9±8.7vs63.6±17.4, p<0.001), indicating that the OPAAT was able to discriminate between the knowledge levels of pharmacists and patients. Conclusion The OPAAT was found to be a valid and reliable instrument for assessing patient’s knowledge about osteoporosis and its prevention in Malaysia. The OPAAT can be used to identify individuals in need of osteoporosis educational intervention. PMID:25938494

  10. Competency-Based Training and Simulation: Making a "Valid" Argument.

    PubMed

    Noureldin, Yasser A; Lee, Jason Y; McDougall, Elspeth M; Sweet, Robert M

    2018-02-01

    The use of simulation as an assessment tool is much more controversial than is its utility as an educational tool. However, without valid simulation-based assessment tools, the ability to objectively assess technical skill competencies in a competency-based medical education framework will remain challenging. The current literature in urologic simulation-based training and assessment uses a definition and framework of validity that is now outdated. This is probably due to the absence of awareness rather than an absence of comprehension. The following review article provides the urologic community an updated taxonomy on validity theory as it relates to simulation-based training and assessments and translates our simulation literature to date into this framework. While the old taxonomy considered validity as distinct subcategories and focused on the simulator itself, the modern taxonomy, for which we translate the literature evidence, considers validity as a unitary construct with a focus on interpretation of simulator data/scores.

  11. Automated Writing Evaluation for Formative Assessment of Second Language Writing: Investigating the Accuracy and Usefulness of Feedback as Part of Argument-Based Validation

    ERIC Educational Resources Information Center

    Ranalli, Jim; Link, Stephanie; Chukharev-Hudilainen, Evgeny

    2017-01-01

    An increasing number of studies on the use of tools for automated writing evaluation (AWE) in writing classrooms suggest growing interest in their potential for formative assessment. As with all assessments, these applications should be validated in terms of their intended interpretations and uses. A recent argument-based validation framework…

  12. Falls risk assessment begins with hello: lessons learned from the use of one home health agency's fall risk tool.

    PubMed

    Flemming, Patricia J; Ramsay, Katherine

    2012-10-01

    Identifying older adults at risk for falls is a challenge all home healthcare agencies (HHAs) face. The process of assessing for falls risk begins with the initial home visit. One HHA affiliated with an academic medical center describes its experience in development and use of a Falls Risk Assessment (FRA) tool over a 10-year period. The FRA tool has been modified since initial development to clarify elements of the tool based on research and to reflect changes in the Outcome and Assessment Information Set (OASIS) document. The primary purpose of this article is to share a validated falls risk assessment tool to facilitate identification of fall-related risk factors in the homebound population. A secondary purpose is to share lessons learned by the HHA during the 10 years using the FRA.

  13. Assessment of validity with polytrauma Veteran populations.

    PubMed

    Bush, Shane S; Bass, Carmela

    2015-01-01

    Veterans with polytrauma have suffered injuries to multiple body parts and organs systems, including the brain. The injuries can generate a triad of physical, neurologic/cognitive, and emotional symptoms. Accurate diagnosis is essential for the treatment of these conditions and for fair allocation of benefits. To accurately diagnose polytrauma disorders and their related problems, clinicians take into account the validity of reported history and symptoms, as well as clinical presentations. The purpose of this article is to describe the assessment of validity with polytrauma Veteran populations. Review of scholarly and other relevant literature and clinical experience are utilized. A multimethod approach to validity assessment that includes objective, standardized measures increases the confidence that can be placed in the accuracy of self-reported symptoms and physical, cognitive, and emotional test results. Due to the multivariate nature of polytrauma and the multiple disciplines that play a role in diagnosis and treatment, an ideal model of validity assessment with polytrauma Veteran populations utilizes neurocognitive, neurological, neuropsychiatric, and behavioral measures of validity. An overview of these validity assessment approaches as applied to polytrauma Veteran populations is presented. Veterans, the VA, and society are best served when accurate diagnoses are made.

  14. Assessing the efficacy of the Measure of Understanding of Macroevolution as a valid tool for undergraduate non-science majors

    NASA Astrophysics Data System (ADS)

    Romine, William Lee; Walter, Emily Marie

    2014-11-01

    Efficacy of the Measure of Understanding of Macroevolution (MUM) as a measurement tool has been a point of contention among scholars needing a valid measure for knowledge of macroevolution. We explored the structure and construct validity of the MUM using Rasch methodologies in the context of a general education biology course designed with an emphasis on macroevolution content. The Rasch model was utilized to quantify item- and test-level characteristics, including dimensionality, reliability, and fit with the Rasch model. Contrary to previous work, we found that the MUM provides a valid, reliable, and unidimensional scale for measuring knowledge of macroevolution in introductory non-science majors, and that its psychometric behavior does not exhibit large changes across time. While we found that all items provide productive measurement information, several depart substantially from ideal behavior, warranting a collective effort to improve these items. Suggestions for improving the measurement characteristics of the MUM at the item and test levels are put forward and discussed.

  15. Validation of Computerized Automatic Calculation of the Sequential Organ Failure Assessment Score

    PubMed Central

    Harrison, Andrew M.; Pickering, Brian W.; Herasevich, Vitaly

    2013-01-01

    Purpose. To validate the use of a computer program for the automatic calculation of the sequential organ failure assessment (SOFA) score, as compared to the gold standard of manual chart review. Materials and Methods. Adult admissions (age > 18 years) to the medical ICU with a length of stay greater than 24 hours were studied in the setting of an academic tertiary referral center. A retrospective cross-sectional analysis was performed using a derivation cohort to compare automatic calculation of the SOFA score to the gold standard of manual chart review. After critical appraisal of sources of disagreement, another analysis was performed using an independent validation cohort. Then, a prospective observational analysis was performed using an implementation of this computer program in AWARE Dashboard, which is an existing real-time patient EMR system for use in the ICU. Results. Good agreement between the manual and automatic SOFA calculations was observed for both the derivation (N=94) and validation (N=268) cohorts: 0.02 ± 2.33 and 0.29 ± 1.75 points, respectively. These results were validated in AWARE (N=60). Conclusion. This EMR-based automatic tool accurately calculates SOFA scores and can facilitate ICU decisions without the need for manual data collection. This tool can also be employed in a real-time electronic environment. PMID:23936639

  16. Psychometric Properties of the Psychosocial Assessment Tool-Chronic Pain Version in Families of Children With Headache.

    PubMed

    Woods, Kristine; Ostrowski-Delahanty, Sarah

    2017-07-01

    Children with headache disorders are at increased psychosocial risk, and no validated screening measures exist to succinctly assess for risk. This study examined the psychometric properties of the Psychosocial Assessment Tool-Chronic Pain, a previously adapted screening measure of risk, in a retrospective sample of families of children diagnosed with headaches. Participants included 127 children and caregivers presenting for behavioral health evaluation of headache. Children and their primary caregivers completed several psychosocial assessment measures. Internal consistency for the Psychosocial Assessment Tool-Chronic Pain total score was high (α = 0.80), and all subscale scores had moderate to high internal consistency (α = 0.597-0.88), with the exception of the caregiver beliefs subscale (α = 0.443). The total score and the majority of subscale scores on the Psychosocial Assessment Tool-Chronic Pain were correlated with caregiver- and child-reported scores on study measures. The results demonstrate that the Psychosocial Assessment Tool-Chronic Pain has adequate psychometric properties, and because of the brief administration time, ease of scoring, and accessibility of the measure, it is a promising measure of screening for psychosocial risk in this population.

  17. The development of the PARENTS: a tool for parents to assess residents' non-technical skills in pediatric emergency departments.

    PubMed

    Moreau, Katherine A; Eady, Kaylee; Tang, Kenneth; Jabbour, Mona; Frank, Jason R; Campbell, Meaghan; Hamstra, Stanley J

    2017-11-14

    Parents can assess residents' non-technical skills (NTS) in pediatric emergency departments (EDs). There are no assessment tools, with validity evidence, for parental use in pediatric EDs. The purpose of this study was to develop the Parents' Assessment of Residents Enacting Non-Technical Skills (PARENTS) educational assessment tool and collect three sources of validity evidence (i.e., content, response process, internal structure) for it. We established content evidence for the PARENTS through interviews with physician-educators and residents, focus groups with parents, a literature review, and a modified nominal group technique with experts. We collected response process evidence through cognitive interviews with parents. To examine the internal structure evidence, we administered the PARENTS and performed exploratory factor analysis. Initially, a 20-item PARENTS was developed. Cognitive interviews led to the removal of one closed-ended item, the addition of resident photographs, and wording/formatting changes. Thirty-seven residents and 434 parents participated in the administration of the resulting 19-item PARENTS. Following factor analysis, a one-factor model prevailed. The study presents initial validity evidence for the PARENTS. It also highlights strategies for potentially: (a) involving parents in the assessment of residents, (b) improving the assessment of NTS in pediatric EDs, and (c) capturing parents' perspectives to improve the preparation of future physicians.

  18. Validation in Support of Internationally Harmonised OECD Test Guidelines for Assessing the Safety of Chemicals.

    PubMed

    Gourmelon, Anne; Delrue, Nathalie

    Ten years elapsed since the OECD published the Guidance document on the validation and international regulatory acceptance of test methods for hazard assessment. Much experience has been gained since then in validation centres, in countries and at the OECD on a variety of test methods that were subjected to validation studies. This chapter reviews validation principles and highlights common features that appear to be important for further regulatory acceptance across studies. Existing OECD-agreed validation principles will most likely generally remain relevant and applicable to address challenges associated with the validation of future test methods. Some adaptations may be needed to take into account the level of technique introduced in test systems, but demonstration of relevance and reliability will continue to play a central role as pre-requisite for the regulatory acceptance. Demonstration of relevance will become more challenging for test methods that form part of a set of predictive tools and methods, and that do not stand alone. OECD is keen on ensuring that while these concepts evolve, countries can continue to rely on valid methods and harmonised approaches for an efficient testing and assessment of chemicals.

  19. Construction and Validation of a Holistic Education School Evaluation Tool Using Montessori Erdkinder Principles

    ERIC Educational Resources Information Center

    Setari, Anthony Philip

    2016-01-01

    The purpose of this study was to construct a holistic education school evaluation tool using Montessori Erdkinder principles, and begin the validation process of examining the proposed tool. This study addresses a vital need in the holistic education community for a school evaluation tool. The tool construction process included using Erdkinder…

  20. Evacuation performance evaluation tool.

    PubMed

    Farra, Sharon; Miller, Elaine T; Gneuhs, Matthew; Timm, Nathan; Li, Gengxin; Simon, Ashley; Brady, Whittney

    2016-01-01

    Hospitals conduct evacuation exercises to improve performance during emergency events. An essential aspect in this process is the creation of reliable and valid evaluation tools. The objective of this article is to describe the development and implications of a disaster evacuation performance tool that measures one portion of the very complex process of evacuation. Through the application of the Delphi technique and DeVellis's framework, disaster and neonatal experts provided input in developing this performance evaluation tool. Following development, content validity and reliability of this tool were assessed. Large pediatric hospital and medical center in the Midwest. The tool was pilot tested with an administrative, medical, and nursing leadership group and then implemented with a group of 68 healthcare workers during a disaster exercise of a neonatal intensive care unit (NICU). The tool has demonstrated high content validity with a scale validity index of 0.979 and inter-rater reliability G coefficient (0.984, 95% CI: 0.948-0.9952). The Delphi process based on the conceptual framework of DeVellis yielded a psychometrically sound evacuation performance evaluation tool for a NICU.

  1. A protocol for the development of a critical thinking assessment tool for nurses using a Delphi technique.

    PubMed

    Jacob, Elisabeth; Duffield, Christine; Jacob, Darren

    2017-08-01

    The aim of this study was to develop an assessment tool to measure the critical thinking ability of nurses. As an increasing number of complex patients are admitted to hospitals, the importance of nurses recognizing changes in health status and picking up on deterioration is more important. To detect early signs of complication requires critical thinking skills. Registered Nurses are expected to commence their clinical careers with the necessary critical thinking skills to ensure safe nursing practice. Currently, there is no published tool to assess critical thinking skills which is context specific to Australian nurses. A modified Delphi study will be used for the project. This study will develop a series of unfolding case scenarios using national health data with multiple-choice questions to assess critical thinking. Face validity of the scenarios will be determined by an expert reference group of clinical and academic nurses. A Delphi study will determine the answers to scenario questions. Panel members will be expert clinicians and educators from two states in Australia. Rasch analysis of the questionnaire will assess validity and reliability of the tool. Funding for the study and Research Ethics Committee approval were obtained in March and November 2016, respectively. Patient outcomes and safety are directly linked to nurses' critical thinking skills. This study will develop an assessment tool to provide a standardized method of measuring nurses' critical thinking skills across Australia. This will provide healthcare providers with greater confidence in the critical thinking level of graduate Registered Nurses. © 2017 John Wiley & Sons Ltd.

  2. Validity of self-assessment in a quality improvement collaborative in Ecuador.

    PubMed

    Hermida, Jorge; Broughton, Edward I; Miller Franco, Lynne

    2011-12-01

    Health care quality improvement (QI) efforts commonly use self-assessment to measure compliance with quality standards. This study investigates the validity of self-assessment of quality indicators. Cross sectional. A maternal and newborn care improvement collaborative intervention conducted in health facilities in Ecuador in 2005. Four external evaluators were trained in abstracting medical records to calculate six indicators reflecting compliance with treatment standards. About 30 medical records per month were examined at 12 participating health facilities for a total of 1875 records. The same records had already been reviewed by QI teams at these facilities (self-assessment). Overall compliance, agreement (using the Kappa statistic), sensitivity and specificity were analyzed. We also examined patterns of disagreement and the effect of facility characteristics on levels of agreement. External evaluators reported compliance of 69-90%, while self-assessors reported 71-92%, with raw agreement of 71-95% and Kappa statistics ranging from fair to almost perfect agreement. Considering external evaluators as the gold standard, sensitivity of self-assessment ranged from 90 to 99% and specificity from 48 to 86%. Simpler indicators had fewer disagreements. When disagreements occurred between self-assessment and external valuators, the former tended to report more positive findings in five of six indicators, but this tendency was not of a magnitude to change program actions. Team leadership, understanding of the tools and facility size had no overall impact on the level of agreement. When compared with external evaluation (gold standard), self-assessment was found to be sufficiently valid for tracking QI team performance. Sensitivity was generally higher than specificity. Simplifying indicators may improve validity.

  3. Adaptation and cross-cultural validation of the United States Primary Care Assessment Tool (expanded version) for use in South Africa

    PubMed Central

    Sayed, Abdul-Rauf; le Grange, Cynthia; Bhagwan, Susheela; Manga, Nayna

    2015-01-01

    Background Measuring primary care is important for health sector reform. The Primary Care Assessment Tool (PCAT) measures performance of elements essential for cost-effective care. Following minor adaptations prior to use in Cape Town in 2011, a few findings indicated a need to improve the content and cross-cultural validity for wider use in South Africa (SA). Aim This study aimed to validate the United States of America-developed PCAT before being used in a baseline measure of primary care performance prior to major reform. Setting Public sector primary care clinics, users, practitioners and managers in urban and rural districts in the Western Cape Province. Methods Face value evaluation of item phrasing and a combination of Delphi and Nominal Group Technique (NGT) methods with an expert panel and user focus group were used to obtain consensus on content relevant to SA. Original and new domains and items with > = 70% agreement were included in the South African version – ZA PCAT. Results All original PCAT domains achieved consensus on inclusion. One new domain, the primary healthcare (PHC) team, was added. Three of 95 original items achieved < 70% agreement, that is consensus to exclude as not relevant to SA; 19 new items were added. A few items needed minor rephrasing with local healthcare jargon. The demographic section was adapted to local socio-economic conditions. The adult PCAT was translated into isiXhosa and Afrikaans. Conclusion The PCAT is a valid measure of primary care performance in SA. The PHC team domain is an important addition, given its emphasis in PHC re-engineering. A combination of Delphi and NGT methods succeeded in obtaining consensus on a multi-domain, multi-item instrument in a resource- constrained environment. PMID:26245610

  4. Development and validation of the health literacy assessment tool for older people in Taiwan: potential impacts of cultural differences.

    PubMed

    Chung, Min-Huey; Chen, Liang-Kung; Peng, Li-Ning; Chi, Mei-Ju

    2015-01-01

    To screen health literacy among urban elderly in Taiwan, who cannot be evaluated easily using the current measurement tools because of the "face", which meant someone felt embarrassed if he did not know how to do something. A literature review was performed to define a framework for developing the health literacy screening tool. Two hundred elderly were recruited to test the validity and reliability for pilot study. One thousand and eighty two elderly who came from quota sampling in Taipei City by administrative areas and gender were interviewed face-to-face to gather health literacy performance by the developed health literacy screening tool and the short-form Mandarin Health Literacy Scale (s-MHLS). 10-items of health literacy screening tool by self-perception were developed. The mean score of screening tool among analysis sample was 42.3 (0-50) and s-MHLS was 9.5 (0-11). Pearson correlation coefficient was 0.441 (p<0.0001) between these two measurements. Multiple regressions showed that, female, younger, higher education, living with family, has no primary caregiver, has few medical companionship, and higher score of health knowledge had better health literacy performance in both measurements. This screening tool should be applied to screen health literacy of elderly came from baby boomer who usually have lower education levels than the general population in Chinese regions. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

  5. 33 CFR 101.510 - Assessment tools.

    Code of Federal Regulations, 2014 CFR

    2014-07-01

    ... 33 Navigation and Navigable Waters 1 2014-07-01 2014-07-01 false Assessment tools. 101.510 Section 101.510 Navigation and Navigable Waters COAST GUARD, DEPARTMENT OF HOMELAND SECURITY MARITIME SECURITY MARITIME SECURITY: GENERAL Other Provisions § 101.510 Assessment tools. Ports, vessels, and facilities...

  6. Protocol for development and validation of a context-appropriate tool for assessing organisational readiness for change in primary health clinics in South Africa

    PubMed Central

    Sorsdahl, Katherine; Lombard, Carl; Petersen-Williams, Petal; Myers, Bronwyn

    2018-01-01

    Introduction A large treatment gap for common mental disorders (such as depression) exists in South Africa. Comorbidity with other chronic diseases, including HIV and diseases of lifestyle, is an increasing public health concern globally. Currently, primary health facilities as points of care for those with chronic disease provide limited services for common mental disorders. Assessing organisational readiness for change (ORC) towards adopting health innovations (such as mental health services) using contextually appropriate measures is needed to facilitate implementation of these services. This study aims to investigate the validity of the Texas Christian University Organisational Readiness for Change (TCU-ORC) scale in the South African context. Subsequently, we will develop a shortened version of this scale. This study is nested within Project MIND, a multiyear randomised controlled trial that is testing two different approaches for integrating counselling for common mental disorders into chronic disease care. Although the modified, contextually appropriate ORC measure resulting from the proposed study will be developed in the context of integrating mental health into primary healthcare services, the potential for the tool to be generalised to further understanding barriers to any change being implemented in primary care settings is high. Methods and analysis We will establish internal consistency (Cronbach’s alpha coefficients), test-retest reliability (intraclass correlation coefficient) and construct validity of the long-form TCU-ORC questionnaire. Survey data will be collected from 288 clinical, management and operational staff from 24 primary health facilities where the Project MIND trial is implemented. A modified Delphi approach will assess the content validity of the TCU-ORC items and identify areas for potential adaptation and item reduction. Ethics and dissemination Ethical approval has been granted by the South African Medical Research Council

  7. Neurobehavioural assessment and diagnosis in disorders of consciousness: a preliminary study of the Sensory Tool to Assess Responsiveness (STAR).

    PubMed

    Stokes, Verity; Gunn, Sarah; Schouwenaars, Katie; Badwan, Derar

    2018-09-01

    The Sensory Tool to Assess Responsiveness (STAR) is an interdisciplinary neurobehavioural diagnostic tool for individuals with prolonged disorders of consciousness. It utilises current diagnostic criteria and is intended to improve upon the high misdiagnosis rate in this population. This study assesses the inter-rater reliability of the STAR and its diagnostic validity in comparison with the Coma Recovery Scale-Revised (CRS-R) and the Wessex Head Injury Matrix (WHIM). Participants were patients with severe acquired brain injury resulting in a disorder of consciousness, who were admitted to the Royal Leamington Spa Rehabilitation Hospital between 1999 and 2009. Patients underwent sensory stimulation sessions during their period of admission, which were recorded on video. Using this footage, patients were re-assessed for this study using the STAR, WHIM and CRS-R criteria. The STAR demonstrated "moderate" inter-rater reliability, "substantial" diagnostic agreement with the CRS-R, and "moderate" agreement with the WHIM. There were no significant differences between diagnoses assigned by the different assessments. The STAR demonstrated a good degree of inter-rater reliability in identification of diagnoses for patients with disorders of consciousness. The diagnostic outcomes of the STAR agreed at a good level with the CRS-R, moderately with the WHIM, and did not significantly differ from either. This demonstrates the reliability and validity of the STAR, showing its appropriateness for clinical use. Future longitudinal studies and research into the STAR's applicability in long-stay rehabilitation are indicated.

  8. [Psychometric validation in Spanish of the Brazilian short version of the Primary Care Assessment Tools-users questionnaire for the evaluation of the orientation of health systems towards primary care].

    PubMed

    Vázquez Peña, Fernando; Harzheim, Erno; Terrasa, Sergio; Berra, Silvina

    2017-02-01

    To validate the Brazilian short version of the PCAT for adult patients in Spanish. Analysis of secondary data from studies made to validate the extended version of the PCAT questionnaire. City of Córdoba, Argentina. Primary health care. The sample consisted of 46% of parents, whose children were enrolled in secondary education in three institutes in the city of Cordoba, and the remaining 54% were adult users of the National University of Cordoba Health Insurance. Pearson's correlation coefficient comparing the extended and short versions. Goodness-of-fit indices in confirmatory factor analysis, composite reliability, average variance extracted, and Cronbach's alpha values, in order to assess the construct validity and the reliability of the short version. The values of Pearson's correlation coefficient between this short version and the long version were high .818 (P<.001), implying a very good criterion validity. The indicators of good global adjustment to the confirmatory factor analysis were good. The value of composite reliability was good (.802), but under the variance media extracted: .3306, since 3 variables had weak factorials loads. The Cronbach's alpha was acceptable (.85). The short version of the PCAT-users developed in Brazil showed an acceptable psychometric performance in Spanish as a quick assessment tool, in a comparative study with the extended version. Copyright © 2016 Elsevier España, S.L.U. All rights reserved.

  9. A comprehensive clinical assessment tool to inform policy and practice: applications of the minimum data set.

    PubMed

    Mor, Vincent

    2004-04-01

    The Minimum Data Set (MDS) for nursing home (NH) resident assessment, designed to assess elders functional status and care needs, exemplifies how the information needs of clinical practice are congruent with those of research. Building on a review of the published literature, this article describes the development of the MDS, its reliability and validity testing, as well as the variety of different policy and research uses to which it has been applied. Interrater reliability of items and internal consistency of MDS summary scales is generally good to excellent. Validation studies reveal good correspondence to research quality instruments for cognition, activities of daily living, and diagnoses with more variable results for vision, pain, mood, and behavior scales. To date, no consistent evidence suggests that applications of MDS data for case-mix reimbursement and quality indicator monitoring systematically bias the data. Although facility variation in data quality could compromise some applications, creation of the MDS as a clinical tool for care planning provides an example of how assessment tools with clinical use can be used in administrative databases for research and policy applications.

  10. Developing a tool to assess motivation among health service providers working with public health system in India.

    PubMed

    Purohit, Bhaskar; Maneskar, Abhishek; Saxena, Deepak

    2016-04-14

    Addressing the shortage of health service providers (doctors and nurses) in rural health centres remains a huge challenge. The lack of motivation of health service providers to serve in rural areas is one of the major reasons for such shortage. While many studies have aimed at analysing the reasons for low motivation, hardly any studies in India have focused on developing valid and reliable tools to measure motivation among health service providers. Hence, the objective of the study was to test and develop a valid and reliable instrument to assess the motivation of health service providers working with the public health system in India and the extent to which the motivation factors included in the study motivate health service providers to perform better at work. The present study adapted an already developed tool on motivation. The reliability and validity of the tool were established using different methods. The first stage of the tool development involved content development and assessment where, after a detailed literature review, a predeveloped tool with 19 items was adapted. However, in light of the literature review and pilot test, the same tool was modified to suit the local context by adding 7 additional items so that the final modified tool comprised of 26 items. A correlation matrix was applied to check the pattern of relationships among the items. The total sample size for the study was 154 health service providers from one Western state in India. To understand the sampling adequacy, the Kaiser-Meyer-Olkin measure of sampling adequacy and Bartlett's test of sphericity were applied and finally factor analysis was carried out to calculate the eigenvalues and to understand the relative impact of factors affecting motivation. A correlation matrix value of 0.017 was obtained narrating multi-co-linearity among the observations. Based on initial factor analysis, 8 out of 26 study factors were excluded from the study components with a cutoff range of less than

  11. Validation of the Arab Youth Mental Health scale as a screening tool for depression/anxiety in Lebanese children

    PubMed Central

    2011-01-01

    Background Early detection of common mental disorders, such as depression and anxiety, among children and adolescents requires the use of validated, culturally sensitive, and developmentally appropriate screening instruments. The Arab region has a high proportion of youth, yet Arabic-language screening instruments for mental disorders among this age group are virtually absent. Methods We carried out construct and clinical validation on the recently-developed Arab Youth Mental Health (AYMH) scale as a screening tool for depression/anxiety. The scale was administered with 10-14 year old children attending a social service center in Beirut, Lebanon (N = 153). The clinical assessment was conducted by a child and adolescent clinical psychiatrist employing the DSM IV criteria. We tested the scale's sensitivity, specificity, and internal consistency. Results Scale scores were generally significantly associated with how participants responded to standard questions on health, mental health, and happiness, indicating good construct validity. The results revealed that the scale exhibited good internal consistency (Cronbach's alpha = 0.86) and specificity (79%). However, it exhibited moderate sensitivity for girls (71%) and poor sensitivity for boys (50%). Conclusions The AYMH scale is useful as a screening tool for general mental health states and a valid screening instrument for common mental disorders among girls. It is not a valid instrument for detecting depression and anxiety among boys in an Arab culture. PMID:21435213

  12. BASINs and WEPP Climate Assessment Tools (CAT): Case ...

    EPA Pesticide Factsheets

    EPA announced the release of the final report, BASINs and WEPP Climate Assessment Tools (CAT): Case Study Guide to Potential Applications. This report supports application of two recently developed water modeling tools, the Better Assessment Science Integrating point & Non-point Sources (BASINS) and the Water Erosion Prediction Project Climate Assessment Tool (WEPPCAT). The report presents a series of short case studies designed to illustrate the capabilities of these tools for conducting scenario based assessments of the potential effects of climate change on streamflow and water quality. This report presents a series of short, illustrative case studies using the BASINS and WEPP climate assessment tools.

  13. Assessing Predictive Validity of Pressure Ulcer Risk Scales- A Systematic Review and Meta-Analysis

    PubMed Central

    PARK, Seong-Hi; LEE, Hea Shoon

    2016-01-01

    Background: The purpose of this study was to present a scientific reason for pressure ulcer risk scales: Cubbin& Jackson modified Braden, Norton, and Waterlow, as a nursing diagnosis tool by utilizing predictive validity of pressure sores. Methods: Articles published between 1966 and 2013 from periodicals indexed in the Ovid Medline, Embase, CINAHL, KoreaMed, NDSL, and other databases were selected using the key word “pressure ulcer”. QUADAS-II was applied for assessment for internal validity of the diagnostic studies. Selected studies were analyzed using meta-analysis with MetaDisc 1.4. Results: Seventeen diagnostic studies with high methodological quality, involving 5,185 patients, were included. In the results of the meta-analysis, sROC AUC of Braden, Norton, and Waterflow scale was over 0.7, showing moderate predictive validity, but they have limited interpretation due to significant differences between studies. In addition, Waterlow scale is insufficient as a screening tool owing to low sensitivity compared with other scales. Conclusion: The contemporary pressure ulcer risk scale is not suitable for uninform practice on patients under standardized criteria. Therefore, in order to provide more effective nursing care for bedsores, a new or modified pressure ulcer risk scale should be developed upon strength and weaknesses of existing tools. PMID:27114977

  14. Revision and validation of a scale to assess pregnancy stress.

    PubMed

    Chen, Chung-Hey

    2015-03-01

    Pregnancy is a potentially stressful event. Prenatal stress alters maternal endocrine and immune systems, has been implicated in the etiology of prenatal complications or postnatal psychiatric disorders, and may adversely affect fetal health. The 30-item Pregnancy Stress Rating Scale (PSRS), initially developed in 1983 by Chen and colleagues, is the only measure to date designed specifically to evaluate prenatal stress. The purpose of this study was to reconsider and revise the 30-item PSRS and validate the new PSRS. A cross-sectional design was used. Adding new items of pregnancy stress generated from clinical experience and expert recommendations resulted in a 40-item revised PSRS that was more reflective of current social conditions. Three hundred pregnant women, recruited from the antenatal clinic of a medical center in southern Taiwan, completed the revised PSRS to assess its internal consistency, test-retest reliability, construct validity, and convergent and discriminate validity. The final 36-item PSRS (PSRS36) was derived by deleting four items with relatively low item-total correlation coefficients or factor loadings. The resultant 36-item scale showed good internal consistency (α = .92) and 2-week test-retest reliability (r = .82). Factor analysis confirmed construct validity and suggested five prenatal stress dimensions, which explained 52.17% of the total variance. Convergent and discriminate validities were indicated by significant correlations among the PSRS36, Perceived Stress Scale, and Interpersonal Support Evaluation List. The PSRS36 is a psychometrically sound and practical tool for nurses and other healthcare providers to assess prenatal stress and to examine intervention protocols in Taiwanese prenatal women. More research is recommended to determine whether the PSRS36 may be used in other racial-ethnic groups.

  15. Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer.

    PubMed

    Konge, L; Vilmann, P; Clementsen, P; Annema, J T; Ringsted, C

    2012-10-01

    Fine-needle aspiration (FNA) guided by endoscopic ultrasonography (EUS) is important in mediastinal staging of non-small cell lung cancer (NSCLC). Training standards and implementation strategies of this technique are currently under discussion. The aim of this study was to explore the reliability and validity of a newly developed EUS Assessment Tool (EUSAT) designed to measure competence in EUS - FNA for mediastinal staging of NSCLC. A total of 30 patients with proven or suspected NSCLC underwent EUS - FNA for mediastinal staging by three trainees and three experienced physicians. Their performances were assessed prospectively by three experts in EUS under direct observation and again 2 months later in a blinded fashion using digital video-recordings. Based on the assessments, intra-rater reliability, inter-rater reliability, and construct validity were explored. The intra-rater reliability was good (Cronbach's α = 0.80), but comparison of results based on direct observations and blinded video-recordings indicated a significant bias favoring consultants (P = 0.022). Inter-rater reliability was very good (Cronbach's α = 0.93). However, one rater assessing five procedures or two raters each assessing four procedures were necessary to secure a generalizability coefficient of 0.80. The assessment tool demonstrated construct validity by discriminating between trainees and experienced physicians (P = 0.034). Competency in mediastinal staging of NSCLC using EUS and EUS - FNA can be assessed in a reliable and valid way using the EUSAT assessment tool. Measuring and defining competency and training requirements could improve EUS quality and benefit patient care. © Georg Thieme Verlag KG Stuttgart · New York.

  16. Making and Executing Decisions for Safe and Independent Living (MED-SAIL): Development and Validation of a Brief Screening Tool

    PubMed Central

    Mills, Whitney L.; Regev, Tziona; Kunik, Mark E.; Wilson, Nancy L.; Moye, Jennifer; McCullough, Laurence B.; Naik, Aanand D.

    2017-01-01

    Objectives Older adults prefer to remain in their own homes for as long as possible. The purpose of this article is to describe the development and preliminary validation of Making and Executing Decisions for Safe and Independent Living (MED-SAIL), a brief screening tool for capacity to live safely and independently in the community. Design Prospective preliminary validation study. Setting Outpatient geriatrics clinic located in a community-based hospital. Participants Forty-nine community-dwelling older adults referred to the clinic for a comprehensive capacity assessment. Measurements We examined internal consistency, criterion-based validity, concurrent validity, and accuracy of classification for MED-SAIL. Results The items included in MED-SAIL demonstrated internal consistency (5 items; α = 0.85). MED-SAIL was significantly correlated with the Independent Living Scales (r = 0.573, p ≤ 0.001) and instrumental activities of daily living (r = 0.440, p ≤ 0.01). The Mann-Whitney U test revealed significant differences between the no capacity and partial/full capacity classifications on MED-SAIL (U(48) = 60.5, Z = −0.38, p <0.0001). The area under the curve was 0.864 (95% confidence interval: 0.84–0.99). Conclusions This study demonstrated the validity of MED-SAIL as a brief screening tool to identify older adults with impaired capacity for remaining safe and independent in their current living environment. MED-SAIL is useful tool for health and social service providers in the community for the purpose of referral for definitive capacity evaluation. PMID:23567420

  17. Validation of Microtox as a first screening tool for waste classification.

    PubMed

    Weltens, R; Deprez, K; Michiels, L

    2014-12-01

    The Waste Framework Directive (WFD; 2008/98/EG) describes how waste materials are to be classified as hazardous or not. For complex waste materials chemical analyses are often not conclusive and the WFD provides the possibility to assess the hazardous properties by testing on the waste materials directly. As a methodology WFD refers to the protocols described in the CLP regulation (regulation on Classification, Labeling and Packaging of chemicals) but the toxicity tests on mammals are not acceptable for waste materials. The DISCRISET project was initiated to investigate the suitability of alternative toxicity tests that are already in use in pharmaceutical applications, for the toxicological hazard assessment of complex waste materials. Results indicated that Microtox was a good candidate as a first screening test in a tiered approached hazard assessment. This is now further validated in the present study. The toxic responses measured in Microtox were compared to biological responses in other bioassays for both organic and inorganic fractions of the wastes. Both fractions contribute to the toxic load of waste samples. Results show that the Microtox test is indeed a good and practical screening tool for the organic fraction. A screening threshold (ST) of 5 geq/l as the EC50 value in Microtox is proposed as this ST allows to recognize highly toxic samples in the screening test. The data presented here show that the Microtox toxicity response at this ST is not only predictive for acute toxicity in other organisms but also for sub lethal toxic effects of the organic fraction. This limit value has to be further validated. For the inorganic fraction no specific biotest can be recommended as a screening test, but the use of direct toxicity assessment is also preferable for this fraction as metal speciation is an important issue to define the toxic load of elutriate fractions. A battery of 3 tests (Microtox, Daphnia and Algae) for direct toxicity assessment of this

  18. A Serious Game for Clinical Assessment of Cognitive Status: Validation Study.

    PubMed

    Tong, Tiffany; Chignell, Mark; Tierney, Mary C; Lee, Jacques

    2016-05-27

    We propose the use of serious games to screen for abnormal cognitive status in situations where it may be too costly or impractical to use standard cognitive assessments (eg, emergency departments). If validated, serious games in health care could enable broader availability of efficient and engaging cognitive screening. The objective of this work is to demonstrate the feasibility of a game-based cognitive assessment delivered on tablet technology to a clinical sample and to conduct preliminary validation against standard mental status tools commonly used in elderly populations. We carried out a feasibility study in a hospital emergency department to evaluate the use of a serious game by elderly adults (N=146; age: mean 80.59, SD 6.00, range 70-94 years). We correlated game performance against a number of standard assessments, including the Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and the Confusion Assessment Method (CAM). After a series of modifications, the game could be used by a wide range of elderly patients in the emergency department demonstrating its feasibility for use with these users. Of 146 patients, 141 (96.6%) consented to participate and played our serious game. Refusals to play the game were typically due to concerns of family members rather than unwillingness of the patient to play the game. Performance on the serious game correlated significantly with the MoCA (r=-.339, P <.001) and MMSE (r=-.558, P <.001), and correlated (point-biserial correlation) with the CAM (r=.565, P <.001) and with other cognitive assessments. This research demonstrates the feasibility of using serious games in a clinical setting. Further research is required to demonstrate the validity and reliability of game-based assessments for clinical decision making.

  19. A Serious Game for Clinical Assessment of Cognitive Status: Validation Study

    PubMed Central

    Chignell, Mark; Tierney, Mary C.; Lee, Jacques

    2016-01-01

    Background We propose the use of serious games to screen for abnormal cognitive status in situations where it may be too costly or impractical to use standard cognitive assessments (eg, emergency departments). If validated, serious games in health care could enable broader availability of efficient and engaging cognitive screening. Objective The objective of this work is to demonstrate the feasibility of a game-based cognitive assessment delivered on tablet technology to a clinical sample and to conduct preliminary validation against standard mental status tools commonly used in elderly populations. Methods We carried out a feasibility study in a hospital emergency department to evaluate the use of a serious game by elderly adults (N=146; age: mean 80.59, SD 6.00, range 70-94 years). We correlated game performance against a number of standard assessments, including the Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and the Confusion Assessment Method (CAM). Results After a series of modifications, the game could be used by a wide range of elderly patients in the emergency department demonstrating its feasibility for use with these users. Of 146 patients, 141 (96.6%) consented to participate and played our serious game. Refusals to play the game were typically due to concerns of family members rather than unwillingness of the patient to play the game. Performance on the serious game correlated significantly with the MoCA (r=–.339, P <.001) and MMSE (r=–.558, P <.001), and correlated (point-biserial correlation) with the CAM (r=.565, P <.001) and with other cognitive assessments. Conclusions This research demonstrates the feasibility of using serious games in a clinical setting. Further research is required to demonstrate the validity and reliability of game-based assessments for clinical decision making. PMID:27234145

  20. Validity and reliability of the Turkish version of the Optimality Index-US (OI-US) to assess maternity care outcomes.

    PubMed

    Yucel, Cigdem; Taskin, Lale; Low, Lisa Kane

    2015-12-01

    Although obstetrical interventions are used commonly in Turkey, there is no standardized evidence-based assessment tool to evaluate maternity care outcomes. The Optimality Index-US (OI-US) is an evidence-based tool that was developed for the purpose of measuring aggregate perinatal care processes and outcomes against an optimal or best possible standard. This index has been validated and used in Netherlands, USA and UK until now. The objective of this study was to adapt the OI-US to assess maternity care outcomes in Turkey. Translation and back translation were used to develop the Optimality Index-Turkey (OI-TR) version. To evaluate the content validity of the OI-TR, an expert panel group (n=10) reviewed the items and evidence-based quality of the OI-TR for application in Turkey. Following the content validity process, the OI-TR was used to assess 150 healthy and 150 high-risk pregnant women who gave birth at a high volume, urban maternity hospital in Turkey. The scores between the two groups were compared to assess the discriminant validity of the OI-TR. The percentage of agreement between two raters and the Kappa statistic were calculated to evaluate the reliability. Content validity was established for the OI-TR by an expert group. Discriminant validity was confirmed by comparing the OI scores of healthy pregnant women (mean OI score=77.65%) and those of high-risk pregnant women (mean OI score=78.60%). The percentage of agreement between the two raters was 96.19, and inter-rater agreement was provided for each item in the OI-TR. OI-TR is a valid and reliable tool that can be used to assess maternity care outcomes in Turkey. The results of this study indicate that although the risk statuses of the women differed, the type of care they received was essentially the same, as measured by the OI-TR. Care was not individualised based on risk and for a majority of items was inconsistent with evidence based practice, which is not optimal. Use of the OI-TR will help to